JP2006180251A - Voice signal processor for enabling callers to perform simultaneous utterance, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means for enabling multiple callers to perform utterance and to discriminate the voice of the different callers concerning a system for synthesizing the voice of the plurality of callers and issuing it. <P>SOLUTION: A voice signal processing server 13 receives the voice signal of a participant 19 from each terminal apparatus 11. A cumulative time data generator 132 generates cumulative time data which indicates the accumulation of an utterance time indicated by each voice signal. A terminal apparatus selector 133 selects a prescribed number of terminal apparatuses 11 of the participants 19 with the long utterance time, based on the cumulative time data. A voice and image position allocating part 135 allocates the respectively different voice and image positions to the selected terminal apparatuses 11. A voice signal working part 136 applies processing so as to allow the voice, and image position of each voice signal to coincide with the allocated voice and image position. The voice signal with the voice and image position adjusted therein is mixed and transmitted to the terminal apparatus 11. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an audio signal processing technique that enables simultaneous speech by a plurality of speakers using an acoustic device in a meeting or the like.

In an audio conference in which speakers arranged at multiple points participate, there is a technique for differentiating the sound image localization of each speaker's voice so that the voice of each speaker can be distinguished by the listener. In such a technique, the sound signal input from the microphones arranged in the vicinity of each speaker is separated into two left and right channel sound signals, and then the sound pressure adjustment is performed on the sound signals of each channel. The sound image position of the sound indicated by the sound signal is changed. At this time, as a result of performing different sound pressure adjustment for each speaker, the sound for each speaker is arranged at a different sound image position. Thereafter, the sound signals for each speaker whose sound pressure has been adjusted are added and output to a speaker arranged in the vicinity of the listener. As a result, the sound image position of each voice is different even when the voices of a plurality of speakers are simultaneously emitted from the speaker, so that the listener can distinguish the voices of the speakers. As a document disclosing such a prior art, there is, for example, Patent Document 1.
Japanese Patent Laid-Open No. 11-127499

  In the case of the prior art, a sound image position corresponding to the number of speakers is provided. Therefore, as the number of speakers increases, the distance between adjacent sound image positions becomes shorter, and it becomes difficult for the listener to distinguish between the sounds of different speakers arranged at adjacent sound image positions.

  In view of the above situation, the present invention provides a means for enabling a speaker to distinguish different speakers' voices while enabling speech by a large number of speakers in a system that synthesizes and generates voices of multiple speakers. The purpose is to do.

  To achieve the above object, the present invention provides an input means for receiving audio signals from a plurality of terminal devices, and a continuation of an audio signal received from each of the plurality of terminal devices by the input means within a past predetermined period. Cumulative time data generating means for generating cumulative time data indicating a cumulative value of time; selection means for selecting a predetermined number of terminal devices from the plurality of terminal devices based on the cumulative time data; and the selection Allocating means for assigning different sound image positions to each of the plurality of terminal devices selected by the means, and when the sound signal is received by the input means from the plurality of terminal devices selected by the selecting means, The signal is processed into an audio signal having a sound image position assigned to the terminal device by the assigning unit, and is not selected by the selecting unit. When an audio signal is received from the terminal device by the input means, processing means for processing the audio signal into an audio signal having a predetermined sound image position, and an output for outputting the audio signal processed by the processing means And an audio signal processing apparatus.

  According to the audio signal processing apparatus having such a configuration, since the audio signals of different speakers are dynamically arranged at different predetermined numbers of sound image positions, the number of speakers is large, and the terminal device that transmits the audio signal Even when the number is large, the interval between adjacent sound image positions is kept at a certain level or more. As a result, the inconvenience that the voices of different speakers included in the voice obtained by reproducing the generated synthesized voice signal are confused by the listener is prevented.

  Also, in a preferred aspect, the assigning means changes the one sound image position from the one sound image position to the other sound image position when changing the one sound image position assigned to the one terminal device to another sound image position. A sound image position that changes over time may be assigned to the sound image position.

  According to the audio signal processing device having such a configuration, even when a change is made to the sound image position assigned to the terminal device, a synthesized audio signal indicating a sound whose sound image position moves smoothly is generated. It becomes easy for the listener of the voice obtained by reproduction to capture the voice of the same speaker.

  In another preferable aspect, the audio signal processing device generates frequency characteristic data indicating a frequency characteristic of an audio signal received from the terminal device for each of the plurality of terminal devices selected by the selection unit. Frequency characteristic specifying means, and the assigning means selects the selecting means so that the similarity between the frequency characteristic data of the audio signals received from the terminal device to which any two sound image positions are assigned satisfies a predetermined condition. The sound image position may be assigned to each of the plurality of terminal devices selected by the above.

  According to the audio signal processing apparatus having such a configuration, when the frequency characteristics of the audio signals of different speakers are close to each other, the audio listener obtained by reproducing the synthesized audio signal can be obtained by separating the sound image positions by a certain distance or more. Therefore, it is possible to more reliably avoid the inconvenience of confusing the voices of different speakers.

  Furthermore, the present invention shows an input unit that receives audio signals from a plurality of terminal devices, and a cumulative value of durations of audio signals received from each of the plurality of terminal devices by the input unit within a predetermined period in the past. Cumulative time data generating means for generating cumulative time data, selection means for selecting a predetermined number of terminal devices from among the plurality of terminal devices based on the cumulative time data, and a plurality selected by the selection means Allocating means for assigning different frequency characteristics to each of the terminal devices, and when the voice signal is received by the input means from a plurality of terminal devices selected by the selecting means, the allocating means The terminal device that has been processed into an audio signal having the frequency characteristics assigned to the terminal device by the selection device and has not been selected by the selection means And a processing means for processing the audio signal into an audio signal having a predetermined frequency characteristic when an audio signal is received by the input means, and an output means for outputting the audio signal processed by the processing means. An audio signal processing apparatus is provided.

  According to the speech signal processing apparatus having such a configuration, processing is performed so that speech signals of different speakers have different frequency characteristics, and a synthesized speech signal that can distinguish speeches of different speakers is generated at the same time. Since the number of frequency characteristics is limited to a predetermined number and these are dynamically assigned to different speaker voice signals, the number of speakers is large and the number of terminal devices that transmit voice signals is large. Even in this case, a certain difference or more is maintained between the frequency characteristics of the processed audio signal. As a result, the inconvenience that the voices of different speakers included in the voice obtained by reproducing the generated synthesized voice signal are confused by the listener is prevented.

  Further, in a preferred aspect, when the allocating unit changes one frequency characteristic allocated to one terminal device to another frequency characteristic, the allocating unit changes the one frequency characteristic from the one frequency characteristic to the other frequency characteristic. A frequency characteristic that changes with the passage of time may be assigned to the frequency characteristic.

  According to the audio signal processing device having such a configuration, even when a change is made to the frequency characteristic assigned to the terminal device, a synthesized audio signal indicating a voice whose frequency characteristic changes smoothly is generated. It becomes easy for the listener of the voice obtained by reproduction to capture the voice of the same speaker.

  The present invention also provides a program that causes a computer to execute processing performed by the above-described audio signal processing apparatus.

[1. First Embodiment]
[1.1. Configuration of audio conference system]
FIG. 1 is a block diagram showing a configuration of an audio conference system 1 according to the first embodiment of the present invention. The audio conference system 1 is a system that enables conference participants in different places to hold a conference by audio. The audio conference system 1 includes a network 10 for connecting a plurality of communication devices to each other, a plurality of terminal devices 11 connected to the network 10, a headset 12 connected to each of the terminal devices 11, and a network 10. A connected audio signal processing server 13 is provided.

  Each of the plurality of terminal devices 11 and the headset 12 is used by each of the conference participants 19. The number of participants who can participate in the conference using the audio conference system 1, that is, the number of the terminal devices 11 and the headsets 12 can be arbitrarily changed, and even if the configuration of the participants fluctuates during the conference, Good.

  As shown in FIG. 1, when different participants 19 and the terminal device 11 and the headset 12 used by the participant 19 need to be distinguished from each other, the participant 19-n and the terminal device 11-n, respectively. And, like the headset 12-n, “-n” is added to the end to distinguish them. However, “n” is an arbitrary natural number. In addition, when there is no need to distinguish different participants 19 and the terminal devices 11 and headsets 12 used by the participants 19, they are simply referred to as participants 19, terminal devices 11 and headsets 12, respectively.

  The network 10 includes one or more relay devices interconnected by wire or wireless, and relays data between different communication devices. The network 10 may be an open network that does not limit users such as the Internet, or may be any of an intranet, a LAN (Local Area Network) using a communication protocol other than the Internet protocol, and the like.

  The terminal device 11 transmits a voice signal indicating the voice of the participant 19 to the voice signal processing server 13 and a synthesized voice signal obtained by synthesizing a voice signal indicating the voice of the other participant 19 from the voice signal processing server 13. For example, a general-purpose personal computer, a PDA (Personal Digital Assistant), or a dedicated terminal device may be used. The terminal device 11 converts an analog audio signal input from the headset 12 into a digital audio signal, converts the digital audio signal into an analog audio signal, and outputs the analog audio signal to the headset 12, and an audio signal processing An audio signal transmitting unit 112 that receives an audio signal from the unit 111 and sends it to the network 10, an audio signal receiving unit 113 that receives the audio signal from the network 10 and writes it to the storage unit 114, and a control program for the terminal device 11 are stored. A storage unit 114 is also used as a work area for other components.

  The headset 12 includes a microphone that generates an analog audio signal indicating the voice of the participant 19 and outputs the analog audio signal to the terminal device 11, and a headphone that converts the analog audio signal input from the terminal device 11 into sound and generates a sound. Yes. The headphones of the headset 12 are stereo headphones that have a left input and a right input and can produce different sounds on the left and right.

  The audio signal processing server 13 receives an audio signal from each of the plurality of terminal devices 11, performs sound image position adjustment and mixing processing on the received audio signal, generates a synthesized audio signal, and generates the generated synthesized audio signal Is transmitted to each of the plurality of terminal devices 11. The audio signal processing server 13 receives the audio signal receiving unit 131 that receives the audio signal from the network 10 and the terminal device 11 that is the transmission source of the accumulated value of the audio duration indicated by the audio signal received within the past predetermined time. An accumulated time data generating unit 132 for generating accumulated time data shown for each of the above, a terminal device selecting unit 133 for selecting a predetermined number of terminal devices from which audio signals to be arranged in the sound space are based on the accumulated time data, and the like The frequency characteristic specifying unit 134 that specifies the frequency characteristic of the audio signal received from each terminal device 11 and generates frequency characteristic data, and the audio signal is transmitted to the terminal device 11 that is the source of the audio signal in the sound space. A sound image position assignment unit 135 for assigning a sound image position to be arranged, and a sound image position assignment unit 135 for assigning each sound image position of the audio signal. An audio signal processing unit 136 for processing an audio signal to generate an adjusted audio signal so as to obtain an adjusted sound image position, a mixing unit 137 for generating a synthesized audio signal by mixing the adjusted audio signal, and a synthesized audio signal Are transmitted to the network 10 and a storage unit 139 that stores a control program of the audio signal processing server 13 and the like and is used as a work area for other components. The storage unit 139 stores condition data indicating conditions for the terminal device selection unit 133 to select a predetermined number of terminal devices and conditions for the sound image position assignment unit 135 to assign sound image positions to the terminal devices 11. A condition database 1391 is stored.

  In a conference in which a large number of participants 19 participate, if the sound image position assignment unit 135 assigns different sound image positions to all the participants 19 in a fixed manner, the distance from the adjacent sound image positions is shortened. As a result, close sound image positions are assigned to similar sounds, and if the sound is simultaneously spoken by those sounds, the mixed sound is discriminated by the listener even if the sound image position is adjusted for those sounds. It will be difficult. Therefore, the terminal device selection unit 133 limits the number of sound image positions by selecting a predetermined number of sounds to which sound image position allocation should be performed by the sound image position allocation unit 135 from time to time. That is, the sound image position assignment unit 135 assigns the sound image position only to the terminal device 11 selected by the terminal device selection unit 133. As a result, the distance between adjacent sound image positions does not become narrower than a certain value, and the above problem does not occur.

  By the way, the selection of the terminal device 11 by the terminal device selection unit 133 and the assignment of the sound image position by the sound image position assignment unit 135 are performed according to the condition data included in the condition database 1391. Each of the terminal device selection unit 133 and the sound image position assignment unit 135 uses the accumulated time data generated by the accumulated time data generation unit 132 in the selection or assignment process. The accumulated time data is data indicating the accumulated value of the speech time of each participant 19 within the past predetermined time, and is constantly changing with the passage of time. Therefore, the terminal device 11 selected by the terminal device selection unit 133 and the sound image position assigned to the terminal device 11 by the sound image position assignment unit 135 dynamically change.

  In addition, the sound image position assignment unit 135 can use the frequency characteristic data generated by the frequency characteristic specifying unit 134 when assigning the sound image position. The frequency characteristic data is data indicating the voice characteristics of each participant 19.

  The audio signal received by the audio signal receiving unit 131 is adjusted by the audio signal processing unit 136 so as to have a sound image position assigned by the sound image position assigning unit 135 to the terminal device 11 that is the source, and then the mixing unit 137. And is transmitted to each terminal apparatus 11 by the audio signal transmission unit 138. The above is the outline of the operation of the audio signal processing server 13.

  Subsequently, the condition database 1391 will be described. FIG. 2 is a diagram illustrating the contents of the condition database 1391. The condition database 1391 includes a plurality of records (each data row in FIG. 2), and each record indicates the condition data and which is prioritized when the contents of the condition data conflict with the contents of other condition data. Includes priority. The priority is an integer value, and the smaller the number, the higher the priority. Hereinafter, the condition data illustrated in FIG. 2 will be described. However, the types of condition data are not limited to those described below. First, the condition data “number of positions: 6” indicates that the number of sound image positions assigned to the sound by the sound image position assignment unit 135 is six. Hereinafter, the sound image positions assigned to the sound by the sound image position assigning unit 135 are referred to as “position 1”, “position 2”,. These sound image positions are specified by the angle of inclination in the left direction (negative) or right direction (positive) with the front of the listener as the reference (0 degree). For example, the positions 1 to 6 are specified as “−50 degrees”, “−30 degrees”, “−10 degrees”, “+10 degrees”, “+30 degrees”, and “+50 degrees”, respectively.

  The condition data “monitoring period: 20 minutes” uses the audio signal received by the audio signal receiving unit 131 for the past 20 minutes for the cumulative time data generating unit 132 and the frequency characteristic specifying unit 134. Is instructing. In this case, the accumulated time data generation unit 132 calculates the duration of the audio signal received from each terminal device 11 based on the audio signal received in the past 20 minutes, thereby obtaining the accumulated value of the speech time of each participant 19. The frequency characteristic specifying unit 134 generates frequency characteristic data indicating the frequency characteristic of each voice signal of the participant 19 by performing frequency analysis on the voice signal received in the past 20 minutes. It will be.

  The condition data “movement speed: 20 seconds” indicates that the movement of the sound image position is completed in 20 seconds when the sound image position assignment unit 135 changes the sound image position assigned to the sound signal received from a certain terminal device 11. Instructed to gradually change the sound image position. For example, condition data such as condition data “movement time: 1 degree / second” may be used as condition data for giving a similar instruction. When changing the sound image position to the sound image position assigning unit 135, this condition data is gradually moved from the previously assigned sound image position to the sound image position to be newly assigned at an angular velocity of once per second. An instruction is given to change the position of the sound image.

  The condition data “permanent holding position: position 4” is the one in which the audio signal received by the audio signal receiving unit 131 is transmitted from the terminal device 11 other than the terminal device 11 selected by the terminal device selecting unit 133 at that time. In this case, it is instructed to place the sound indicated by the sound signal at position 4. Unlike the case of the temporary holding position described below, the permanent holding position does not include the voice indicated by the voice signal transmitted from the terminal device 11 selected by the terminal device selection unit 133.

  The condition data “temporary holding position: position 3” is transmitted from the terminal device 11 other than the terminal device 11 selected by the terminal device selection unit 133 at the time when the audio signal received by the audio signal receiving unit 131 is received. In addition, for the position 4 indicated by the condition data “permanent holding position: position 4”, it is instructed to individually arrange the audio indicated by the audio signal at the position 3. That is, the permanent holding position is a position that is reserved for the audio signal transmitted from the terminal device 11 that is originally selected by the terminal device selection unit 133, whereas the temporary holding position is temporarily different. This position is permitted to be used for an audio signal transmitted from the terminal device 11.

  The condition data “cumulative time ranking 1: position 6” is based on the cumulative time data generated by the cumulative time data generation unit 132 with respect to the terminal device 11 of the participant 19 whose speech time is the longest in the past predetermined time. , Position 6 is instructed. In this case, the past predetermined time is 20 minutes indicated by the condition data “monitoring period: 20 minutes”.

The condition data “frequency characteristic similarity> 0.90 → interval 2 or greater” is a distance of 2 or more apart from speech whose frequency characteristics indicated by the frequency characteristic data generated by the frequency characteristic identification unit 134 are similar to each other. It is instructed to assign the sound image position. Here, the interval is a number obtained by adding 1 to the number of other positions included between two positions. For example, the interval between position 1 and position 4 is “3”. The frequency characteristic similarity is an index indicating the similarity between the frequency characteristics of two audio signals, and is calculated by the sound image position assignment unit 135 based on the frequency characteristic data. Various indexes for indicating the similarity of the frequency characteristics can be considered. In the following description, as an example, the degree of difference between the graphs indicating the frequency characteristics of two audio signals is used. More specifically, it is assumed that the sound image position assignment unit 135 calculates the frequency characteristic similarity as follows.
(A) A graph showing the frequency characteristic of each audio signal is drawn from the frequency characteristic data. At that time, the vertical axis represents the sound pressure level (dB), and the horizontal axis represents the logarithm of frequency (Hz).
(B) The sound pressure level is increased or decreased over the entire frequency band so that the area of the figure surrounded by the graph drawn in (a) and the horizontal axis becomes the predetermined value S. That is, the graph is moved up and down in the vertical axis direction. This means that the volume of the audio indicated by the audio signal is raised and lowered to match the overall volume between the different sounds.
(C) For the two audio signals, the area s of the overlapping portion between the graphs surrounded by the graphs and the horizontal axis is calculated when the graphs whose volume adjustment is performed in (b) are superimposed.
(D) R = s / S is calculated from S used in (b) and s calculated in (c), and this R is defined as the frequency characteristic similarity for the two audio signals.

  R calculated as described above takes a value from 0 to 1, and the larger the value, the more similar the frequency characteristics of the two audio signals. In accordance with the condition data “frequency characteristic similarity> 0.90 → interval 3 or greater”, if the frequency characteristic similarity of two audio signals is greater than 0.90, the audio indicated by those audio signals has an interval of 3 or greater. The sound image position is assigned to the terminal device 11 so as to be spaced apart.

  The condition data “terminal ID“ 0123 ”: position 1” indicates that position 1 is assigned to the terminal device 11 whose terminal ID is “0123”, which is an identifier in the network 10 of the terminal device 11. That is, the position 1 is reserved for the participant 19 who is a user of the specific terminal device 11, and the position 1 is not assigned to the voice of the other participant 19 due to the speaking time of the participant 19. . Such condition data is used when it is convenient for all listeners to always place the sound at the same sound image position, although the speaking time is not necessarily long, for example, as a conference chairperson. It is done.

  Condition data “default position: in order of terminal ID” indicates to the sound image position assignment unit 135 that the sound image position is assigned to the sound indicated by each sound signal according to the size of the terminal ID of the terminal device 11 that is the transmission source of the sound signal. I am instructing. The priority associated with this condition data is “9”, which is lower than the other condition data. That is, this condition data indicates a condition used when the sound image position is not uniquely determined by other condition data. For example, since the accumulated time data has not yet been generated immediately after the start of the conference, the voice position of the first speaker cannot be determined by the condition data such as the condition data “cumulative time ranking 1: position 6”. In such a case, according to the condition data “default position: terminal ID order”, the voices are arranged in the order of the terminal ID of the terminal device 11 used by the speaking participant 19 in the order of position 1, position 2,. Will be.

[1.2. Operation of the audio conference system]
Next, the operation of the audio conference system 1 when a plurality of participants 19 hold a conference using the audio conference system 1 will be described. First, each participant 19 participating in the conference operates his / her own terminal device 11 to establish a communication connection between the terminal device 11 and the audio signal processing server 13. The audio signal processing server 13 establishes a communication connection with each of the plurality of terminal apparatuses 11, and the plurality of communication connections are identified by a connection ID assigned by the audio signal processing server 13. Since the method for establishing the communication connection is the same as that according to the prior art, the description thereof is omitted. Since each of the terminal devices 11 can establish a communication connection with the audio signal processing server 13 at an arbitrary timing, the participant 19 joins or leaves the conference at an arbitrary timing. be able to.

  When the communication connection is established between the terminal device 11 and the audio signal processing server 13, the storage unit 114 of the terminal device 11 stores the audio signal received from the audio signal processing server 13 for a predetermined time (for example, 5 A data buffer 1141 for temporary storage is secured by FIFO (First-in First-out) only for seconds). However, as will be described later, since the audio signal received by the terminal device 11 from the audio signal processing server 13 is the left and right channels, the data buffer 1141 can store the audio signals of the left and right channels for a predetermined time. Its storage capacity is determined.

  Similarly, the storage unit 139 of the audio signal processing server 13 stores the audio signal received from the terminal device 11 for each established communication connection in the condition data “monitoring period: included in the condition database 1391 (see FIG. 2). A data buffer 1392 for temporarily storing data by the FIFO is secured for the time indicated by “20 minutes” (that is, for 20 minutes). These data buffers are identified by the connection ID. In the audio signal processing server 13, audio signals received from different terminal apparatuses 11 are sequentially stored in different data buffers 1392. Since the audio signal received by the audio signal processing server 13 from the terminal device 11 is monaural, the storage capacity of the data buffer 1392 is different from that of the data buffer 1141, so that one channel of audio signal can be stored for 20 minutes. Storage capacity is determined.

  Also, the audio signal processing server 13 assigns a connection ID to each terminal device 11, and simultaneously obtains a terminal ID from the terminal device 11 to which the connection ID is assigned, and creates a correspondence table between the assigned connection ID and the terminal ID. To do. FIG. 3 shows an example of a correspondence table of connection IDs and terminal IDs created in the audio signal processing server 13. The audio signal processing server 13 can specify the terminal device 11 of the communication partner using the communication connection specified by the connection ID by the connection ID according to the correspondence table. For example, when the terminal ID “0123” is associated with the connection ID “0023”, the data is transmitted from the terminal device 11 identified by the terminal ID “0123” to the data buffer 1392 identified by the connection ID “0023”. The recorded audio signal is stored.

  As described above, when a communication connection is established between the terminal device 11 and the audio signal processing server 13, the participant 19 can speak using the headset 12 connected to the terminal device 11. The sound produced by the participant 19 is converted into an analog sound signal by the microphone of the headset 12 and input to the sound signal processing unit 111 of the terminal device 11. The audio signal processing unit 111 converts the analog audio signal received from the headset 12 into a digital audio signal, and then generates a data packet including the converted audio signal.

  FIG. 4 is a diagram schematically illustrating how an audio signal is included in a plurality of data packets. The audio signal processing unit 111 divides the audio signal into audio signal blocks having a predetermined data length in order from the top. The audio signal processing unit 111 adds a block number indicating the order between the audio signal blocks to the front of the audio signal block. Furthermore, the audio signal processing unit 111 sequentially adds a connection ID, a transmission source ID, and a transmission destination ID before the block number. The transmission source ID is a terminal ID of the terminal device 11, and the transmission destination ID is an ID in the network 10 of the audio signal processing server 13 (hereinafter referred to as “server ID”). Further, the audio signal processing unit 111 adds HOD (Head of Data) and EOD (End of Data) as data indicating a delimiter of a series of data before and after the audio signal block to which the block number and the like are added. A series of data starting with the HOD generated in this way and ending with EOD is a data packet.

  The audio signal processing unit 111 sequentially transfers the data packets generated as described above to the audio signal transmission unit 112, and the audio signal transmission unit 112 sequentially transmits the received data packets to the network 10. The relay device included in the network 10 stores a routing table indicating a communication path that can reach the communication device specified in the network 10 by the transmission destination ID, and is included in the data packet transmitted from the terminal device 11. Based on the destination ID, the data packet is transferred to an adjacent relay device on the communication path that can reach the communication device specified by the destination ID according to the routing table. As a result, the data packet is delivered to the audio signal processing server 13. Since the routing table update method and the like are the same as those in the prior art, the description thereof is omitted.

  The data packet sent to the audio signal processing server 13 as described above via the network 10 is received by the audio signal receiving unit 131 of the audio signal processing server 13. The audio signal receiving unit 131 is included in the data packet in an area corresponding to the block number included in the data packet in the data buffer 1392 identified by the connection ID according to the connection ID included in the received data packet. The audio signal block to be recorded is stored. The plurality of data packets transmitted from the terminal device 11 are not necessarily received by the audio signal processing server 13 in the order of transmission as a result of the different communication paths passing through the network 10. However, as a result of the audio signal receiving unit 131 storing the audio signal blocks in the data buffer 1392 in the order corresponding to the block numbers, the series of audio signals stored in the data buffer 1392 is divided into data packets in the terminal device 11. It is a reproduction of the audio signal before being played. When a part of the data packet does not reach the audio signal processing server 13 for some reason, the audio signal included in the data packet that the audio signal processing server 13 did not reach is interpolated based on the preceding and following audio signals, etc. However, since these processes are the same as those according to the prior art, description thereof will be omitted.

  As described above, the audio signal receiving unit 131 stores the audio signal received from each of the plurality of terminal devices 11 in the data buffer 1392 identified by the corresponding connection ID for the past predetermined period. However, the sound signal stored in each data buffer 1392 includes a sound when the participant 19 is not speaking, that is, a signal indicating almost silence. Therefore, the accumulated time data generation unit 132 extracts, from the audio signal stored in each data buffer 1392 at predetermined time intervals, a portion where the absolute value of the amplitude exceeds a predetermined threshold, and the time of the portion of the extracted audio signal Cumulative time data indicating is generated. Hereinafter, as an example, it is assumed that the accumulated time data generation unit 132 generates accumulated time data at intervals of 5 seconds. The accumulated time data is associated with a connection ID for identifying the data buffer 1392 in which the audio signal used for the generation is stored. That is, the accumulated time data indicates the accumulated speech time within a predetermined past time by the participant 19 of the terminal device 11 identified by the associated connection ID.

  When the cumulative time data generation unit 132 extracts a part indicating the speech of the participant 19 from the audio signal stored in the data buffer 1392, for example, after the amplitude of the audio signal exceeds a predetermined threshold, a predetermined time (for example, Even if the amplitude is equal to or less than a predetermined threshold for 2 seconds, it may be considered that the speech continues. Then, even when there is a brief pause between words in the speech by the participant 19, those series of speech times are reflected in the accumulated time data.

  Further, the terminal device 11 is prevented from transmitting an audio signal during a period when the participant 19 does not speak to the audio signal processing server 13, and the accumulated time data generation unit 132 is indicated by the audio signal received by the terminal device 11. The sound production time may be simply accumulated. In that case, in the audio signal transmission unit 112 of the terminal device 11 that is the transmission source of the audio signal, only a portion of the audio signal whose amplitude exceeds a predetermined threshold may be transmitted to the network 10 as a data packet. Data traffic is reduced.

  When the accumulated time data generating unit 132 generates accumulated time data at intervals of 5 seconds as described above, the accumulated time data generating unit 132 delivers the generated accumulated time data to the terminal device selecting unit 133 and the sound image position allocating unit 135.

  Upon receiving the accumulated time data from the accumulated time data generating unit 132, the terminal device selecting unit 133 performs a plurality of participations according to the condition data included in the condition database 1391 stored in the storage unit 139 based on the received accumulated time data. A predetermined number is selected from the terminal device 11 used by the person 19 in descending order of the accumulated speech time of the participant 19 in the past predetermined period. According to the condition database 1391 illustrated in FIG. 2, the total number of sound image positions is 6, but the position 4 is reserved as a permanent reservation position, and the position 1 is secured in the terminal device 11 with the terminal ID “0123”. Therefore, the terminal device selection unit 133 selects the four terminal devices 11 in order from the terminal device 11 of the participant 19 with the long accumulated time indicated by the accumulated time data. However, when the four terminal devices 11 selected as described above include those having the terminal ID “0123”, since the terminal device 11 has already been secured in position 1, the terminal device The selection unit 133 further selects one terminal device 11.

  More specifically, the selection processing of the terminal device 11 by the terminal device selection unit 133 is performed by selecting the upper one when the accumulated time data is arranged in the order of the accumulated time indicated by the accumulated time data and selecting the accumulated time. This is done by selecting a connection ID associated with the data. The terminal device selection unit 133 arranges the connection IDs selected as described above, for example, in the order of the corresponding accumulated time, and passes them to the frequency characteristic specifying unit 134 and the sound image position assignment unit 135 as selection data. Since the terminal device selection unit 133 receives the accumulated time data at intervals of 5 seconds, the terminal device selection unit 133 also delivers the selection data to the frequency characteristic specifying unit 134 and the sound image position assignment unit 135 at intervals of 5 seconds.

  When the frequency characteristic specifying unit 134 receives the selection data from the terminal device selection unit 133, the frequency characteristic specifying unit 134 performs frequency analysis on the audio signal stored in the associated data buffer 1392 for each connection ID included in the selection data, Frequency characteristic data indicating the relationship between each frequency and the sound pressure level of the frequency included in the audio signal is generated. The frequency characteristic specifying unit 134 passes the generated frequency characteristic data to the sound image position assignment unit 135. The frequency characteristic specifying unit 134 also delivers frequency characteristic data to the sound image position allocating unit 135 at intervals of 5 seconds.

  When receiving the selection data from the terminal device selection unit 133 and the frequency characteristic data corresponding to each connection ID included in the selection data from the frequency characteristic specifying unit 134, the sound image position allocation unit 135 receives the condition data based on the received data. In accordance with the condition data included in the database 1391, sound image position assignment processing for the terminal device 11 is performed as follows.

  For example, when the selection data is “0034, 0015, 0004, 0009” and the terminal ID (see FIG. 3) associated with the connection ID included in the selection data is “0278, 0301, 0041, 0084”. The sound image position assigning unit 135 assigns position 6 to the connection ID “0034” corresponding to the terminal ID “0278” that is ranked first in the cumulative time according to the condition data “cumulative time ranking 1: position 6”. Similarly, position 2 is assigned to connection ID “0015”, position 5 is assigned to connection ID “0004”, and position 3 is assigned to connection ID “0009”.

  FIG. 5 is a diagram schematically showing which participant 19's voice is placed in which position as a result of the positions assigned to the connection IDs by the sound image position assigning unit 135 as described above. In FIG. 5, the voice of the participant 19 who uses the terminal device 11 with the terminal ID “0123” is fixedly arranged at position 1. In addition, the voices of the participants 19 who use the terminal devices 11 whose terminal IDs are “0301”, “0084”, “0041”, and “0278” are arranged at positions 2, 3, 5, and 6, respectively. However, these arrangements are arrangements temporarily determined based on the cumulative value of the speaking time of the participant 19, and these arrangements change dynamically. In addition, when a speech is made by a participant 19 who uses one or a plurality of terminal devices 11 that are not assigned to any of positions 1 to 3, 5, or 6 at position 4, the voice of the participant 19 is recorded. Is placed. Furthermore, in the case where position 3 is simultaneously spoken by two participants 19 using different terminal devices 11 that are not assigned to any of positions 1 to 3, 5 or 6, for example, Among them, the voice of the participant 19 who started speaking later is temporarily arranged.

  Subsequently, the sound image position allocating unit 135 makes necessary corrections to the positions allocated to the respective connection IDs as described above in accordance with the condition data “frequency characteristic similarity> 0.90 → interval 2 or more”. Specifically, the sound image position assigning unit 135 sequentially selects two of the frequency characteristic data corresponding to the connection ID assigned to each position as described above, and uses the frequency characteristic data according to the method already described from the frequency characteristic data. Calculate similarity. Then, the sound image position assignment unit 135 changes the position assignment so as to satisfy the condition when the assignment of the position that violates the condition indicated by the condition data “frequency characteristic similarity> 0.90 → interval 2 or more” is performed. I do.

  For example, the frequency characteristic similarity calculated from the frequency characteristic data corresponding to the connection ID “0015” assigned position 2 and the frequency characteristic data corresponding to the connection ID “0009” assigned position 3 is 0.95. In this case, for example, the sound image position assigning unit 135 assigns position 5 to the connection ID “0009” and assigns position 3 to the connection ID “0004” to which the position 5 has been assigned. Is not to be more similar than a certain level. However, in the case where the condition data “frequency characteristic similarity> 0.90 → interval 2 or more” cannot be satisfied regardless of how the positions are assigned to the plurality of connection IDs, the sound image position assigning unit 135, for example, Positions are assigned according to a predetermined rule, such as selecting an arrangement that minimizes the frequency characteristic similarity between audio signals corresponding to matching connection IDs.

  When the sound image position assignment unit 135 assigns a position to each connection ID as described above, the sound image position assignment unit 135 generates sound image position assignment data indicating the connection ID assigned to each position. For example, when the position allocation illustrated in FIG. 5 is performed, the sound image position allocation data generated by the sound image position allocation unit 135 is [0023: −50 degrees, 0015: −30 degrees, 0009 (temporary hold): −10 degrees, (permanent hold): +10 degrees, 0004: +30 degrees, 0034: +50 degrees]. The sound image position assignment unit 135 generates sound image position assignment data and passes it to the sound signal processing unit 136.

  The delivery of the sound image position assignment data to the audio signal processing unit 136 by the sound image position assignment unit 135 is normally performed at intervals of 5 seconds. However, as will be described later, when the sound image position assignment unit 135 needs to change the assignment of the sound image position, the sound image position assignment data of the sound image position assignment data is set so that the sound image position assigned to the connection ID related to the change gradually moves. Generate and deliver at shorter time intervals.

  The audio signal processing unit 136 constantly monitors the amplitude of the newly written audio signal for all the data buffers 1392 secured in the storage unit 139, and if the absolute value of the amplitude exceeds a predetermined threshold, the data buffer It is determined that there is a statement from the participant 19 corresponding to 1392, and the sound image position adjustment process is performed on the audio signal. The audio signal processing unit 136 determines the sound image position according to the sound image position allocation data received last from the sound image position allocation unit 135 when performing the adjustment process of the sound image position with respect to the audio signal.

  For example, it is assumed that a participant 19 having a terminal ID “0301” makes a statement. As a result, the audio signal written to the data buffer 1392 specified by the connection ID “0015” (see FIG. 3) corresponding to the terminal ID “0301” exceeds a predetermined threshold. In this case, the audio signal processing unit 136 searches the connection ID “0015” from the sound image position allocation data received last. Now, assuming that the sound image position assigned by the sound image position assignment unit 135 is the one shown in FIG. 5, the audio signal processing unit 136 includes the connection ID “0015” in the sound image position assignment data received last. And that the connection ID is associated with “−30 degrees”. Therefore, the audio signal processing unit 136 is arranged such that the sound image position of the audio signal written in the data buffer 1392 specified by the connection ID “0015” is a position indicated by “−30 degrees”, that is, a left front position. Process the audio signal. In other words, the audio signal processing unit 136 generates two sets of duplicate audio signals, multiplies one of them by the amplification factor a (a> 1) and amplifies the amplitude thereof, and simultaneously increases the amplification factor b (b < Multiply by 1) to reduce its amplitude. The two sets of audio data whose amplitudes are adjusted in this way are used as the audio signals of the left channel and the right channel, respectively, so that the sound image position of the audio signal written in the data buffer 1392 is located on the left front side. It is.

  For example, it is assumed that the participant 19 having the terminal ID “0075” has made a statement. As a result, the audio signal written to the data buffer 1392 specified by the connection ID “0021” (see FIG. 3) corresponding to the terminal ID “0075” exceeds a predetermined threshold. In that case, the audio signal processing unit 136 searches the connection ID “0021” from the sound image position assignment data received last. When the sound image position assigned by the sound image position assigning unit 135 is as illustrated in FIG. 5, the audio signal processing unit 136 cannot find the connection ID “0021” in the sound image position assigning data. In that case, the audio signal processing unit 136 follows the “+10 degrees” associated with (permanent reservation) in the sound image position allocation data, and the sound image of the audio signal written in the data buffer 1392 specified by the connection ID “0021” The audio signal is processed so that the position becomes a position indicated by “+10 degrees”, that is, a position slightly ahead and to the right.

  The audio signal processing unit 136 performs sound image position adjustment processing on the audio signal whose amplitude exceeds a predetermined threshold as described above, and delivers the audio signal in the left and right channels, that is, in stereo, to the mixing unit 137. When the mixing unit 137 receives a stereo audio signal indicating the audio of one participant 19 from the audio signal processing unit 136, the mixing unit 137 delivers the received stereo audio signal as it is to the audio signal transmission unit 138. When the mixing unit 137 receives a stereo audio signal indicating the audio of two or more participants 19 from the audio signal processing unit 136, the mixing unit 137 mixes the received audio signal for each of the left and right channels, and generates the audio signal by mixing. The stereo synthesized audio signal is delivered to the audio signal transmission unit 138. In addition, the mixing unit 137 generates a stereo audio signal having an amplitude of 0 while receiving no audio signal from the audio signal processing unit 136, and delivers the generated audio signal to the audio signal transmission unit 138.

  When receiving the stereo audio signal from the mixing unit 137, the audio signal transmission unit 138 generates a data packet including the received audio signal. FIG. 6 is a diagram schematically showing how a data packet is generated in the audio signal transmission unit 138. The data packet generated by the audio signal transmission unit 138 differs from the data packet generated by the audio signal transmission unit 112 (see FIG. 4), instead of the audio signal block, the audio signal of the left channel and the audio of the right channel. A left audio signal block and a right audio signal block obtained by dividing the signal into predetermined data lengths are included. Further, since the audio signal processing server 13 needs to transmit an audio signal to the terminal devices 11 of all the participants 19 who are currently participating in the conference, the audio signal transmission unit 138 is established at that time. With respect to all communication connections, a plurality of data packets including a connection ID for specifying the communication connection in the header are generated simultaneously. The transmission destination ID included in each data packet is a terminal ID (see FIG. 3) corresponding to the connection ID.

  The audio signal transmission unit 138 sequentially transmits the data packets generated as described above to the network 10. The data packet sent to the network 10 is sent to each terminal device 11 based on the destination ID included in each data packet. When the audio signal receiving unit 113 of each terminal device 11 receives the data packet, the left audio signal block and the right audio signal block included in the received data packet are sequentially written in the area of the data buffer 1141 corresponding to the block number. . On the other hand, the audio signal processing unit 111 sequentially converts the audio signal written in the data buffer 1141 into an analog audio signal for each of the left and right, and outputs them to the left input and the right input of the headphones of the headset 12, respectively. The headphones of the headset 12 convert the sound signals input to the left input and the right input into sounds and generate sounds. The sound generated from the headset 12 is a sound obtained by mixing the voices of the participants 19 in different places, but the voices produced by the voices of the different participants 19 are arranged at different sound image positions. Even when a plurality of participants 19 speak at the same time, the listener can easily discriminate those comments.

  By the way, as described above, the accumulated time data generation unit 132 generates accumulated time data at intervals of 5 seconds, and the terminal device selection unit 133 generates selection data at intervals of 5 seconds. The frequency characteristic specifying unit 134 generates frequency characteristic data related to the audio signal corresponding to each connection ID included in the selection data at intervals of 5 seconds. Therefore, the sound image position assignment unit 135 receives selection data from the terminal device selection unit 133 and frequency characteristic data from the frequency characteristic specifying unit 134 at intervals of 5 seconds, and performs sound image position assignment processing based on them.

  When there is no change in the allocation of the sound image position, the sound image position allocation unit 135 generates sound image position allocation data having the same content at intervals of 5 seconds, and delivers it to the audio signal processing unit 136. On the other hand, when it is necessary to change the allocation of the sound image position, the sound image position allocation unit 135 sets the connection-related sound ID of the connection ID before the change at a time interval shorter than the 5-second interval, for example, an interval of 0.1 second. Sound image position assignment data is generated so as to gradually move from the sound image position to the changed sound image position, and delivered to the audio signal processing unit 136.

  FIG. 7 shows the result of frequent remarks made by the participant 19 of the terminal device 11 using the communication connection specified by the connection ID “0021” in the state in which the sound image position exemplified in FIG. 5 is assigned. FIG. 11 is a diagram schematically showing how the sound image position assigning unit 135 assigns a sound image position to a connection ID “0021” when a change occurs in the ranking of the accumulated speech time for the past 20 minutes. FIG. 7A shows a state before a change occurs in the ranking of the accumulated value of the speech time indicated by the accumulated time data.

In the state of FIG. 7A, when the ranking of the accumulated value of the speech time of the participant 19 corresponding to the connection ID “0021” is fourth, the position 3 assigned to the connection ID “0009” is changed to the connection ID “0009”. "0021". In that case, the sound image position assigning unit 135 follows the condition data “movement time: 20 seconds” included in the condition database 1391 (see FIG. 2), and the angular velocity is 1 degree per second (20 degrees ÷ 20 seconds = 1 degree / second). The sound image position that gradually moves from the position 4 to the position 3 is assigned to the connection ID “0021”. More specifically, the sound image position assignment unit 135 sequentially delivers the following sound image position assignment data to the audio signal processing unit 136 at intervals of 0.1 seconds.
[0023: -50 degrees, 0015: -30 degrees, 0021 (temporary hold): +9.9 degrees, (permanent hold): +10 degrees, 0004: +30 degrees, 0034: +50 degrees]
[0023: -50 degrees, 0015: -30 degrees, 0021 (temporary hold): +9.8 degrees, (permanent hold): +10 degrees, 0004: +30 degrees, 0034: +50 degrees]
[0023: -50 degrees, 0015: -30 degrees, 0021 (temporary hold): +9.7 degrees, (permanent hold): +10 degrees, 0004: +30 degrees, 0034: +50 degrees]
[0023: -50 degrees, 0015: -30 degrees, 0021 (temporary hold): +9.6 degrees, (permanent hold): +10 degrees, 0004: +30 degrees, 0034: +50 degrees]
(Hereinafter the same)

  As described above, “−10 degrees” indicating the position 3 is not immediately designated as the sound image position of the connection ID “0021” newly ranked fourth, but the position “ An angle that changes at an interval of 0.1 second is designated so as to move from “+10 degrees” to “−10 degrees” and gradually move to the left. The audio signal processing unit 136 sequentially receives the sound image position assignment data as described above at intervals of 0.1 seconds, and the sound of the participant 19 corresponding to the connection ID “0021” is assigned to the sound image position according to the received sound image position assignment data. Processing is performed so that is arranged. As a result, in the audio signal mixed by the mixing unit 137, the audio of the participant 19 corresponding to the connection ID “0021” moves from the front and right to the front and left slightly as time passes. FIG. 7B shows a state when 10 seconds have elapsed after the start of the movement of the sound image position, and FIG. 7C shows a state after the movement is completed.

  As described above, as a result of the sound image position being changed smoothly, the voice of the same speaker is not suddenly jumped from one sound image position to another sound image position. It ’s easier to see if it ’s a different speaker or a different speaker. However, whether the sound image position is changed smoothly or quickly can be arbitrarily changed by changing the parameter of the condition data indicating the moving speed or moving time included in the condition database 1391. That is, the sound image position may be immediately moved to the changed position by not including the condition data instructing the moving speed or moving time in the condition database 1391.

  As described above, according to the audio conference system 1, even when the number of participants 19 participating in the conference becomes large, a predetermined number of sound image positions are appropriately assigned to the audio of the participants 19. As a result, the interval between adjacent sound image positions does not become too narrow, and the inconvenience that the listener mixes the voices of different participants 19 arranged at adjacent sound image positions is prevented.

  Note that once the position is confirmed, the position may be maintained until it is out of the ranking.

  By the way, the audio signal processing server 13 and the terminal device 11 may be realized by dedicated hardware or by causing a general-purpose computer capable of inputting and outputting audio signals to execute processing according to an application program. Also good. When the audio signal processing server 13 is realized by a general-purpose computer, the accumulated time data generation unit 132, the terminal device selection unit 133, the frequency characteristic identification unit 134, the sound image position allocation unit 135, the audio signal processing unit 136, and the mixing unit 137 include: Functions of the general-purpose computer are achieved by a CPU (Central Processing Unit) provided in the general-purpose computer and a DSP (Digital Signal Processor) operating under the control of the CPU concurrently performing processing according to each module included in the application program. As realized. The audio signal receiving unit 131 and the audio signal transmitting unit 138 of the audio signal processing server 13 include an input / output interface provided for a general-purpose computer to transmit and receive data packets to and from the network 10, and each module included in the application program. This is realized as a function of a general-purpose computer by the processing of the CPU related to generation and assembly of data packets according to the above.

  Further, the audio conference system 1 may be configured to add a function of performing display or sound notification to the listener so that the listener can specify the voice speaker arranged at each sound image position. . For example, the audio signal processing server 13 transmits an audio signal to the terminal device 11, and transmits image data indicating the layout of the participants 19 at that time as illustrated in FIG. 5 to the terminal device 11. In that case, the terminal device 11 includes a display unit, and displays a layout diagram of the participants 19 on the display unit according to the image data transmitted from the audio signal processing server 13. As a result, all the participants 19 who are listeners can easily confirm which voice included in the sound generated from the headphones of the headset 12 is the speech of which participant 19. Further, for example, in the audio signal processing unit 136, an audio signal indicating the voice that reads out the name or nickname of the participant 19 arranged at each sound image position is synthesized at a predetermined time interval, and the sound image position of the synthesized audio signal is synthesized. An adjustment process may be performed and handed over to the mixing unit 137. In that case, the mixing unit 137 mixes the audio signal indicating the speech of the participant 19 and the synthesized audio signal that reads out the name of the speaker and the like at the same sound image position, and as a result, It is possible to easily know which participant 19 speaks the voice arranged at the sound image position.

  Further, the audio conference system 1 may be configured to encrypt an audio signal included in a data packet transmitted / received between the terminal device 11 and the audio signal processing server 13. In that case, the audio signal transmission unit 112 and the audio signal transmission unit 138 encrypt the audio signal to be transmitted, then divide the data into data blocks and include them in the data packet. The audio signal receiving unit 131 and the audio signal receiving unit 113 assemble a data block included in the data packet, and then decode the data block to restore the audio signal. Encrypting the audio signal in this way prevents the contents of the conference from leaking to a third party.

[1.3. Modified example]
In the above-described audio conference system 1, each of the terminal devices 11 is arranged at a position distant from each other, and performs data communication with the audio signal processing server 13 via the network 10, thereby allowing multipoint conferences. Realize. However, the first embodiment of the present invention can be modified such that each terminal device 11 is directly connected to the audio signal processing server 13 without going through the network 10. The voice conference system to which such a modification is added is a convenient system when, for example, dozens of people conduct a conference through the same.

  Further, the above-described sound image position assignment processing and audio signal processing in the server may be performed by each terminal device. In this case, for example, each terminal device 11 includes a condition database 1391, a terminal device selection unit 133, a frequency characteristic specifying unit 134, a sound image position assignment unit 135, an audio signal processing unit 136, and a mixing unit 137, while an audio signal processing server 13 does not include these components, and transmits the audio signal received from each terminal apparatus 11 to each terminal apparatus 11 together with the accumulated time data as an audio signal of a plurality of tracks without mixing. Each terminal apparatus 11 uses a plurality of tracks of audio signals and accumulated time data received from the audio signal processing server 13 to select a predetermined number of audio signals and assign sound image positions to the selected audio signals.

  Furthermore, a plurality of terminal devices having the same configuration as the audio signal processing server 13 described above may be interconnected peer-to-peer to constitute an audio conference system. In that case, for example, each terminal device 11 transmits the audio signal of the participant 19 to all the other terminal devices 11 and receives the audio signal of each participant 19 from all the other terminal devices 11. The terminal device 11 performs a sound signal selection process, a sound image position assignment process, and the like similar to the case of the sound signal processing server 13 described above for the plurality of sound signals thus received.

[2. Second Embodiment]
In the audio conference system 1 according to the first embodiment, it is possible to prevent the listener from confusing the speech of the different participants 19 by assigning different sound image positions to the audio signals indicating the speech of the different participants 19. An audio signal was generated. In the audio conference system 2 according to the second embodiment, by assigning different frequency characteristics to audio signals indicating the speech of different participants 19, an audio signal capable of preventing the listener from confusion with the speech of different participants 19. Is generated. The audio conference system 2 is common in many respects to the audio conference system 1 in its configuration and operation. Therefore, in the following description, only the points where the audio conference system 2 is different from the audio conference system 1 will be described. In addition, among the components of the audio conference system 2, components that are the same as or correspond to the components of the audio conference system 1 are assigned the same reference numerals.

[2.1. Configuration and operation of audio conference system]
FIG. 8 is a block diagram showing the configuration of the audio conference system 2. The audio signal processing server 13 of the audio conference system 2 includes a frequency characteristic allocation unit 235 instead of the sound image position allocation unit 135 provided in the audio signal processing server 13 of the audio conference system 1. The frequency characteristic assigning unit 235 assigns different frequency characteristics to each connection ID indicated by the selection data. There are various methods for making a plurality of frequency characteristics different from each other, but in this embodiment, as an example, by performing processing by a parametric equalizer having different center frequencies on speech signals of different speakers, The frequency characteristics should be greatly different from each other. Therefore, the frequency characteristic assigning unit 235 assigns a different center frequency to each connection ID. FIG. 9 is a diagram schematically showing how different center frequencies are assigned to connection IDs by the frequency characteristic assigning unit 235.

  When the frequency characteristic allocating unit 235 allocates connection IDs for the respective center frequencies in accordance with the condition data included in the condition database 1391 based on the selection data received from the terminal device selection unit 133 and the frequency characteristic data received from the frequency characteristic specifying unit 134. Then, the frequency characteristic allocation data indicating the result is delivered to the audio signal processing unit 136. When the center frequency allocation shown in FIG. 9 is performed, the frequency characteristic allocation data are [0023: 5.0 kHz, 0015: 5.9 kHz, 0009 (temporary hold): 7.0 kHz, (permanent hold): 8. 5 kHz, 0004: 10.6 kHz, 0034: 13.4 kHz].

  The audio signal processing unit 136 processes the parametric equalizer according to the frequency characteristic assignment data received last, that is, the audio signal whose amplitude exceeds a predetermined threshold among the audio signals sequentially stored in the data buffer 1392, that is, FIG. In FIG. 5, the filter processing of the frequency characteristics as shown in the graph having the respective center frequencies is performed, and the generated audio signal is delivered to the mixing unit 137. When the mixing unit 137 receives audio signals that have been subjected to filter processing with different frequency characteristics for each participant 19, the mixing unit 137 mixes them to generate one audio signal. The audio signal generated by the audio signal processing server 13 is transmitted to all the terminal devices 11 via the network 10 and is generated as sound via the headset 12. Here, the audio signal transmitted from the audio signal processing server 13 to the terminal device 11 in the audio conference system 2 may be a monaural audio signal, and thus the headphones included in the headset 12 may be monaural headphones.

  As described above, the sound that is generated from the headphones of the headset 12 is obtained by mixing the sound indicating the speech in the conference. However, in the mixed audio, since the audio indicating the speech by the different participants 19 is adjusted to have greatly different frequency characteristics, the listener can easily distinguish the speech by the different participants 19. At that time, even when the number of participants 19 participating in the conference becomes large, a predetermined number of frequency characteristics are appropriately assigned to the voices of the different participants 19 and, as a result, specified by adjacent center frequencies. The difference between the frequency characteristics is always maintained to a certain extent. As a result, the inconvenience of confusing the voices of participants 19 with different listeners is always prevented.

  Also in the audio conference system 2, as in the case of the sound image position allocation unit 135 in the audio conference system 1, when the frequency characteristic allocation unit 235 changes the allocation of the center frequency, the center frequency is gradually changed as time elapses. The frequency characteristic assignment data is generated and delivered to the audio signal processing unit 136 so as to be changed to the changed one. As a result, the listener can easily capture the voice of the same speaker even when the frequency characteristics of the voice are changed.

  Also in the second embodiment, as in the first embodiment, the audio signal processing server 13 and the terminal device 11 may be realized by dedicated hardware or applied to a general-purpose computer capable of inputting and outputting audio signals. You may implement | achieve by performing the process according to a program. Further, the audio conference system 2 may be configured to add a function of performing display or sound notification to the listener so that the listener can identify the voice speaker to which each center frequency is assigned. Further, in the audio conference system 2, the audio signal may be encrypted. Furthermore, in the second embodiment, the same modification as the modification in the first embodiment can be added.

  The sound image position assignment in the first embodiment may be combined with the frequency characteristic assignment in the second embodiment. In that case, the audio signal processing server 13 includes both the sound image position allocating unit 135 and the frequency characteristic allocating unit 235, and each of the audio signals selected by the terminal device selecting unit 133 has both a different sound image position and a different frequency characteristic. Assign. As a result, even if voice signals of similar timbres are arranged at close sound image positions, the processing is performed so that their frequency characteristics are different, so that it is easier for the listener to distinguish the voices of different speakers. Will be able to. Note that it is also possible to configure an audio conference system that can provide a voice that is easier for the listener to distinguish by assigning significantly different frequency characteristics to the audio signals arranged at the sound image positions close to each other.

It is the block diagram which showed the structure of the audio conference system concerning 1st Embodiment of this invention. It is the figure which illustrated the contents of the condition database concerning a 1st embodiment of the present invention. It is the figure which illustrated the correspondence table of connection ID and terminal ID concerning 1st Embodiment of this invention. It is the figure which showed typically the relationship between the audio | voice signal and data packet concerning 1st Embodiment of this invention. It is the figure which showed typically the mode of the allocation of the sound image position concerning 1st Embodiment of this invention. It is the figure which showed typically the relationship between the audio | voice signal and data packet concerning 1st Embodiment of this invention. It is the figure which showed typically the mode of the movement of the sound image position concerning 1st Embodiment of this invention. It is the block diagram which showed the structure of the audio conference system concerning 2nd Embodiment of this invention. It is the figure which showed typically the mode of the allocation of the frequency characteristic concerning 2nd Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... 2 Audio conference system, 10 ... Network, 11 ... Terminal device, 12 ... Headset, 13 ... Audio signal processing server, 111 ... Audio signal processing unit, 112/138 ... Audio signal transmission unit, 113/131 ... Audio Signal receiving unit, 114 · 139 ... storage unit, 132 ... cumulative time data generation unit, 133 ... terminal device selection unit, 134 ... frequency characteristic specifying unit, 135 ... sound image position assignment unit, 136 ... audio signal processing unit, 137 ... mixing , 235... Frequency characteristic assignment unit, 1141 and 1392... Data buffer, 1391.

Claims (7)

Input means for receiving audio signals from a plurality of terminal devices;
Accumulated time data generating means for generating accumulated time data indicating accumulated values of durations of audio signals received within a predetermined past period from each of the plurality of terminal devices by the input means;
Selection means for selecting a predetermined number of terminal devices from the plurality of terminal devices based on the accumulated time data;
Assigning means for assigning different sound image positions to each of the plurality of terminal devices selected by the selecting means;
When an audio signal is received by the input unit from a plurality of terminal devices selected by the selection unit, the audio signal is processed into an audio signal having a sound image position allocated to the terminal device by the allocation unit. Processing means for processing the audio signal into an audio signal having a predetermined sound image position when an audio signal is received by the input means from a terminal device not selected by the selection means;
An audio signal processing apparatus comprising: output means for outputting an audio signal processed by the processing means. The assigning means changes the time interval from the one sound image position to the other sound image position with respect to the one terminal device when changing the one sound image position assigned to the one terminal device to another sound image position. The sound signal processing apparatus according to claim 1, wherein a sound image position that changes with progress is assigned. With respect to each of the plurality of terminal devices selected by the selection means, comprising frequency characteristic specifying means for generating frequency characteristic data indicating the frequency characteristics of the audio signal received from the terminal device,
The assigning means includes a plurality of terminals selected by the selecting means so that the similarity between the frequency characteristic data of the audio signals received from the terminal device that assigns two arbitrary sound image positions satisfies a predetermined condition. The sound signal processing apparatus according to claim 1, wherein a sound image position is assigned to each of the apparatuses. Input means for receiving audio signals from a plurality of terminal devices;
Accumulated time data generating means for generating accumulated time data indicating accumulated values of durations of audio signals received within a predetermined past period from each of the plurality of terminal devices by the input means;
Selection means for selecting a predetermined number of terminal devices from the plurality of terminal devices based on the accumulated time data;
Assigning means for assigning different frequency characteristics to each of the plurality of terminal devices selected by the selecting means;
When an audio signal is received by the input means from a plurality of terminal devices selected by the selection means, the audio signal is processed into an audio signal having a frequency characteristic assigned to the terminal device by the assigning means. Processing means for processing the audio signal into an audio signal having a predetermined frequency characteristic when an audio signal is received by the input means from a terminal device not selected by the selection means;
An audio signal processing apparatus comprising: output means for outputting an audio signal processed by the processing means. The assigning means, when changing one frequency characteristic assigned to one terminal apparatus to another frequency characteristic, for the one terminal apparatus, time is changed from the one frequency characteristic to the other frequency characteristic. The audio signal processing apparatus according to claim 4, wherein a frequency characteristic that changes over time is assigned. Receiving audio signals from a plurality of terminal devices;
Generating accumulated time data indicating accumulated values of durations of audio signals received from each of the plurality of terminal devices within a predetermined period in the past;
A process of selecting a predetermined number of terminal devices from the plurality of terminal devices based on the accumulated time data;
A process of assigning different sound image positions to each of the plurality of selected terminal devices;
When an audio signal is received from the selected terminal devices, the audio signal is processed into an audio signal having a sound image position assigned to the terminal device, and a terminal device other than the selected terminal device A process of processing the audio signal into an audio signal having a predetermined sound image position when the audio signal is received from
A program for causing a computer to execute the process of outputting the processed audio signal. Input means for receiving audio signals from a plurality of terminal devices;
Generating accumulated time data indicating accumulated values of durations of audio signals received from each of the plurality of terminal devices within a predetermined period in the past;
A process of selecting a predetermined number of terminal devices from the plurality of terminal devices based on the accumulated time data;
A process of assigning different frequency characteristics to each of the selected plurality of terminal devices;
When an audio signal is received from the selected terminal devices, the audio signal is processed into an audio signal having a frequency characteristic assigned to the terminal device, and other than the terminal device selected by the selection unit When a voice signal is received from the terminal device, a process of processing the voice signal into a voice signal having a predetermined frequency characteristic;
A program for causing a computer to execute the process of outputting the processed audio signal.

Images