JP2018101826A - Voice speech system, voice speech method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce computational complexity on a device for mixing speeches.SOLUTION: A voice speech system includes multiple client devices and a server device. Each client device includes voice acquisition means 30 for acquiring voice data as time-domain data, first conversion means 31 for sampling the acquired time-domain data, dividing it into blocks of constant sample number, and converting it into frequency region data in units of divided blocks, transmission means 32 for transmitting the converted frequency region data to the server device 11, reception means 33 for receiving mixed data generated by mixing multiple frequency region data from the server device 11, and second conversion means 31 for converting the mixed data received by the reception means 33 into the time-domain data. Thus, the server device 11 can mix multiple frequency region data, without converting the frequency region data into the time-domain data.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a voice call system for exchanging voice data to make a voice call, a voice call method, and a program for causing a computer to execute the method.

  Pear To Pear (hereinafter referred to as P2P) is widely used as a network connection form in VoIP systems, video conferencing systems, voice chat systems, and the like in which a plurality of users have a conversation via the Internet (see Patent Document 1). However, P2P connects clients, so as the number of voice calls increases, the communication volume of the entire system increases enormously.

  As a method of solving the problem that the amount of communication in P2P communication increases enormously, in the relay device (server) provided in the middle of the communication path, the voices uttered by other users to one user (client) There are systems that mix, transmit to a terminal used by the one user, and output audio. A network system employing such a communication method is called a client / server model. In the client / server model, audio data is encoded and transmitted at a terminal used by the other users, and is decoded into time domain data by a relay device (see, for example, Patent Document 2). The result is mixed, and the result is encoded and transmitted to the terminal used by the one user.

  In a system in which a plurality of users have a conversation via the Internet, a voice codec based on differential pulse code modulation (DPCM) such as ITU-T G.711 is widely used for compressing call voice. However, the voice codec based on DPCM has a low compression rate and the call quality cannot be improved due to the low bit rate. In order to improve call quality while suppressing the amount of communication, it is desirable to use a voice codec based on transform coding, in which time domain data is converted into frequency domain data and then encoded.

JP 2005-328178 A JP 2010-044175 A

  However, in the above relay device, it is necessary to decode the voice of the user performing the conversation into time domain data and re-encode it, so that the amount of calculation on the device that mixes the voice increases enormously as the number of people increases. There was a problem.

  For this reason, provision of the system and method which can reduce the computational complexity on the apparatus which mixes an audio | voice was desired.

  In view of the above problems, the present invention is a voice call system including a plurality of client devices and a server device, and each client device acquires voice data as time domain data representing a change in sound pressure with respect to time. An acquisition unit, a first conversion unit that samples time-domain data, divides the data into blocks having a fixed number of samples, and converts the divided frequency units into frequency-domain data representing the strength of each frequency component; Including transmission means for transmitting region data to the server device, and reception means for receiving mixed data generated by mixing a plurality of frequency domain data from the server device, wherein the mixed data received by the receiving means is time domain There is provided a voice call system comprising: a second conversion means for converting data.

  By providing the system of the present invention, it is possible to reduce the amount of calculation on a device for mixing audio.

It is the figure which showed the structural example of the voice call system. It is a figure for demonstrating the outline | summary of the voice call system of this invention. It is a figure which shows the process which a client apparatus performs data conversion in the block unit with a fixed sample number. It is a figure which shows the processing content of a server apparatus. It is the figure which illustrated the hardware constitutions of the client apparatus. It is a functional block diagram of a client apparatus. It is the flowchart which showed the flow of the transmission process of the audio | voice data performed in a client apparatus. It is the flowchart which showed the flow of the reception process of the audio | voice data performed in a client apparatus. It is the functional block diagram which showed 1st Embodiment of the server apparatus. It is the figure which illustrated the correspondence table. It is the flowchart which showed the flow of the mixing process of the audio | voice data performed in a server apparatus. It is the functional block diagram which showed 2nd Embodiment of the server apparatus.

  FIG. 1 is a diagram showing a configuration example of a voice call system. In a voice call performed by a plurality of users, it is necessary to transmit the voices of all other users to one user. When performing a voice call between two users, it is only necessary to transmit voice data between users. However, as the number of people increases, the amount of communication increases enormously, so the voice data is mixed (mixed). It is desirable to deliver. Therefore, the voice call system includes a plurality of client devices 10a to 10n that obtain voice data of voices uttered by a user and output voices of all other users, and a server device 11 that mixes the voice data. Composed.

  The plurality of client devices 10 a to 10 n and the server device 11 are connected to the network 12 and can communicate with each other via the network 12. The client devices 10 a to 10 n accept voices uttered by each user using the client devices 10 a to 10 n as voice data, and transmit the voice data to the server device 11 via the network 12.

  The server apparatus 11 mixes and distributes the audio data received from the client apparatuses 10b to 10n other than the client apparatus 10a to the client apparatus 10a. Similarly, the server apparatus 11 mixes and distributes the audio data received from the client apparatuses 10a, 10c to 10n other than the client apparatus 10b to the client apparatus 10b. In this way, the server apparatus 11 also mixes and distributes the audio data to the client apparatuses 10c to 10n in the same manner.

  Each user inputs and transmits his / her voice from the client device he / she uses, receives and outputs the voices of all other users, and listens to it, so that multiple users can Simultaneous voice calls between users can be realized.

  The client devices 10a to 10n may be any devices as long as they can execute the above processing, and are, for example, home game devices, amusement devices, personal computer devices, mobile phone devices, and embedded devices capable of voice input / output. be able to.

  The network 12 may be any of a LAN (Local Area Network), a WAN (Wide Area Network), and the Internet, and may be a wired network or a wireless network. Wireless communication can be performed by wireless LAN, Bluetooth (registered trademark), infrared communication, and when using a wireless LAN, can be performed via an access point connected to the network 12. The network 12 is not limited to one network, and may be two or more networks, and the two or more networks can be connected by a relay device such as a router or a proxy server.

  The plurality of client devices 10a to 10n constituting the voice call system can be authenticated client devices by, for example, logging in by a user who uses the voice call service. User authentication may be authentication by inputting a user ID or password, biometric authentication, or authentication performed by holding an IC card, a mobile phone, a smartphone, or the like over a reading device. Good. These authentication methods are examples, and any authentication method known so far can be adopted.

  When the client devices 10a to 10n acquire voice data of a voice uttered by the user and transmit it to the server device 11, the client devices 10a to 10n add user identification information or device identification information for identifying the user or the client devices 10a to 10n. Can be sent. Thereby, the user and apparatus which transmitted each audio | voice data can be identified. The identification information may be transmitted with both of them added, but since the data size becomes large, it is preferable to transmit only one of the pieces of information.

  A user name or a user ID can be used as the user identification information, and a device name, a device ID, an IP address, a MAC (Media Access Control) address, or the like can be used as the device identification information. The user identification information can be obtained from information input by the user at the time of login, and the device identification information can be obtained by using information registered in advance in the client device.

  The server device 11 may identify the audio data received from the client devices 10b to 10n other than the client device 10a by the user identification information or the device identification information, and mix and distribute the identified audio data to the client device 10a. it can. Similarly, the server apparatus 11 can identify the audio data, mix and distribute it to the other client apparatuses 10b to 10n.

  Next, the outline of the present invention will be described. FIG. 2 is a diagram for explaining the outline of the voice call system of the present invention. Here, FIG. 2A shows a conventional voice call system, and FIG. 2B shows a voice call system of the present invention. FIG. 3 is a diagram illustrating processing in which the client device performs data conversion in units of blocks having a fixed number of samples. FIG. 4 is a diagram illustrating processing contents of the server device. Here, FIG. 4A shows the processing contents of the server apparatus in the conventional voice call system, and FIG. 4B shows the processing contents of the server apparatus in the voice call system of the present invention.

  As shown in FIGS. 2 and 3, on the client device side, the microphone 100 collects the voice uttered by the user and acquires the voice data (see FIGS. 2A and 2B and FIG. 3A). ). The microphone 100 corresponds to a voice input device 26 in FIG. 5 and a voice acquisition unit 30 in FIG. The client device converts analog audio data into digital audio data (see A / D conversion in FIG. 3B). In the A / D conversion processing, the client device converts continuous analog audio data into discrete digital audio data by averaging the values at predetermined sampling intervals and sequentially extracting the values. As shown in FIG. 3 (3), when the sampling frequency is, for example, 44.1 kHz, the audio data after the A / D conversion is data of 44100 samples per second. The sampling frequency is not limited to 44.1 kHz, and may be 48 kHz.

  Next, the client device divides the audio data after A / D conversion into blocks having a fixed number of samples (see FIGS. 3 (3) and (4)). As the block size, for example, the number of samples is assumed to be 128 or 1024. The audio data (sampling data) shown in FIG. 3 (3) is time domain data representing a change in sound pressure with respect to time. In FIG. 2, the time domain data is represented as “wave”.

  Next, the client device converts the time domain data into frequency domain data representing the strength of each frequency component in units of blocks (see FIGS. 2A and 2B and FIG. 3B). In the example shown in FIGS. 2 and 3, the modified discrete cosine transformation (MDCT) is used for the conversion from the time domain data to the frequency domain data. However, Fourier transform (Fourier Transformation), discrete Fourier transform (DFT: Discrete Fourier Transformation), fast Fourier transform (FFT), discrete cosine transform (DCT: Discrete Cosine Transformation), etc. may be used. In FIG. 2, the frequency domain data is expressed as “frequency”. Thereafter, the client device transmits the frequency domain data after the modified discrete cosine transform to the server device via the network 12.

  In the conventional voice call system shown in FIGS. 2 (A) and 4 (A), when the server apparatus receives the frequency domain data transmitted from the client apparatus, the inverse corrected discrete cosine transform is performed on the received frequency domain data. (IMDCT: Inverse Modified Discrete Cosine Transformation) is performed to return the frequency domain data to the time domain data.

  As described above, the reason why the frequency domain data is returned to the time domain data by performing the inversely modified discrete cosine transform in the conventional voice communication system is as follows. That is, in general speech encoding processing, processing is performed while switching block sizes in order to increase the compression rate. If the block size is made variable in order to increase the compression rate, a plurality of frequency domain data cannot be mixed (mixed). In other words, if the block size when performing the modified discrete cosine transform is not the same size, a plurality of frequency domain data cannot be directly added and mixed.

  Therefore, in the conventional voice call system, the server device performs inverse correction discrete cosine transform on the frequency domain data to temporarily convert the frequency domain data into time domain data (waveform), and a plurality of time domain data. (Waveform) is added to generate mixed data. Thereafter, the server device performs the modified discrete cosine transform again to convert the generated mixed data (time domain data) into frequency domain data and transmit it to the client device of the other party.

  On the other hand, in the voice call system of the present invention shown in FIGS. 2B and 4B, as described above, the block size when performing the modified discrete cosine transform is the same size, so the server device It is possible to directly add and mix (mix) a plurality of frequency domain data. Therefore, the server device generates mixed data by directly adding a plurality of frequency domain data without performing conversion processing, and transmits the generated mixed data (frequency domain data) to the client device of the other party.

  As shown in FIGS. 2A and 2B, when the client device receives the mixed data transmitted from the server device, the client device performs inverse correction discrete cosine transform on the received mixed data (frequency domain data). The frequency domain data is converted into time domain data. Then, the client device outputs the time domain data from the speaker 200 as a sound. In FIGS. 2 and 4, inverse modified discrete cosine transform (IMDCT) is used, but not limited to such processing, inverse Fourier transform, inverse discrete Fourier transform, or the like may be used. As described above, in the voice call system shown in FIGS. 2B and 4B, the server device directly adds and mixes the plurality of frequency domain data without performing the inverse correction cosine transform, and thus the processing load is also increased. It is greatly reduced.

  FIG. 5 is a diagram illustrating a hardware configuration of the client devices 10a to 10n for realizing the above processing. Hereinafter, the client device 10a to 10n will be referred to as the client device 10 unless a specific client device is indicated. The client device 10 includes a CPU 20, ROM 21, RAM 22, HDD 23, communication I / F 24, input / output I / F 25, audio input device 26, and audio output device 27 as hardware. The CPU 20, ROM 21, RAM 22, HDD 23, communication I / F 24, and input / output I / F 25 are connected to the bus 28 and allow data and the like to be exchanged via the bus 28. The client device 10 may have a configuration including a display device instead of the audio output device 27 or a configuration including a display device separately. Moreover, you may further provide the input device for inputting information.

  The CPU 20 controls the entire client device 10. The ROM 21 stores a boot program for starting the client device 10, firmware for controlling the HDD 23, and the like. The RAM 22 provides a work area for the CPU 20. The HDD 23 stores audio data, stores a program for converting the acquired audio data, an OS and other programs, various setting data, and the like.

  The communication I / F 24 connects the client device 10 to the network 12 and controls communication with the server device 11 and other client devices. The input / output I / F 25 controls input of audio data from the audio input device 26 and output of audio data to the audio output device 27. The sound input device 26 is a device that inputs sound such as a microphone, and the sound output device 27 is a device that outputs sound such as a speaker.

  Although the server device 11 is not described with reference to the drawings, the input / output I / F 25, the voice input device 26, and the voice output device 27 are not required, so the CPU 20, ROM 21, RAM 22, HDD 23, and communication I / F 24 are hard-wired. Can be provided as wear. In these apparatuses, the configuration using the HDD 23 is illustrated, but the present invention is not limited to this, and an SSD (Solid State Drive) or the like may be used. In addition, when this voice call system is used for a video conference system or the like, an imaging device such as a camera can be further provided to acquire and distribute video.

  FIG. 6 is a functional block diagram for explaining functions provided in the client device 10. The client device 10 has a function of acquiring voice uttered by a user as voice data, a function of transmitting voice data to the server device 11 via the network 12, and receiving mixed voice data from the server device 11 via the network 12. And a function of outputting mixed audio data. Therefore, the client device 10 can be a device provided with each of the above functions as functional means. These functions can be realized by the CPU 20 reading and executing a program stored in the HDD 23.

  The client device 10 includes a voice acquisition unit 30, a conversion unit 31, a transmission unit 32, a reception unit 33, and a voice output unit 34 as functional units.

  The voice acquisition unit 30 acquires voice data of a voice uttered by a user as time domain data representing a change in sound pressure with respect to time. When using a microphone, audio data is acquired as time domain data. The conversion unit 31 converts the time domain data acquired by the audio acquisition unit 30 into frequency domain data representing the strength of each frequency component. The frequency domain data is data indicating how many frequency components are included.

  An audio codec can be used to convert time domain data to frequency domain data. A codec is a program that can bidirectionally perform encoding (encoding) of data and decoding (decoding) of encoded data. The conversion means 31 can use a modified discrete cosine transformation (MDCT) or the like for this conversion. In addition, Fourier transform, discrete Fourier transform, fast Fourier transform, or the like can be used. In addition, when transforming frequency domain data into time domain data, inverse modified discrete cosine transform (IMDCT) or the like can be used, and in addition, inverse Fourier transform, inverse discrete Fourier transform, or the like can also be used.

  Also, for the conversion from time domain data to frequency domain data used for speech compression, a subband filter that converts to the frequency domain with multiple bandpass filters and the above-mentioned MDCT and IMDCT other than the orthogonal orthogonal transform (LOT) It can also be used. The bandpass filter is a filter circuit that passes only a specific frequency and does not pass other frequencies. Orthogonal transformation transforms a time domain signal into frequency components.

  As the above codec, any codec can be used as long as the number of samples to be converted from the frequency domain to the time domain is constant (the block size is the same). An example is a codec that performs fixed-size conversion, such as MPEG-1 Audio Layer-2. Incidentally, codecs such as mp3 and AAC that can change the processing unit for each frame cannot be used unless the frame size is limited.

  In particular, the use of overlapping orthogonal transforms, such as MDCT / IMDCT, can interpolate with a small amount of computation for packet arrival delays and packet loss failures that occur frequently in Internet communications, and suppress voice communication quality degradation. Can do.

  Frequency domain data transmitted / received via the network 12 is transmitted / received by being divided into transmission units generally called packets. When a voice call is realized on the Internet used as the network 12, this packet may be lost during transmission (packet lost), and the packet order may be changed. If the audio data becomes discontinuous, it will sound like a sudden and annoying noise for humans.

  In order to reduce this noise, it is possible to add redundant audio data to audio data exchanged on the network 12, or to add a process for smoothing discontinuous parts after decoding into time domain data. .

  However, in the method of adding redundant audio data, the data size becomes large, and the processing steps increase in the smoothing process. Thus, when transforming time domain data to frequency domain data, overlapping orthogonal transformation can be used. The above MDCT can be used as an example of overlapping orthogonal transformation. MDCT has the data before and after within one processing unit and the crossfading component that switches while the audio overlaps, so even if part of the audio data is lost and becomes discontinuous, the discontinuous part is smoothed A connecting effect can be obtained. For this reason, the secondary effect which suppresses the noise by packet loss can be acquired. In order to obtain the same effect, when the frequency domain data is converted into the time domain data, the overlapping orthogonal transformation can be used, and the IMDCT described above can be used as an example.

  The transmission unit 32 transmits the frequency domain data converted by the conversion unit 31 to the server device 11 via the network 12. The transmission unit 32 can add the user identification information input by the user by login or the device identification information registered in the client device 10 to the frequency domain data and transmit it.

  The transmission unit 32 may transmit the frequency domain data as it is, but can also transmit the data encoded by the conversion unit 31. Encoding is data compression, and the conversion means 31 can be performed simultaneously with conversion to frequency domain data using the above-described codec.

  The receiving unit 33 receives audio data obtained by mixing the frequency domain data from the server device 11, that is, mixed data. The receiving unit 33 may receive mixed data obtained by mixing the frequency domain data, or may receive mixed data obtained by encoding the mixed data in the server device 11. The conversion unit 31 converts the mixed data received by the reception unit 33 into time domain data using the codec. When the reception unit 33 receives the encoded mixed data, the conversion unit 31 can decode the mixed data and convert the decoded mixed data into time domain data. The audio output unit 34 outputs the time domain data converted by the conversion unit 31 as audio.

  Audio data transmission processing performed by each client device 10 will be described with reference to FIG. In response to logging in to the voice call service using the client device 10, the user starts processing from Step 400. In step 405, the voice acquisition unit 30 determines whether there is a voice input. If there is a voice input, the process proceeds to step 410, and if there is no input, the process proceeds to step 425.

  In step 410, the voice acquisition means 30 acquires voice data as time domain data. In step 415, the conversion means 31 converts time domain data into frequency domain data. In step 420, the transmission unit 32 transmits the frequency domain data to the server device 11. In step 425, it is determined whether the user has terminated the voice call service and has logged off. If not logged off, the process returns to step 405, and the processing from step 405 to step 425 is repeated. If it is logged off, the process proceeds to step 430 and the process is terminated.

  Next, audio data reception processing performed by each client device 10 will be described with reference to FIG. In response to logging in to the voice call service using the client device 10, the user starts processing from Step 500. In step 505, the receiving unit 33 receives the mixed data from the server device 11. In step 510, the conversion means 31 converts the mixed data into time domain data.

  In step 515, the audio output means 34 outputs the time domain data as audio. In step 520, it is determined whether the user has terminated the voice call service and logged off. If not logged off, the process returns to step 505 and the processing from step 505 to step 520 is repeated. In the case of logoff, the process proceeds to step 525 and the process is terminated.

  In the client device 10, the process illustrated in FIG. 7 and the process illustrated in FIG. 8 can be performed in parallel. Note that if only the conversation between other users is heard, only the processing shown in FIG. 8 need be performed. In this case, the client device 10 may not include the voice acquisition unit 30.

  FIG. 9 is a functional block diagram illustrating functions provided in the server device 11. The server device 11 has a function of receiving audio data from each client device, a function of mixing audio data, and a function of distributing audio data to each client device. Therefore, the server device 11 can be a device provided with each of the above functions as functional means. These functions can be realized by the CPU reading and executing the program stored in the HDD, as in the client device 10.

  The server device 11 includes a receiving unit 40, a mixing unit 41, and a distribution unit 42 as functional units. The receiving means 40 receives the frequency domain data transmitted from each client device. The receiving unit 40 extracts user identification information or device identification information added to the frequency domain data, and passes the frequency domain data and the identification information to the mixing unit 41. When receiving the encoded frequency domain data from the client device 10, the receiving unit 40 can decode it.

  The mixing unit 41 mixes the frequency domain data received from all the other client devices and generates mixed data for each client device. For example, for the client device 10a, the frequency domain data received from the client devices 10b to 10n is identified by user identification information or the like, and the identified frequency domain data is mixed to generate mixed data.

  Since the frequency domain data is data digitized for each frequency component indicating how much frequency component is included, it can be mixed by adding the digitized data for each frequency component.

  The distribution unit 42 distributes the mixed data generated by the mixing unit 41 for each client device to each client device. For example, the distribution unit 42 refers to a correspondence table in which device IDs and IP addresses are associated with each other as shown in FIG. 10, acquires an IP address from the correspondence table, and uses this to distribute frequency domain data Can do.

  In addition to the IP address, the correspondence table can also include device status information for identifying the device logged into this service. The state of the device can be set to either log-in or log-off. In FIG. 8, since the device IDs “client1” to “client4” are logged in and “client5” is logged off, it indicates that a voice call is being performed with the devices “client1” to “client4”.

  By referring to the correspondence table, the mixing unit 41 and the distribution unit 42 recognize the logged-in device as the client device 10 constituting the voice call system, mix which frequency domain data to generate mixed data, It is possible to identify which client device 10 is to be distributed. The status information can be set by the receiving means 40 when the receiving means 40 accepts that the client device 10 has logged in or logged off.

  The server apparatus 11 generates mixed data by mixing audio data received from all other client apparatuses with respect to one client apparatus, and distributes the mixed data to the one client apparatus. The distribution means 42 can also encode and distribute the frequency domain data. However, not all the other client devices are transmitting audio data. In this case, the server device 11 can generate mixed data by mixing audio data received from a client device that is transmitting audio data among all other client devices.

  Further, when the number of simultaneous speakers is smaller than the number of simultaneously connected persons, the server apparatus 11 transmits mixed data for a certain client apparatus as mixed data for another client apparatus, and further reduces the amount of calculation of the mixing process. be able to.

  In addition, when the server apparatus 11 simply mixes audio data received from all other client apparatuses to create mixed data for one client apparatus, the audio data actually received from all client apparatuses Can be further reduced by mixing only once and creating common mixed data to be transmitted to all client devices. The received client device generates mixed data of audio data transmitted from all other client devices by inverting the phase of the transmitted audio data to the mixed data created in this way and adding them together. can do. For this reason, each client device can further include an adding means for inverting the phase of the audio data, that is, the frequency domain data, and adding it to the mixed data.

  With reference to FIG. 11, the audio data mixing process performed by the server apparatus 11 will be described in detail. The server apparatus 11 starts processing from step 800 in response to a plurality of users logging in to the service using each client apparatus. In step 805, the receiving means 40 receives frequency domain data from the plurality of client devices 10.

  In step 810, the mixing unit 41 generates the mixed data for each client device 10 by mixing the frequency domain data received from all the other client devices and generating the mixed data for one client device. To do. Since the frequency domain data is identified by the added user identification information or device identification information, it is possible to identify the frequency domain data to be mixed according to the client device 10 and to mix the identified frequency domain data.

  In step 815, the distribution unit 42 distributes each mixed data generated by the mixing unit 41 for each client device 10 to each client device 10. The distribution means 42 can distribute the mixed data to the distribution destination client device 10 with reference to the correspondence table shown in FIG. In step 820, it is determined whether all users have logged off. If not logged off, since the voice call is still being made, the process returns to step 805 to repeat the processing from step 805 to step 820. If it is logged off, the process proceeds to step 825 to end the process.

  As described above, since the server apparatus 11 does not perform decoding and encoding but performs conversion by the client apparatus 10, it is possible to reduce the amount of calculation in the server apparatus 11 that mixes audio. Thereby, since the load of the server apparatus 11 can be reduced, it becomes possible to realize simultaneous calls among more users.

  The server device 11 can apply an audio effect to the received frequency domain data. The sound effect is processing of sound, and examples thereof include noise removal, voice change, volume change, panning, and the like. These are only examples, and other effects can be applied. As described above, by performing volume change and panning in the server device 11, it is possible to provide an effect that voice data of a certain user is arranged in a different positional relationship for each user who listens to it. That is, one voice can be heard differently for each user.

  For this reason, the server apparatus 11 can further be provided with the processing means 43 for processing frequency domain data, as shown in FIG. For the above noise removal, a low-pass filter that removes frequency components higher than a predetermined frequency and a high-pass filter that removes frequency components lower than a predetermined frequency can be used. In the above-mentioned voice change, the timbre of the sound can be changed by raising or lowering the specific frequency of the sound. Further, it is possible to change the frequency characteristics using an equalizer that operates the level for each frequency band.

  Since many sound effects are applied to frequency domain data, in the past, the time domain data is once converted to frequency domain data, the effect is applied, and the time domain data is converted again to mix the audio. Had been sent to. However, in this method, since the client apparatus 10 converts the frequency domain data into the frequency domain data and transmits the data to the server apparatus 11, the client apparatus 10 directly applies the effect to the frequency domain data before transmission and returns the time domain data to the server apparatus 11. Can be sent. For this reason, it is possible to apply a sound effect for manipulating the frequency to the sound with a small amount of calculation.

  The system, method, and program of the present invention have been described in detail so far. However, the present invention is not limited to the above-described embodiments, and those skilled in the art can add other embodiments, additions, modifications, deletions, and the like. It can be changed within the range that can be conceived, and any aspect is included in the scope of the present invention as long as the effects and effects of the present invention are exhibited.

  Accordingly, it is possible to provide a recording medium such as a CD-ROM or SD card in which the above program is recorded, a program providing server that holds the program, and provides the program in response to a download request.

DESCRIPTION OF SYMBOLS 10, 10a-10n ... Client apparatus, 11 ... Server apparatus, 12 ... Network, 20 ... CPU, 21 ... ROM, 22 ... RAM, 23 ... HDD, 24 ... Communication I / F, 25 ... Input / output I / F, 26 ... voice input device, 27 ... voice output device, 28 ... bus, 30 ... voice acquisition means, 31 ... conversion means (first conversion means, second conversion means), 32 ... transmission means, 33 ... reception means, 34 ... voice Output means, 40 ... receiving means, 41 ... mixing means, 42 ... distributing means, 43 ... processing means

Claims (12)

A voice call system including a plurality of client devices and server devices,
The client device is
Voice acquisition means for acquiring voice data as time-domain data representing a change in sound pressure with respect to time;
Sampling the time-domain data acquired by the voice acquisition unit, dividing the block into blocks having a fixed number of samples, and converting the frequency-domain data representing the strength of each frequency component into divided blocks; ,
Transmitting means for transmitting the frequency domain data converted by the converting means to the server device;
Receiving means for receiving mixed data generated by mixing a plurality of frequency domain data from the server device;
And a second conversion means for converting the mixed data received by the receiving means into time domain data.   2. The voice call system according to claim 1, wherein the transmission unit transmits the frequency domain data with user identification information for identifying a user or device identification information for identifying the client device.   The voice call system according to claim 1, further comprising voice output means for outputting the time domain data converted by the conversion means by voice.   The voice call system according to any one of claims 1 to 3, wherein the conversion means converts the time domain data or the frequency domain data by overlapping orthogonal transformation.   5. The voice call system according to claim 1, wherein the conversion unit encodes the converted frequency domain data and decodes the mixed data encoded in the server device. 6. The server device generates the mixed data by mixing frequency domain data received from all of the plurality of client devices, and transmits the mixed data to the client device,
6. The client device according to claim 1, further comprising an adding unit that inverts and adds the phase of the frequency domain data transmitted by the client device to the received mixed data. Voice call system.   The server device includes: a receiving unit that receives the frequency domain data from the client device; a mixing unit that mixes the plurality of frequency domain data received by the receiving unit to generate mixed data; and the mixing unit The voice call system according to claim 1, further comprising a distribution unit that distributes the generated mixed data to the client device.   The mixing unit generates the mixed data for the client device by mixing the frequency domain data received from all other client devices with one client device to generate mixed data, and the distribution unit The voice call system according to claim 7, wherein the mixed data generated by the mixing unit is distributed to the client device.   The voice call system according to claim 7 or 8, further comprising processing means for processing the frequency domain data.   10. The audio according to claim 7, wherein the reception unit decodes the frequency domain data encoded in the client device, and the distribution unit encodes and distributes the mixed data. Call system. A voice call method implemented in a voice call system including a plurality of client devices and a server device,
A plurality of client devices acquiring audio data as time domain data representing a change in sound pressure with respect to time;
A plurality of the client devices sample the time-domain data, divide the data into blocks having a fixed number of samples, and convert the frequency-domain data representing the strength of each frequency component in divided blocks;
A plurality of client devices transmitting the frequency domain data to the server device;
The server device receiving a plurality of the frequency domain data;
The server device generates the mixed data for the client device by mixing the frequency domain data received from all other client devices with one client device to generate mixed data;
The server device delivering the mixed data to the client device;
The client device receiving the mixed data;
The client device converting the mixed data into time domain data;
A voice communication method comprising: a step of outputting the time domain data by voice as the client device. A program for executing voice call processing in a voice call system including a plurality of client devices and server devices,
A plurality of client devices;
Obtaining audio data as time domain data representing a change in sound pressure with respect to time;
Sampling the time domain data, dividing the sample into blocks having a fixed number of samples, and converting the frequency domain data representing the strength of each frequency component in divided blocks;
A plurality of the client devices transmitting the frequency domain data to the server device;
In the server device,
Receiving a plurality of said frequency domain data;
Generating the mixed data for the client device by mixing the frequency domain data received from all the other client devices with one client device to generate mixed data;
Delivering the mixed data to the client device;
In the client device,
Receiving the mixed data;
Converting the mixed data into time domain data;
Executing the step of outputting the time domain data as a voice.