JP2020098961A - Information processing apparatus, control method thereof, and program - Google Patents

Abstract

To provide a mechanism in which the number of buffers can be adjusted according to a communication environment and sound can be played back smoothly.SOLUTION: An information processing apparatus according to the present invention accumulates received first sound data and second sound data, and deletes the second sound data when the accumulated first sound data and second sound data are accumulated so as to be equal to or more than a predetermined value. Then, the information processing apparatus adjusts the predetermined value and adjusts the predetermined value for deleting the second sound data when the accumulated first sound data and second sound data satisfy a predetermined condition.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information processing device, a control method thereof, and a program.

In a video conference system, a technique for providing a voice receiving terminal with a buffer in order to cope with communication fluctuations is generally performed. When there is no buffer, that is, when the received voice is played back as it is, a short time voice break is heard as noise due to communication fluctuation, and smooth voice playback cannot be performed. Therefore, since a temporarily stored voice is played back by having a buffer at the time of playing a voice, even if a communication fluctuation occurs, a short time voice break is eliminated and a high quality voice can be played back.

However, if the buffer size is too large, the reproduction is not performed until the voice is accumulated, and thus the reproduction delay increases and smooth communication is hindered. The optimum buffer size is the minimum buffer size that does not exhaust the buffer due to communication fluctuations. However, since the degree of communication fluctuation varies depending on the communication environment, the optimum value of the number of buffers varies depending on the communication environment. Therefore, the number of buffers needs to be set individually, but it is very troublesome for the user.

Therefore, Patent Document 1 discloses a mechanism for returning the allowable storage amount of the jitter absorption buffer increased when the communication network is congested to the default value based on the monitoring result of the delay time and the data loss rate. ing.

JP, 2012-124689, A

In Patent Document 1, when the congestion of the communication network is considered to be recovered, the allowable storage amount of the jitter absorption buffer is returned to the default value, so the value after the number of buffers is reduced is fixed to the default value, and is adjusted to the communication environment. Is not set to a value that Therefore, it is difficult to improve the voice quality when the default value is not suitable for the communication environment.

Therefore, a function of adjusting the number of buffers to an optimum value according to the communication environment is desired.

Therefore, it is an object of the present invention to provide a mechanism capable of adjusting the number of buffers according to the communication environment and smoothly reproducing audio.

An information processing apparatus of the present invention is a storage unit for storing received first voice data and second voice data, the first voice data and the second voice data stored in the storage unit. When a predetermined value or more is stored, the deleting means for deleting the second voice data, the first voice data and the second voice data accumulated by the accumulating means, when a predetermined condition is satisfied, Adjusting means for adjusting the predetermined value.

According to the present invention, the number of buffers can be adjusted according to the communication environment, and there is an effect that audio can be smoothly reproduced.

It is a figure showing an example of a system configuration of an information processing system in an embodiment of the present invention. It is a figure which shows an example of the hardware constitutions of the web conference server 100, the information processing apparatus 101, and the information processing apparatus 102 in embodiment of this invention. It is a figure which shows an example of a functional structure of the web conference server 100 and the information processing apparatus 101 in embodiment of this invention. It is a flow chart of processing at the time of voice data reception in an embodiment of the present invention. 6 is a flowchart of audio data reproduction processing according to the embodiment of the present invention. It is an example of the figure of the buffer field in the embodiment of the present invention. 6 is an example of an operation at the time of discarding silence of a buffer in the embodiment of the present invention. 5 is an example of a voice data structure table 800 and a buffer number transition table 810 according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is an example of a diagram showing a system configuration according to an embodiment of the present invention. The web conference server 100, the information processing apparatus 101 (host computer), and the information processing apparatus 102 (guest computer) can be connected via a network. In the present embodiment, the information processing apparatus 101 is used as a host computer and the information processing apparatus 102 is used as a guest computer, but any information processing apparatus may play any role (host computer, guest computer). In addition, since the hardware configurations of the information processing apparatus 101 and the information processing apparatus 102 are the same as those of yesterday, the following description will focus on the information processing apparatus 101.

In the information processing system (web conference system) of FIG. 1, one of the users participating in the conference becomes the host computer (the information processing device 101 in this embodiment), and the information processing device 101 accesses the web conference server 100. Then book a meeting room. The conference room is a virtual conference space, and only the participants invited by the host computer can enter the room. Further, as a free space, an unspecified user may participate, or may not be able to speak but may only be able to view.

The user of the information processing apparatus 101 determines the conference ID (or conference room ID, room number, etc.) that identifies the conference, the time when the conference room is used, and the like. When inviting a specific participant, the participant's notification destination (e.g., mail address) registered in the web conference server 100 may be used to call for participation. In order to allow only a specific user to participate, the information processing apparatus 101 can describe the password for entering the conference room in the notification.

The voices and images acquired by the information processing device 101 and the information processing device 102 are transmitted to the web conference server 100 as voice data by a microphone (not shown) or image data by a camera (not shown). For example, the voice is transmitted from each information processing apparatus 101 to the web conference server 100 as upstream voice data for the number of information processing apparatuses 101.

The above-described upstream voice data is mixed by the web conference server 100 with the upstream voice data received from the plurality of information processing devices 101 and distributed to another information processing device 102 (downstream voice data). However, it is not necessary to mix the audio data of the information processing apparatus 101 itself with the downlink audio data sent back to the information processing apparatus 101 that has transmitted certain upstream audio data upon mixing.

Although the information processing apparatus 101 and the web conference server 100 are described as separate housings here, one certain information processing apparatus 101 may be configured so that the functions of the web conference server 100 are configured in the same housing. Is also good.

In the embodiment of the present invention, the web conference system will be described as an example, but the present invention is not limited to the conference system. For example, the present invention provides a technique that can be used when voice data is exchanged via a network and packet (data including voice data) delay occurs due to network delay or the like.

FIG. 2 is an example of a diagram showing a hardware configuration according to the embodiment of the present invention. As illustrated in FIG. 2, the web conference server 100, the information processing apparatus 101, and the information processing apparatus 102 include a CPU (Central Processing Unit) 201, a RAM (Random Access Memory) 202, and a ROM (Read Only Memory) via a system bus 204. ) 203, input controller 205, video controller 206, memory controller 207, communication I/F controller 208, etc. are connected.

The CPU 201 centrally controls each device and controller connected to the system bus 204.

Further, the ROM 203 or the external memory 211 will be described later, which is necessary for realizing the functions executed by the control programs of the CPU 201, such as BIOS (Basic Input/Output System), OS (Operating System), and each server or each PC. Various programs are stored. In addition, information necessary for carrying out the present invention is stored. The external memory may be a database.

The RAM 202 functions as a main memory, a work area, etc. of the CPU 201. The CPU 201 loads various programs necessary for executing the processing from the ROM 203 or the external memory 211 into the RAM 202, and executes the loaded programs to realize various operations.

The input controller 205 controls inputs from a keyboard (KB) 209 and a pointing device such as a mouse (not shown).

The video controller 206 controls display on a display device such as the display 210. The display may be a display such as a liquid crystal display. These are used by the administrator as needed.

The memory controller 207 is an external storage device (hard disk (HD)) that stores a boot program, various applications, font data, user files, edit files, various data, or the like, a flexible disk (FD), or a PCMCIA (Personal Computer). Controls access to an external memory 211 such as a CompactFlash (registered trademark) memory that is connected to a Memory Card International Association) card slot via an adapter.

The communication I/F controller 208 connects/communicates with an external device via a network and executes a communication control process on the network. For example, communication using TCP/IP (Transmission Control Protocol/Internet Protocol) is possible.

The CPU 201 can display on the display 210, for example, by executing an outline font rasterizing process in the display information area in the RAM 202. Further, the CPU 201 enables a user instruction with a mouse cursor (not shown) or the like on the display 210.

Various programs to be described later for implementing the present invention are recorded in the external memory 211 and are executed by the CPU 201 by being loaded into the RAM 202 as necessary. Further, the definition file and various information tables used when the above program is executed are also stored in the external memory 211.

FIG. 3 is an example of a diagram showing a functional configuration of software of the web conference server 100 and the information processing apparatus 101 according to the embodiment of the present invention.

The web conference server 100 includes a voice data transmission unit 301.

The audio data transmitting unit 301 is a functional unit that mixes the audio data received by the web conference server 100 from each information processing apparatus 101 and then transmits the mixed audio data to each information processing apparatus 101.

The information processing device 101 includes an audio data reception unit 311, a decoding unit 312, a buffer management unit 313, a storage unit 314, and a reproduction processing unit 315.

The voice data receiving unit 311 is a functional unit that receives voice data from the web conference server 100.

The decoding unit 312 is a functional unit that decodes the audio data received by the audio data receiving unit 311.

The buffer management unit 313 is a functional unit that manages a buffer of audio data received by the information processing apparatus 101. In the present embodiment, the buffer management unit 313 adjusts the number of buffers of audio data accumulated in the buffer area and the threshold value for discarding silent audio data (hereinafter, silent data) (hereinafter, silent discard threshold) according to the communication environment. By doing so, appropriate audio data reproduction processing is performed.

The storage unit 314 is a functional unit that stores various data tables such as the transition table 810 of the number of buffers managed by the buffer management unit 313.

The reproduction processing unit 315 is a functional unit that reproduces the audio data received from the web conference server 100. In the present embodiment, the voice data received from the web conference server 100 is temporarily stored as a buffer, and the reproduction process is performed after a certain number of voice data are stored. As a result, even if the communication environment is bad and the communication fluctuates, a certain number of accumulated voice data is reproduced, so that the reproduced voice data is not chopped and is heard as noise.

Next, with reference to FIG. 4, a series of flow of processing at the time of receiving voice data in the embodiment of the present invention will be described. FIG. 4 is an example of a process in which the received audio data is managed as a buffer.

In step S 401, the CPU 201 of the web conference server 100 transmits audio data to the information processing apparatus 101.

In step S402, the CPU 201 of the information processing apparatus 101 receives audio data from the web conference server 100.

In step S403, the CPU 201 of the information processing apparatus 101 decodes the received audio data.

In step S404, the CPU 201 of the information processing apparatus 101 determines whether the received voice data is silent data. The silence data refers to data of sounds that are composed only of environmental sounds and noise that are not voices. When the CPU 201 of the information processing device 101 determines that the audio data is silent, the process proceeds to step S405. If it is determined that the audio data is silent, the process proceeds to step S406. It should be noted that the silent data mentioned here does not necessarily need to be silent, and may be determined as silent data as long as it does not include voice data at a certain value.

Here, the structure table 800 of the audio data in FIG. 8 will be described. FIG. 8 is an example of a structure table 800 of audio data. The audio data structure table 800 includes information items 801 and type items 802. For example, when media data is stored in the information item 801, it is stored in the type item 802 in a byte array. When a silent flag is stored in the information item 801, a boolean or false is stored as a Boolean type in the type item 802.

In step S405 of FIG. 4, the CPU 201 of the information processing apparatus 101 substitutes false into the silence flag item of the audio data of FIG. 8 stored by the function of the storage unit 314.

In step S406, the CPU 201 of the information processing apparatus 101 assigns true to the silence flag item of the audio data of FIG.

In step S407, the CPU 201 of the information processing apparatus 101 adds the received audio data to the buffer area 600.

Here, FIG. 6 will be described. FIG. 6 is a diagram showing an example of the buffer area. The buffer area 600, which indicates the entire buffer area, can store up to 100 pieces of audio data. The normal voice 601 indicates that the silence flag accumulated in the buffer is false, that is, normal voice data. The silence 602 indicates that the silence flag stored in the buffer is true, that is, silence data. Reference numeral 603 denotes audio data to be added to the buffer area 600. When audio data is added, the audio data is added to the smallest area of the index 811 in the empty portion of the buffer area 600. Reference numeral 604 represents audio data extracted from the buffer area 600. The audio data to be extracted is audio data whose index 811 is 0. After taking out, the index 811 (described later) is renumbered as 1 to 0, 2 to 1,..., 100 to 99, 0 to 100. The silence discard threshold 607 indicates a reference threshold for deleting the silence 602. When the voice data including the normal voice 601 and the silence 602 accumulated in the buffer area 600 reaches or exceeds the silence discard threshold 607, the silence 602 in the buffer area 600 is discarded (deleted). Before silence cancellation 605 and after silence cancellation 606 are diagrams of buffer areas before and after discarding silent data. The voice data (silence 602) in which the silence flag is true before the silence is discarded 605 is discarded in the diagram after the silence is discarded 606. After the silent data is discarded, the index 811 is renumbered to the remaining voice data.

In step S408, the CPU 201 of the information processing apparatus 101 acquires the number of buffers from the buffer area 600. The acquired number of buffers is stored in the buffer number transition table 810.

The buffer number transition table 810 in FIG. 8 is a data table that records the number of buffers acquired in step S408 each time audio data is received. The buffer number transition table 810 includes an index 811 indicating a column in which an index is registered and a buffer number item 812 indicating a column in which a buffer number is registered. The initial value of the number of buffers is 0. The information processing apparatus 101 uses the index 811 No. By recording the number of buffers in order from 1, the information on the transition of the number of buffers is held.

In step S409, the CPU 201 of the information processing apparatus 101 compares the acquired number of buffers with the silent discard threshold 607. The silent discard threshold 607 is set when the user enters the virtual conference room of the web conference, and is stored in the external memory 211. If the number of buffers is equal to or larger than the value of the silent discard threshold 607, the CPU 201 of the information processing apparatus 101 advances the process to step S410. If the number of buffers is less than or equal to the silent discard threshold 607, the process ends.

In step S410, the CPU 201 of the information processing apparatus 101 scans all the audio data in the buffer area 600 and discards the audio data whose silent flag is true from the buffer area. (For the image of silent discard, refer to 605 before silent discard and 606 after silent discard in FIG. 6.) This shows a process of discarding silent data when the number of buffers is accumulated over a predetermined value. By this processing, by discarding the extra silent data, it is possible to recover the delay required for the reproduction of the audio data due to the temporary storage of the audio data by the function of the buffer.

If all the silent data is discarded from the beginning, there will be no sound during a conversation that does not contain any voice, and when the voice data is played back an unnatural voice will be played. I will end up. Therefore, as in the present embodiment, the silent data is also accumulated to some extent, and the silent data is discarded when a certain threshold value (predetermined value) is exceeded, so that the minimum silent data is accumulated and the number of buffers is reduced. When a certain number of data are accumulated, it is possible to efficiently discard a plurality of silent data. Furthermore, this processing makes it possible to reproduce the audio data with a small delay.

This is the end of the description of the flow of the process at the time of receiving voice data in FIG.

Next, with reference to FIG. 5, a series of flows of the process in which the information processing apparatus 101 reproduces audio data will be described.

In step S501, the CPU 201 of the information processing apparatus 101 acquires the number of buffers from the buffer area 600.

In step S502, the CPU 201 of the information processing apparatus 101 stores the number of buffers acquired in step S502 in the buffer number transition table 810 by the function of the storage unit 314. More specifically, the storage unit 314 moves to the next row after overwriting the value of the buffer number item of the buffer number transition table 810.

In step S503, the CPU 201 of the information processing apparatus 101 determines whether the number of buffers is 0. When the CPU 201 of the information processing device 101 determines that the number of buffers is 0, the process proceeds to step S504. When the CPU 201 of the information processing device 101 determines that the number of buffers is not 0, the process proceeds to step S505.

In step S504, the CPU 201 of the information processing apparatus 101 waits until audio data is added to the buffer area 600 and the number of buffers is accumulated.

In step S505, the CPU 201 of the information processing apparatus 101 scans and confirms the record of the buffer number transition of the buffer number transition table 810.

In step S506, the CPU 201 of the information processing apparatus 101 determines whether or not there is data with a buffer number item of 0 in the record of the buffer number transition confirmed in step S505. When the CPU 201 of the information processing device 101 determines that there is 0 data, the process proceeds to step S508. When the CPU 201 of the information processing device 101 determines that there is no 0 data, the process proceeds to step S507. In the present embodiment, it is determined whether or not the number-of-buffers item is 0, but it may be an arbitrary predetermined number instead of 0.

In step S507, the CPU 201 of the information processing apparatus 101 subtracts a fixed value from the silence discard threshold value 607 in the memory of the buffer area 600. The fixed value to be subtracted is set when the user enters the virtual conference room and is stored in the external memory 211. FIG. 7 is an image diagram in which a fixed value is subtracted from the silence discard threshold value 607.

In step S508, the CPU 201 of the information processing apparatus 101 acquires audio data from the buffer area 600. The acquired audio data is sent to a playback device (not shown) included in the information processing apparatus 101.

In step S509, the CPU 201 of the information processing device 101 reproduces the audio data by the function of the reproduction processing unit 315.

In step S503, the process of FIG. 5 determines whether or not there is data in which the number-of-buffers item is 0, thereby confirming whether or not there is room to reduce the buffer. If there is no data in which the number of buffer items is 0, it means that the silence discard threshold 607 can be subtracted because the voice data is not exhausted and the buffer has a margin. This makes it possible to adjust the optimum number of buffers in consideration of the communication environment and the reproduction status even when the buffer is exhausted and the allowable storage amount of the buffer area 600 and the upper limit of the silent discard threshold 607 are increased. Become. In other words, the silence discard threshold 607 can be subtracted depending on the communication environment, so that it is possible to perform processing for voice delay recovery by efficiently discarding silence data while accumulating a buffer in the buffer area.

Even if a certain amount of time has passed after raising the allowable storage amount of the buffer area 600 and the upper limit of the silent discard threshold 607, it is better not to subtract the silent discard threshold 607 if the number of buffers is not sufficient. That is, if there is no room in the number of buffers, it is better to accumulate silent data, and therefore the process of step S503 is necessary. When determining the exhaustion of the number of buffers, the value does not necessarily have to be 0, and may be determined based on whether a predetermined condition such as an arbitrary value is satisfied.

In addition, voice data accumulated on the side of the web conference server 100 due to fluctuations in the communication environment may be received by the information processing apparatus 101 at a stroke due to improvement in the communication environment. In that case, a large amount of audio data will be accumulated in the buffer area 600. In this case, even if the silent data is deleted little by little on a fixed basis, the voice data only accumulates. Therefore, according to the present embodiment, the silent data can be efficiently discarded at one time. Furthermore, the present invention has an effect even in the case where the reproduction driver (not shown) of the information processing apparatus 101 is not able to reproduce well and the audio data is accumulated in the buffer area 600.

Although the silent discard threshold 607 is subtracted in the present embodiment, the silent discard threshold 607 may be increased according to the communication environment and the number of buffer items accumulated in the buffer area 600.

The silent discard threshold 607 may be a default value or a fixed value. If the silent discard threshold 607 is set to a fixed value that can be changed according to the communication environment, it is possible to adjust the value to match the communication environment, and thus it is possible to adjust the optimum number of buffers and the reproduced voice quality. Leads to improvement of.

This is the end of the description of the flow of the audio data reproduction process in FIG.

As described above, according to the present invention, the number of buffers can be adjusted according to the communication environment, and the effect of smoothly reproducing audio can be obtained.

In the embodiment of the present invention, the information processing apparatus 101, the other information processing apparatus 102, and the web conference server 100 have been described as an example. However, guests other than the host computer (information processing apparatus 101) participating in the web conference may participate. There may be a plurality of computers.

Although the present invention has been described by taking the web conference as an example in the above-described embodiment, the present invention can be applied to, for example, a video conference, a video telephone, and the like. A conference that uses electronic data such as a web conference, a video conference, and a videophone is called an electronic conference.

As described above, the present invention can be embodied as a system, a device, a method, a program, a recording medium, or the like. Specifically, it may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of one device.

Further, the program in the present invention is a computer-executable program for executing the processing method of the flowchart shown in this embodiment, and the computer-executable program is stored in the storage medium of the present invention. The program in the present invention may be a program for each processing method of each device.

As described above, the recording medium recording the program that realizes the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

A recording medium for supplying the program includes, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a DVD-ROM, a magnetic tape, a non-volatile memory card, a ROM, an EEPROM, and a silicon. A disk or the like can be used.

Further, not only the functions of the above-described embodiments are realized by executing the program read by the computer, but also the OS (operating system) or the like running on the computer is actually executed based on the instructions of the program. It goes without saying that a case where some or all of the processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

Furthermore, after the program read from the recording medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function expansion board is instructed based on the instruction of the program code. Needless to say, this also includes a case where the CPU or the like included in the function expansion unit performs some or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

Further, the present invention may be applied to a system including a plurality of devices or an apparatus including a single device. Further, it goes without saying that the present invention can be applied to the case where it is achieved by supplying a program to a system or an apparatus. In this case, by reading the recording medium storing the program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

Further, by downloading and reading a program for achieving the present invention from a server, a database or the like on a network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that the present invention also includes all configurations that combine the above-described embodiments and modifications thereof.

100 Web conference server 101 Information processing device 102 Information processing device 201 CPU
202 RAM
203 ROM
208 Communication I/F controller 211 External memory

Claims (6)

Storage means for storing the received first voice data and second voice data,
Deletion means for deleting the second voice data when the first voice data and the second voice data accumulated by the storage means are accumulated more than a predetermined value,
Adjusting means for adjusting the predetermined value when the first voice data and the second voice data accumulated by the accumulating means satisfy a predetermined condition,
An information processing apparatus comprising: The information processing apparatus according to claim 1, wherein the first voice data is voice data including voice, and the second voice data is voice data indicating that voice is not included. Reproduction means for reproducing the first audio data and the second audio data accumulated by the accumulating means,
Further equipped with,
The information processing apparatus according to claim 1 or 2, wherein the reproducing means reproduces the first audio data when the second audio data is deleted by the deleting means. 4. The adjusting unit subtracts the predetermined value when the first sound data and the second sound data satisfy the predetermined condition. The information processing device according to 1. A method of controlling an information processing device, comprising:
A storage step of storing the received first voice data and second voice data,
A deletion step of deleting the second voice data when the first voice data and the second voice data accumulated in the accumulation step have accumulated a predetermined value or more,
An adjustment step of adjusting the predetermined value when the first voice data and the second voice data accumulated in the accumulation step satisfy a predetermined condition,
A method for controlling an information processing device, comprising: A program for causing a computer to function as each unit of the information processing apparatus according to claim 1.

Images