JP2019117998A - Web conference system, control method of web conference system, and program - Google Patents

Abstract

To provide a web conference system capable of changing a communication method according to the amount of audio noise of a client terminal.SOLUTION: A web conference system capable of displaying a plurality of captured images captured by client terminals connected to a plurality of bases includes a voice acquisition means for acquiring voice of a client terminal, a noise acquisition means for acquiring the amount of noise included in the voice acquired by the audio acquisition means, and a display control means for displaying a text box that enables inputting of a text to be transmitted to another client terminal on a display unit of a client terminal of an acquisition source of the voice including the noise amount when the amount of noise acquired by the noise acquisition means is larger than a threshold value.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a web conference system using a client terminal connected to a plurality of bases, a control method of the web conference system, and a program.

  2. Description of the Related Art Conventionally, there is known a web conference system that performs virtual conferences using a plurality of terminals connected via a network. Because multiple terminals are located at different locations, the conference environment varies from location to location. For example, there are bases where meetings are conducted in quiet conference rooms, and bases where meetings are performed in a noisy shared space or the like.

  Patent Document 1 discloses a teleconference system which transmits text messages to a plurality of devices. By exchanging text messages as in the technology described in Patent Document 1, it becomes possible to hold a conference without worrying about noise.

Japanese Patent Application Publication No. 2006-528804

  However, as in the case of the teleconferencing system described in Patent Document 1, when the text of the discussion of the conference is exchanged, it takes time and effort to enter the text, so it is not suitable for real-time discussions. On the other hand, it has been difficult to remove noise on the assumption of the communication environment between all bases and bases.

  Then, this invention is made in view of said subject, and an object of this invention is to provide the web conference system which can select a communication method easily according to the noise amount of the audio | voice acquired from the client terminal.

  In order to achieve the above object, the present invention is a web conference system capable of displaying a plurality of photographed images photographed by a client terminal connected to a plurality of bases, and acquires audio of the client terminal. Voice acquiring means, noise acquiring means for acquiring the noise amount included in the audio acquired by the audio acquiring means, and the noise including the noise amount when the noise amount acquired by the noise acquiring means is larger than a threshold And a display control unit configured to display a text box capable of inputting a text to be transmitted to another client terminal on a display unit of a client terminal of an acquisition source.

  According to the present invention, it is possible to provide a web conference system capable of easily selecting a communication method according to the noise amount of voice acquired from a client terminal.

It is a figure which shows an example of the system configuration | structure figure of the web meeting system concerning embodiment of this invention. It is a block diagram showing an example of hardware constitutions of an information processor in a web meeting system concerning an embodiment of the present invention. It is a functional block diagram which shows an example of a functional structure in the web meeting system which concerns on embodiment of this invention. It is a flowchart which shows an example of the control process of the web meeting server in the web meeting system which concerns on embodiment of this invention. It is a figure which shows an example of the database of the web meeting system which concerns on embodiment of this invention. It is a figure which shows an example of the display screen of each client terminal in the web conference system which concerns on embodiment of this invention. It is a figure which shows an example of the display screen of the client terminal of the speaker in the web conference system which concerns on embodiment of this invention. It is a figure which shows an example of the display screen of client terminals other than the speaker in the web conference system which concerns on embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing an example of a system configuration diagram of a web conference system according to the present embodiment.

  In the web conference system of the present invention, for example, a web conference server (server apparatus) 200 that makes a web conference system available between client terminals 100 (300, 310) applicable to personal computers is connected via the network 110. It is done. The web conferencing system is also referred to as a web conferencing system and a video conferencing system. The client terminals 100 (300, 310) function as client terminals of the web conference server (server apparatus) 200. The web conference system can display an image (still image or moving image) captured by each client terminal connected to a plurality of bases on each of the plurality of client terminals. In addition, the web conference system can output voices emitted by each client terminal connected to a plurality of bases to each of the plurality of client terminals. The client terminal 100 is an aspect of an information processing apparatus represented by a PC or the like.

  Each device may be connected to the network 110 by a wired network, or may be connected by a wireless network or wireless communication.

  The web conference server 200 is a server for realizing a web conference using a web browser. The client terminal 100 can obtain a web conference screen (interface) and conduct a web conference by accessing the web conference server 200. The web conference server 200 includes a control circuit such as a CPU and a storage area such as a RAM and a ROM. Also, the web conference server 200 includes a communication circuit for communicating with each client terminal 100.

  In this embodiment, a tablet PC is described as an example of the client terminal 100 or 300, but a notebook PC such as the information processing apparatus 310 or a desktop PC (not shown) may be used. The information processing apparatus can display an image of content distributed from the web conference server in a predetermined display area.

  Content to be distributed (video, audio, volume, etc.) is an image or audio acquired by the camera 101 or the microphone 102, and is transmitted from the information processing apparatus to the web conference server at predetermined intervals. The web conference server 200 storing these pieces of information is distributed to the connected information processing apparatus.

  In these client terminals 100, a web browser for accessing the web conference server 200 and a dedicated module are installed. Note that this dedicated module is, for example, an ActiveX component downloaded from the web conference server 200 via a web browser.

  In addition, when transmitting a moving image of oneself in a web conference, the camera 101 such as a web camera which is a video device is connected. In the case of transmitting voice, the microphone 102 is connected, and in order to view the voice of the other party, the speaker 103 is connected. The microphone 102 and the speaker 103 are audio devices. For example, in the present embodiment, it is possible to determine a base to be focused on using changes in sound collected by the microphone 102.

  In addition, some devices are provided in the same housing hardware, and for example, there are devices provided with a camera function and a microphone function, a microphone function and a speaker function, and all functions.

  In addition, the web conference system can perform screen sharing in which the screen of the client terminal 100 is displayed on the screen of another information processing apparatus (300 or 310). The screen to be shared can be designated by the client terminal 100, and may be the screen of the designated application or the entire desktop.

  The present web conference system can set a plurality of virtual conference rooms on the web called a room, and can set a plurality of rooms to perform a plurality of conferences.

  FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus according to the embodiment of the present invention.

  As shown in FIG. 2, in the client terminal 100, a central processing unit (CPU) 201, a read only memory (ROM) 202, a random access memory (RAM) 203, a storage device 204, an input controller 205, and a system bus 215. A voice input / output controller 206, a video controller 207, a memory controller 208, and a communication I / F controller 209 are connected.

  The CPU 201 centrally controls the devices and controllers connected to the system bus 215.

  The ROM 202 or the storage device 204 is a control program executed by the CPU 201, such as a BIOS (Basic Input / Output System) or an OS (Operating System), a computer readable and executable program for realizing the information processing method, and various necessary items. It holds data (including data tables).

  The RAM 203 functions as a main memory, a work area, and the like of the CPU 201. The CPU 201 loads programs and the like necessary for execution of processing from the ROM 202 or the storage device 204 to the RAM 203, and implements various operations by executing the loaded programs.

  The input controller 205 controls input from an input device 210 such as a keyboard / touch panel / camera. The input device is not limited to this, and may be a touch panel capable of detecting a position touched with a plurality of fingers, such as a mouse or a multi touch screen.

  The audio input / output controller 206 controls input / output from an audio input / output device 211 such as a microphone / speaker. It is possible to recognize speech input from the microphone. It is also possible to connect an earphone to the speaker.

  The video controller 207 controls display on an external output device such as the display 212. The display shall also include the display of a notebook computer integrated with the main unit. The external output device is not limited to the display, and may be, for example, a projector. Moreover, it becomes possible to receive the input from a keyboard / touch panel about the apparatus which can be received by the above-mentioned touch operation.

  The video controller 207 can control a video memory (VRAM) for display control, and can use a part of the RAM 203 as a video memory area, or can separately provide a dedicated video memory. It is possible.

  In the present invention, the first video memory area used for display when the user normally operates the information processing apparatus and the display content superimposed on the display content of the first video memory area when the predetermined screen is displayed And a second video memory area used for The video memory area is not limited to two, and can be plural as long as the resource of the information processing apparatus permits.

  The memory controller 208 controls access to the external memory 213. The external memory is connected via an adapter to an external storage device (hard disk) that stores a boot program, various applications, font data, user files, editing files, various data, etc., a flexible disk (FD), or a PCMCIA card slot. Compact flash (registered trademark) memory etc. can be used.

  The communication I / F controller 209 connects and communicates with an external device via the network 214, and executes communication control processing in the network. For example, communications using TCP / IP, telephone lines such as ISDN, and 3G lines of mobile phones are possible.

  The storage device 204 is a medium for permanently storing information, and the form thereof is not limited to a storage device such as a hard disk. For example, a medium such as a solid state drive (SSD) may be used.

  Moreover, it can also be used as a temporary memory area at the time of various processes performed by the communication terminal in the present embodiment.

  FIG. 3 is a functional configuration diagram showing an example of a functional configuration in the web conference system according to the present embodiment. Each function is a function executed by a CPU or the like included in the web conference server 200.

  The web conference server 200 of the web conference system includes a speaker identification unit 311, a captured image acquisition unit 312, a voice acquisition unit 313, and a noise acquisition unit 314. Furthermore, the web conference server 200 includes a display control unit 315, an audio output control unit 316, a conference mode setting unit 317, and a storage unit 318.

  The speaker specifying unit 311 specifies a client terminal in which a person who is speaking is present. Specifically, when voice is acquired from a client terminal participating in a conference, the speaker identifying unit 311 can identify the client terminal in which a person who is speaking is present. For example, when detecting that a person using the client terminal has made a sound, the speaker specifying unit 311 can specify that the client terminal is a client terminal in which a person who is speaking is present. The method for specifying the client terminal by the speaker specifying unit 311 is not limited to these. For example, the speaker identifying unit 311 may identify from the action of the person appearing in the captured image. In addition, the speaker identifying unit 311 may receive a signal other than voice indicating that the user is speaking, prior to speaking from the client terminal, and may specify the information based on the information.

  The captured image acquisition unit 312 acquires a captured image captured by a client terminal participating in a web conference. The captured image acquisition unit 312 acquires a moving image that is a captured image captured continuously.

  The voice acquisition unit 313 acquires the voice acquired by the client terminal participating in the web conference. In addition, the voice acquisition unit 313 can obtain not only voice but also noise and noise emitted at a site on the client terminal side.

  The noise acquisition unit 314 acquires the amount of noise included in the sound acquired by the sound acquisition unit 313. Here, the noise amount is the amount of noise. Also, the noise may include various components such as howling noise, switching noise, and quantization noise. Howling noise is noise that occurs as the sound of the pickup of the microphone is circulated. Switching noise is noise caused by the switching power supply of the device. Quantization noise is noise at the time of converting speech into a digital signal. In addition, ambient sound other than voice at the time of utterance is also an aspect of noise. Then, the noise acquiring unit 314 has a determination function of determining whether the acquired noise amount is larger than a preset threshold. Here, a predetermined SN ratio (signal-to-noise ratio) is used as the threshold.

  The display control unit 315 displays the photographed image acquired from each client terminal on the client terminal participating in the conference. Furthermore, when the noise amount acquired by the noise acquisition unit 314 is larger than the threshold, the display control unit 315 displays a text box on the display unit of the client terminal that is the acquisition source of the voice including the noise amount. The text box is an aspect of a display item capable of inputting text to be transmitted to another client terminal. Furthermore, the display control unit 315 displays a warning display together with the text box. Here, the warning display may be a display indicating that noise is detected at least a predetermined amount. The display control unit 315 can alert the speaker by performing the display. The display mode of the conference screen on the speaker side including the text box and the warning display will be described later with reference to FIG. Further, when a text is input in the text box, the display control unit 315 performs a display indicating that the text is being input to the client terminal 100 other than the speaker. Therefore, the participant of the client terminal 100 having the display can understand that another participant is going to talk by text input. In addition, the display control unit 315 can display the content of the input text on another client terminal 100. In this case, the display control unit 315 can also display an image (enlarged image) obtained by enlarging the video displayed on the client terminal on another client terminal. The display mode of the conference screen other than the speaker will be described later with reference to FIG.

  The voice output control unit 316 outputs the voice acquired by the voice acquisition unit 313 to the client terminal 100.

  The conference mode setting unit 317 sets the conference mode of each client terminal. The participants of each meeting can select the meeting mode in advance from the participation screen (the meeting reservation screen 600 of FIG. 6) on the client terminal. The conference mode setting unit 317 stores the acquired setting information in the storage unit 318 of the web conference server as a database (conference mode database 500 in FIG. 5). Here, the conference mode indicates a mode of display or audio output to each client terminal in the conference. The noise determination mode is a mode in which determination based on the amount of noise acquired by the noise acquisition unit 314 is performed. Even in the case where it is known in advance by a noise determination mode that it is a noisy base, it is possible to participate in the conference. The normal mode is a mode in which the above-described determination by the noise acquisition unit 314 is not performed.

  The storage unit 318 is a functional unit that stores each database shown in FIG. 5 and manages these.

  In the present embodiment, the web conference server is described as having the functions shown in the functional configuration diagram, but some functions may be executed by each client terminal, or a plurality of servers may cooperate. You may make it function.

  Here, an example of a database in the web conference system according to the present embodiment will be described with reference to FIG.

  The conference mode database 500 is information in which a room ID 501, a user ID 502, and a conference mode 503 are associated. The room ID 501 is information for identifying a conference held in the web conference system. The user ID 502 is information for identifying a participant of the web conference system. The conference mode 503 is information for identifying a conference mode set by the user. If FIG. 5 is demonstrated to an example, room ID 501 will be "100,000", and user ID 502 will show that each user of "100000" and "100001" participates in the same web meeting. In addition, the user with the user ID 502 “100000” participates in the meeting in the “noise determination mode”, and the user with the user ID 502 “100001” participates in the meeting in the “normal mode”.

  The meeting management database 510 is information in which a room ID 511, a room name 512 which is the name of a meeting, and a use time 513 which is the use time thereof are associated with each other.

  The user management database 520 is information in which a user ID 521, a user name 522 indicating the name of a user, and a group name 523 to which the user belongs are associated.

  Next, control processing in the web conference system of the present embodiment will be described using FIG. FIG. 4 is a flowchart showing an example of control processing in the web conference system according to the embodiment of the present invention.

  In S401, the web conference server 200 receives conference participant information. Here, the participant specifically indicates the client terminal 100 operated by each user. The participant information is a request for participation of the user registered in the user management database 520. Further, the participant information is information indicating which of the noise determination mode and the normal mode the conference mode participates in. In order to receive the participant information, the web conference server 200 receives the information input by the user from the conference reservation screen 600 (FIG. 6) displayed on the client terminal. Here, the conference reservation screen 600 will be described with reference to FIG. The conference reservation screen 600 is a screen displayed by the web conference server 200 on each client terminal. The conference reservation screen 600 includes a join button (601, 602) and a noise determination mode selection icon (603, 604). The join button (601, 602) is a button set for each conference, and when the selection is accepted, the client terminal requests the web conference server to participate. The noise determination mode selection icons (603, 604) are icons for selecting whether to participate in the noise determination mode. The noise determination mode selection icon 603 is a case where participation is selected in the noise determination mode, and the noise determination mode selection icon 604 shows a case in which it does not participate in the noise determination mode but participates in the normal mode.

  In S402, the web conference server 200 stores the conference mode of each participant. Specifically, the web conference server 200 stores in the conference mode database 500 whether each participant is in the noise determination mode or in the normal mode.

  In S403, the web conference server 200 transmits the conference screen to the client terminal of the participant. Here, a screen when a conference is performed will be described using the conference screen 610 of FIG. The conference screen 610 in FIG. 6 is a screen displayed on the client terminal 100 that has entered the conference room set as the web conference room (virtual conference room). In the present embodiment, a total of five users participate in the web conference at four sites (User 001 to 004) besides themselves. The conference screen 610 can further display its own captured image. The web conference server 200 displays an image captured by the camera of each client terminal in the image display area 613. The screen layout of the conference screen 610 in FIG. 6 is a layout in which four image display areas 613 are arranged. The participant name display unit 612 displays the name of the transmission source of the image displayed in the image display area 613. For example, the user name of the user who has logged in to the web conference server 200 on the client terminal 100 that is the transmission source of the image is displayed. In the button display area 611, a “document sharing” button for starting sharing of the designated image, a “exit” button 614 for exiting the meeting (exiting from the meeting room), and the like are displayed.

  In S404, the web conference server 200 receives the photographed image and the sound from each client terminal.

  In S405, the web conference server 200 determines whether there is a participant in the noise determination mode. When there is no participant in the noise determination mode, the web conference server 200 executes the process of S406. On the other hand, when there is a participant in the noise determination mode, the web conference server 200 executes the process of S407.

  In step S406, the web conference server 200 transmits each captured image and sound received in step S404 to the client terminal of the participant. The web conference server 200 repeats the process while the conference continues.

  In step S407, the web conference server 200 identifies a client terminal in which the speaker is present.

  In S408, the web conference server 200 acquires the noise amount from the voice transmitted from the client terminal specified in S407. When the noise amount is larger than the threshold, the web conference server 200 executes the process of S409. On the other hand, when the noise amount is equal to or less than the threshold, the web conference server 200 executes the process of S409.

  In S409, the web conference server 200 transmits a text box and a warning display to the client terminal of the speaker. Here, the display screen displayed on the client terminal of the speaker using the text box display screen 700 of FIG. 7 will be described.

  The text box display screen 700 of FIG. 7 is a screen on which the text input notification 710 is superimposed and displayed on the conference screen 610. The text box display screen 700 is displayed including at least the text input notification 710. The text input notification 710 is displayed when the amount of noise included in the voice acquired from the client terminal 100 of its own is larger than the threshold. The text input notification 710 includes a text box 711, a warning display 712, a send button 713, and a cancel button 714.

  The text box 711 is an input receiving unit capable of inputting text by the user. The text input to the text box 711 is transmitted by the web conference server 200 to another client terminal 100 when the transmission button 713 is pressed. As a result, the participant can transmit to other participants a message that was originally intended to speak but was not transmitted due to a large amount of noise. The cancel button 714 can close the text input notification 710. This is used when the user decides that the sound quality will be improved if the communication environment is adjusted, and he wishes to conduct a conference again by inputting a text. The warning display 712 is a display indicating that noise is detected at least a predetermined amount. The warning display 712 can prompt the participant of the client terminal 100 to adjust the environment such as the sound quality of the microphone of his own base.

  Here, the display screen of the client terminal 100 other than the speaker will be described with reference to FIG. The in-input display screen 800 in FIG. 8 is a display in which the in-input display 801 is superimposed on the conference screen 610. The in-input display 801 is displayed in the image display area 613 of the participant in which the text input notification 710 is displayed. The input display screen 800 allows the user to recognize which site is trying to input text.

  In S410, the web conference server 200 determines whether text has been received from the speaker. When the send button 713 of the text box display screen 700 of FIG. 7 is pressed, the web conference server 200 acquires the content of the input text. The web conference server 200 repeats the process of S409 until acquiring the content of the text. On the other hand, the web conference server 200 executes the processing of step S411 when the content of the text is acquired.

  In step S411, the web conference server 200 transmits the text and the enlarged image obtained by enlarging the photographed image of the speaker who has input the text to another client terminal. Here, a display screen displayed on a client terminal other than the speaker at S411 will be described using the text display screen 810 of FIG. The text display screen 810 is a screen where the enlarged image 811 and the text display field 812 are displayed on the conference screen 610 of FIG. Here, the User 002 is a photographed image of the client terminal on which the text input notification 710 is displayed in S409. The web conference server 200 may transmit the audio together with the enlarged image 811 and the text display field 812.

  In the present embodiment, although the virtual meeting room used in the web meeting system is described as a web meeting room, a virtual meeting room, or simply a meeting room, it is the same.

  As described above, according to the above-described embodiment, the web conference system can display a plurality of captured images captured by client terminals connected to a plurality of bases. Then, the noise amount included in the voice of the client terminal is acquired. When the acquired noise amount is larger than the threshold, the web conferencing system can input text for transmitting to another client terminal to the display unit of the client terminal from which the sound including the noise amount is acquired. Display a text box. This makes it possible to easily select the communication method according to the noise amount. That is, the web conferencing system can conduct a web conference with normal voice when the amount of noise is small, and can conference using text when the amount of noise is large. Therefore, it is possible to appropriately hold the meeting in response to the amount of noise.

  As described above, the recording medium recording the program for realizing the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can be achieved also by reading and executing.

  In this case, the program itself read out from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, non-volatile memory card, ROM, EEPROM, silicon A disk etc. can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiment are realized, but also an operating system (OS) or the like running on the computer is actually executed based on the instructions of the program. It goes without saying that the processing is partially or entirely performed, and the processing realizes the functions of the above-described embodiments.

  Furthermore, after the program read from the recording medium is written to the memory provided to the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read based on the instruction of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiment are realized by the processing.

  Further, the present invention may be applied to a system constituted by a plurality of devices or to an apparatus comprising a single device. It goes without saying that the present invention can also be applied to the case where it is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can receive the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server on a network, a database or the like by a communication program, the system or apparatus can receive the effects of the present invention.

100 information processing apparatus 101 camera 200 web conference server 311 speaker identification unit 312 captured image acquisition unit 313 speech acquisition unit 314 noise acquisition unit 315 display control unit

Claims (7)

A web conference system capable of displaying a plurality of captured images captured by client terminals connected to a plurality of bases,
Voice acquisition means for acquiring the voice of the client terminal;
Noise acquisition means for acquiring an amount of noise contained in the audio acquired by the audio acquisition means;
When the amount of noise acquired by the noise acquisition means is larger than a threshold, it is possible to input a text for transmission to another client terminal to the display unit of the client terminal of the acquisition source of the voice including the amount of noise Display control means for displaying a text box;
Web conferencing system characterized by having.   When the noise amount acquired by the noise acquisition means is larger than a threshold, the display control means causes the noise amount to be larger than the threshold with respect to the display unit of the client terminal of the acquisition source of the voice including the noise amount. The web conference system according to claim 1, characterized in that a notification indicating that the user is in contact is displayed. The apparatus further includes an audio output control unit that outputs the audio acquired by the audio acquisition unit to the client terminal.
3. The web conference system according to claim 1, wherein the voice output control means outputs voice to the client terminal when the noise amount obtained by the noise obtaining means is equal to or less than a threshold. .   4. The display control unit according to claim 1, wherein when the text is input in the text box, the display control unit displays a notification indicating that the text is input on the display unit of the other client terminal. The web conferencing system according to any one of the above.   The display control means is characterized in that when the text is input in the text box, the display control unit displays an enlarged image obtained by enlarging an image of the client terminal in which the text input is performed on the display unit of the other client terminal. The web conferencing system according to any one of claims 1 to 4, wherein the web conferencing system is configured as follows. A control method of a web conference system capable of displaying a plurality of photographed images photographed by a client terminal connected to a plurality of bases.
A voice acquisition step of acquiring the voice of the client terminal;
A noise acquisition step of acquiring an amount of noise included in the audio acquired by the audio acquisition step;
When the amount of noise acquired in the noise acquisition step is larger than a threshold, it is possible to input a text for transmission to another client terminal to the display unit of the client terminal of the acquisition source of the voice including the amount of noise A display control process for displaying a text box,
A control method of a web conference system characterized by having.   The program for making a computer perform the control method of the web meeting system of Claim 6.

Images