JP2023121274A - Method for controlling conference system, terminal device, and program - Google Patents

Abstract

To provide a method for promoting equalization of opportunities for speech when executing a conference by using a plurality of terminals.SOLUTION: A method for controlling a conference system 100 includes: displaying a first image corresponding to a first terminal with a first size based on a first speech amount indicating the time length of a first voice input from the first terminal; and displaying a second image corresponding to a second terminal side by side with the first image with a second size based on a second speech amount indicating the time length of a second voice input from the second terminal. When the first speech amount is larger than the second speech amount, the first size is smaller than the second size.SELECTED DRAWING: Figure 1

Description

The present invention relates to a conference system control method, a terminal device, and a program.

Conventionally, a conference system using a computer is known. For example, Patent Literature 1 discloses a configuration in which, in a system including a management server and participant terminals, the right to speak is given to participants who speak less frequently than participants who speak more frequently. According to this configuration, it is said that the opportunities for the participants to speak are equalized.

Japanese Patent Application Laid-Open No. 2003-304337

The configuration of Patent Document 1 is a configuration in which the management server gives the right to speak. Since what kind of method is preferable as a method for equalizing opportunities to speak differs depending on the user and the style of the meeting, diversification of the method has been desired. Therefore, a new technique for promoting equalization of speaking opportunities has been desired.

An aspect of the present disclosure is to display a first image corresponding to the first terminal with a first size based on a first speech volume indicating a duration of a first voice input from the first terminal. displaying a second image corresponding to the second terminal alongside the first image in a second size based on a second utterance volume indicating the time length of the second voice input from the second terminal; and , wherein when the first speech volume is greater than the second speech volume, the first magnitude is smaller than the second magnitude.

Another aspect of the present disclosure includes a control circuit, wherein the control circuit outputs the first voice at a first magnitude based on a first speech volume indicating a duration of a first voice input from a first terminal. outputting a first image corresponding to the terminal to a display device, and arranging the image side by side with the first image in a second size based on a second utterance volume indicating a time length of a second voice input from the second terminal; and outputting a second image corresponding to the second terminal to the display device, and if the first speech volume is greater than the second speech volume, the first magnitude is A terminal device smaller than the second size.

Another aspect of the present disclosure is a program executed by a computer communicable with a first terminal, a second terminal, and a third terminal, wherein a first voice input from the first terminal to the computer displaying, by the third terminal, a first image corresponding to the first terminal at a first size based on a first speech volume indicating a duration of a second voice input from the second terminal; causing the third terminal to display a second image corresponding to the second terminal alongside the first image in a second size based on a second amount of speech indicating the length of time of A program according to claim 1, wherein said first magnitude is smaller than said second magnitude when said first speech volume is greater than said second speech volume.

The figure which shows an example of a structure of the conference system which concerns on embodiment. Explanatory diagram showing an overview of the operation of the conference system. FIG. 2 is a block diagram of devices that make up the conference system; The figure which shows an example of the screen displayed on a terminal. The figure which shows an example of the screen displayed on a terminal. 4 is a flowchart showing an example of server operation. 4 is a flowchart showing an example of server operation.

[1. Overview of conference system]
Hereinafter, this embodiment will be described with reference to the drawings.
FIG. 1 is a diagram showing an example of the configuration of a conference system 100 according to this embodiment.
The conference system 100 is a system in which a plurality of users U use computers to hold a conference by sharing audio and video. A conference system 100 includes a plurality of terminals 1 used by a plurality of users U and a server 50 . The terminal 1 and the server 50 are connected by a communication network 7 so as to be able to communicate with each other.

The number of terminals 1 included in the conference system 100 is not limited. Also, the server 50 may be a single computer, may be composed of a plurality of computers, or may be a cloud server. Server 50 corresponds to an example of a control device. A configuration in which the server 50 is included in the conference system 100 as a device different from the terminal 1 is an example. For example, any terminal 1 participating in a conference in the conference system 100 may have the functions of the server 50 .

In this embodiment, as shown in FIG. 1, an example of holding a conference using four terminals 1 will be described. In the following description, the four terminals 1 are called terminals 1A, 1B, 1C, and 1D. Terminal 1 is used when these terminals are not distinguished from each other. Terminal 1A, terminal 1B, terminal 1C, and terminal 1D are computers having a communication function. Specifically, they are desktop PCs (Personal Computers), tablet PCs, smartphones, and the like. Terminals 1A, 1B, 1C, and 1D correspond to examples of terminal devices.

In the example shown in FIG. 1, the terminal 1A is installed at the site S1, the terminal 1B is installed at the site S2, the terminal 1C is installed at the site S3, and the terminal 1D is installed at the site S4. The geographic relationship of sites S1-S4 is not restricted. The bases S1 to S4 may be separated from each other, may be located in the same building, or may be located in the same room separated from each other. In the following description, the bases S1 to S4 are referred to as bases S when they are not distinguished.

The number of users U using the terminal 1 is not limited. For example, one terminal 1 may be used by multiple users U to participate in a conference. If the terminal 1 is a portable computer, the user U may carry and use the terminal 1 . In this embodiment, the terminal 1A is used by the user U1, the terminal 1B is used by the user U2, the terminal 1C is used by the user U3, and the terminal 1D is used by the user U4. Users U1, U2, U3, and U4 are referred to as user U when not distinguished.

The communication network 7 may be a LAN (Local Area Network) or a WAN (Wide Area Network). Alternatively, it may be a global network including a dedicated line, a public line network, the Internet, and the like.

Terminal 1A has display 14a, keyboard 151a, mouse 152a, camera 16a, microphone 17a, and speaker 18a. Each of these devices is wired or wirelessly connected to the main body of the terminal 1A. At least one of each device may be integrally incorporated into the main body of the terminal 1A. In either case, these devices may be referred to as the display 14a of the terminal 1A, the microphone 17a of the terminal 1A, and the like. The display 14a is a display device that has a display panel and displays images and characters in a display area 141a of the display panel. The display panel of the display 14a is, for example, a liquid crystal display panel, an organic EL (Electro-Luminescence) panel, a plasma display panel, or the like. A keyboard 151a and a mouse 152a are input devices used by the user U1 for input operations. Camera 16a photographs user U1. The microphone 17a collects the voice of the user U1. The speaker 18a outputs conference audio. User U1 uses each of these devices to participate in the conference.

Terminal 1B includes display 14b, keyboard 151b, mouse 152b, camera 16b, microphone 17b, and speaker 18b. Each of these devices is wired or wirelessly connected to the main body of the terminal 1B. The display 14b has a display panel like the display 14a, and is a display device that displays images and characters in a display area 141b of the display panel. Keyboard 151b, camera 16b, microphone 17b, and speaker 18b are configured similarly to keyboard 151a, camera 16a, microphone 17a, and speaker 18a, respectively.

Camera 16b photographs user U2. The microphone 17b collects the voice of the user U2. The speaker 18b outputs conference audio. User U2 uses each of these devices to participate in the conference.

Terminal 1C includes display 14c, keyboard 151c, mouse 152c, camera 16c, microphone 17c, and speaker 18c. Each of these devices is wired or wirelessly connected to the main body of the terminal 1C. The display 14c has a display panel like the display 14a, and is a display device that displays images and characters in a display area 141c of the display panel. The keyboard 151c, camera 16c, microphone 17c, and speaker 18c are configured similarly to the keyboard 151a, camera 16a, microphone 17a, and speaker 18a, respectively.

Camera 16c captures user U3. The microphone 17c collects the voice of the user U3. The speaker 18c outputs conference audio. User U3 uses each of these devices to participate in the conference.

Terminal 1D includes display 14d, keyboard 151d, mouse 152d, camera 16d, microphone 17d, and speaker 18d. Each of these devices is wired or wirelessly connected to the main body of the terminal 1D. The display 14d is a display device that has a display panel like the display 14a and displays images and characters in a display area 141d of the display panel. The keyboard 151d, the camera 16d, the microphone 17d, and the speaker 18d are configured similarly to the keyboard 151a, the camera 16a, the microphone 17a, and the speaker 18a, respectively.

Camera 16d captures user U4. The microphone 17d collects the voice of the user U4. The speaker 18d outputs conference audio. User U4 uses each of these devices to participate in the conference.
The displays 14a, 14b, 14c and 14d will be referred to as the display 14 when not distinguished.

FIG. 2 is an explanatory diagram showing an overview of the operation of the conference system 100. As shown in FIG.
A conference held using the conference system 100 is held by a plurality of users U sharing at least their voices. Also, the conference system 100 may be configured to hold a conference by sharing images and videos captured by the cameras 16a, 16b, 16c, and 16d. In this embodiment, the conference system 100 conducts a conference by sharing images captured by the cameras 16a, 16b, 16c, and 16d.

Data transmitted from the terminals 1A, 1B, 1C, and 1D to the server 50 is called terminal data D1. The terminal data D1 includes terminal data D11 transmitted by the terminal 1A, terminal data D12 transmitted by the terminal 1B, terminal data D13 transmitted by the terminal 1C, and terminal data D14 transmitted by the terminal 1D.

Server 50 transmits server data D2 to terminals 1A, 1B, 1C, and 1D. Server data D2 includes server data D21, D22, D23, and D24.

The terminal 1A transmits terminal data D11 including video data based on video captured by the camera 16a, audio data based on audio collected by the microphone 17a, and operation data based on operation of the keyboard 151a or mouse 152a to the server 50. Send. Similarly, the terminal 1B sends terminal data D11 including video data based on video captured by the camera 16b, audio data based on audio collected by the microphone 17b, and operation data based on the operation of the keyboard 151b or mouse 152b. Send to server 50 . The terminal 1C sends terminal data D11 including video data based on video captured by the camera 16c, audio data based on audio collected by the microphone 17c, and operation data based on operation of the keyboard 151c or mouse 152c to the server 50. Send. The terminal 1D transmits to the server 50 terminal data D11 including video data based on video captured by the camera 16d, audio data based on audio collected by the microphone 17d, and operation data based on operation of the keyboard 151d or mouse 152d. Send.

The server 50 distributes the audio of the conference by the conference system 100 and the video of each user U during the conference to each terminal 1 . The server 50 generates integrated voice data based on the voice data included in the terminal data D1. Integrated audio data includes audio collected by each of the microphones 17a, 17b, 17c, and 17d. The server 50 generates display data based on the video data included in the terminal data D1. The display data includes images captured by each of the cameras 16a, 16b, 16c, and 16d.

The terminal 1A receives the server data D21 and outputs sound from the speaker 18a based on the integrated sound data included in the server data D21. Also, the terminal 1A displays the display data included in the server data D21 on the display 14a. Similarly, the terminal 1B performs audio output from the speaker 18b and display on the display 14b based on the server data D22. The terminal 1C performs audio output from the speaker 18c and display on the display 14c based on the server data D23. The terminal 1D performs audio output from the speaker 18d and display on the display 14d based on the server data D24. As a result, all users U using the conference system 100 can hear the voices of other users U, and can hold voice conferences.

The server 50 may transmit different integrated audio data to the terminals 1A, 1B, 1C, and 1D. For example, the server 50 transmits to the terminal 1A server data D21 that does not contain the sound collected by the microphone 17a. In this case, the speaker 18a does not output the sound collected by the microphone 17a. That is, the voice of user U1 is not output from speaker 18a. As a result, the user U1 does not hear the voice of the user U1 through the speaker 18a, so that the user U1 can avoid discomfort. The server 50 can perform similar control on server data D22, D23, and D24.

The terminal 1A may generate terminal data D11 including video data obtained by processing the image captured by the camera 16a. Alternatively, the terminal 1A may generate terminal data D11 that does not include the image captured by the camera 16a. In this case, the terminal 1A may transmit the terminal data D11 containing dummy video data instead of the image captured by the camera 16a, or may transmit the terminal data D11 containing no video data. When the server 50 receives the terminal data D11 that does not include the video data, the server 50 may generate the server data D2 using an icon image that is associated with the terminal 1A and stored in advance. The same applies to terminal 1B, terminal 1C, and terminal 1D.

[2. Configuration of Devices Constituting Conference System]
FIG. 3 is a block diagram of devices that make up the conference system 100. As shown in FIG.
FIG. 3 illustrates functional configurations of the terminal 1A and the server 50. As shown in FIG. Since the terminals 1B, 1C and 1D are configured in the same manner as the terminal 1A, detailed illustrations of the configurations of the terminals 1B, 1C and 1D are omitted.

As shown in FIG. 3, the terminal 1A has a control circuit 11 to which a display 14a, an input device 15, a camera 16a, a microphone 17a, a speaker 18a, and a communication device 19 are connected.

The input device 15 is a device that the user U uses for input operations. The input device 15 includes, for example, a keyboard 151a and a mouse 152a. The input device 15 may be a touch sensor placed over the display panel of the display 14a, or may be another device.

The communication device 19 is connected to the communication network 7 and performs data communication with the server 50 via the communication network 7 . The communication device 19 includes, for example, a connector for connecting a communication cable and a communication interface circuit. The communication device 19 may include an antenna and a wireless communication circuit, and may be configured to be connected to the communication network 7 through a wireless communication line.

The control circuit 11 has a processor 12 and a memory 13 . The processor 12 is configured by, for example, a CPU (Central Processing Unit), MPU (Micro-Processing Unit), or other integrated circuits. Processor 12 may be a single processor or multiple processors. Processor 12 controls each part of terminal 1A by executing a program.

The memory 13 is a storage device that stores programs and data executed by the processor 12 in a non-volatile manner. The memory 13 is composed of a semiconductor storage device such as a magnetic storage device, a flash ROM (Read Only Memory), or other types of non-volatile storage devices. The memory 13 may include a RAM (Random Access Memory) forming a work area of the processor 12 . The memory 13 stores data processed by the control circuit 11 and control programs executed by the processor 12 .

The processor 12 acquires the photographed data of the camera 16a and generates video data based on the photographed data. The processor 12 converts the voice collected by the microphone 17a into digital voice data. Processor 12 generates audio data based on the digital audio data. The processor 12 receives input from the input device 15 and generates operation data based on the received input.

The processor 12 generates terminal data D<b>11 including video data, audio data, and operation data, and transmits the terminal data D<b>11 to the server 50 via the communication device 19 .

The processor 12 receives the server data D<b>21 transmitted by the server 50 through the communication device 19 . The processor 12 causes the display 14a to display the conference screen 61 by controlling the display 14a based on the display data included in the server data D21. The processor 12 causes the speaker 18a to output sound based on the integrated sound data included in the server data D21.

Terminal 1B, terminal 1C, and terminal 1D are configured in the same manner as terminal 1A. That is, terminal 1B, terminal 1C, and terminal 1D are provided with control circuits including processors. The terminals 1B, 1C, and 1D receive the server data D2 from the server 50 under the control of the processor, and perform audio output and display. Also, the terminals 1B, 1C, and 1D generate terminal data D1 including audio data, video data, and operation data under the control of the processors, and transmit the terminal data D1 to the server 50 .

The server 50 has a server control circuit 51 . A communication device 54 is connected to the server control circuit 51 . The communication device 54 is connected to the communication network 7 and performs data communication with the terminal 1 via the communication network 7 . The communication device 54 includes, for example, a connector for connecting a communication cable and a communication interface circuit. The communication device 54 may include an antenna and a wireless communication circuit, and may be configured to be connected to the communication network 7 through a wireless communication line.

The server control circuit 51 has a processor 52 and a memory 53 . The processor 52 is composed of a CPU, MPU, or other integrated circuits. Processor 52 controls each part of terminal 1C by executing a program. The specific configuration of the processor 52 is, for example, similar to that of the processor 12 , and the specific configuration of the memory 53 is similar to that of the memory 13 . The server control circuit 51 corresponds to an example of a computer.

The processor 52 functions as a communication control unit 521 , a setting unit 522 , a terminal identification unit 523 , an audio processing unit 524 and a display processing unit 525 by executing a control program 531 stored in the memory 53 . The memory 53 stores a control program 531 and conference data 532 . Meeting data 532 includes information about meetings. Information about the conference is input by the user U operating the terminal 1, for example. The information about the conference is, for example, the date and time of the conference, the start time, the end time, the participating users U, and the title of the conference. Conference data 532 may include information generated by server control circuit 51 during execution of the conference.

The communication control unit 521 controls the communication device 54 to perform data communication with the terminal 1 . The communication control unit 521 causes the communication device 54 to receive the terminal data D1 transmitted by the terminal 1 . The communication control unit 521 transmits server data D2 generated by the processor 52 to the terminal 1 .

The setting unit 522 sets the conference using the conference system 100 based on the operation data included in the terminal data D1.

When the terminal data D1 is received from the terminal 1, the terminal identification unit 523 identifies the terminal 1 that has transmitted the terminal data D1. In the conference system 100 of this embodiment, one user U uses one terminal 1 . Therefore, the result of identifying the terminal 1 by the terminal identification unit 523 can be regarded as the result of identifying the user U. FIG. For example, when the terminal identification unit 523 identifies that the terminal 1A is the transmission source of the terminal data D1, the audio processing unit 524 regards the terminal data D1 as data including the audio of the user U1.

The audio processing unit 524 processes audio data included in the terminal data D1. The audio processing unit 524 generates integrated audio data, for example, based on the audio data included in the terminal data D1. The voice processing unit 524 may generate integrated voice data common to all terminals 1 or may generate different integrated voice data for each terminal 1 .

The audio processing unit 524 analyzes the audio data included in the terminal data D1. The voice processing unit 524 has a function of detecting the voice of the user U included in the voice data, and detects that the user U has started speaking and that the user U has finished speaking.

The voice processing unit 524 calculates the amount of user U's speech. The amount of speech may be the length of time of speech, that is, the length of time of speech, the number of speeches in a conference, or the frequency of speech. Speech frequency refers to the number of speeches per unit time in a conference. In this embodiment, as an example, the amount of speech is the length of time that the user U has spoken. The length of time may be expressed in seconds, minutes, hours, or some other unit. More specifically, the amount of speech is the length of time from when user U's speech starts to when it ends. This is called the time length of speech time.

The voice processing unit 524 accumulates the length of speech time for each user U who uses the terminal 1 during the execution of the conference. For example, when detecting the end of user U1's speech during the execution of the conference, the speech processing unit 524 causes the memory 13 to store the length of speech time. After that, when the speech processing unit 524 detects the end of the speech by the user U1, the speech processing unit 524 adds the detected speech duration to the speech duration of the user U1 stored in the memory 13, thereby calculating the speech duration. Calculate the integrated value of The voice processing unit 524 causes the memory 13 to store the calculated integrated value of the speech time. The integrated value of speech time is included in the conference data 532, for example. The voice processing unit 524 executes a process of updating the integrated value of speech time for each user U who participates in the conference. Therefore, the conference data 532 includes an integrated value of speech time for each user U who is a participant in the conference. The voice processing unit 524 continues the process of updating the integrated value of the speech time until the conference ends. The voice processing unit 524 may reset the integrated value of the speech time for each user U stored in the memory 13 after the conference ends.

In this embodiment, the server 50 uses the integrated value of the length of speech time accumulated by the voice processing unit 524 as an index of the amount of speech. The integrated value of the speech duration of any one user U corresponds to an example of the first speech volume. In this case, the integrated value of the speech duration of one user U among the other users U corresponds to an example of the second speech volume.

The display processing unit 525 generates display data for displaying a conference screen. The display processing unit 525 generates server data D<b>2 including display data and integrated audio data, and transmits the generated server data D<b>2 to the terminal 1 .

The display processing unit 525 generates display data for the terminal 1 to display the conference screen 61 including the image captured by the camera 16a, the image captured by the camera 16b, the image captured by the camera 16c, and the image captured by the camera 16d.

FIG. 4 is a diagram showing an example of the screen displayed on the terminal 1, and shows an example of the conference screen 61. As shown in FIG. The conference screen 61 is displayed by the terminal 1 based on the server data D2, and is displayed during the execution of the conference. FIG. 4 shows the conference screen 61 displayed on the display 14a by the terminal 1A, but the terminals 1B, 1C, and 1D also display similar conference screens 61. FIG.

The conference screen 61 has a plurality of display frames 611a, 611b, 611c, 611d. These are collectively described as a display frame 611 . A display frame 611 is a display area corresponding to one terminal 1 . A user image 621 of the user U who uses the terminal 1 is displayed in the display frame 611 . The conference screen 61 specifically includes user images 621a, 621b, 621c, and 621d. When these are not distinguished, they are described as a user image 621 . Any of the user images 621a, 621b, 621c, and 621d corresponds to an example of the first image, and any of the others corresponds to an example of the second image.

The user image 621a is an image corresponding to the user U1, and can be said to be an image corresponding to the terminal 1A. The user image 621a is a video or still image based on video captured by the terminal 1A with the camera 16a. For example, the display processing unit 525 generates the user image 621a based on the video data included in the terminal data D11. The user image 621a may be the same image as the image captured by the camera 16a, or may be an image obtained by processing the image captured by the camera 16a. For example, the user image 621a may be an image obtained by trimming the image captured by the camera 16a. Also, the user image 621a may be an image unrelated to the captured image of the camera 16a. For example, the user image 621a may be an icon or a dummy image that the server 50 stores in the memory 53 in association with the terminal 1A.

The user image 621b is an image corresponding to the user U2, and can be said to be an image corresponding to the terminal 1B. The user image 621b is a video or a still image based on video captured by the terminal 1B with the camera 16b. For example, the display processing unit 525 generates the user image 621b based on the video data included in the terminal data D12. The user image 621b may be the same image as the image captured by the camera 16b, or may be an image obtained by processing the image captured by the camera 16b. Also, the user image 621b may be an image unrelated to the captured image of the camera 16b.

A user image 621c and a user image 621d are images similar to the user images 621a and 621b. That is, the user image 621c is an image corresponding to the user U3, and can be said to be an image corresponding to the terminal 1C. The user image 621d is an image corresponding to the user U4, and can be said to be an image corresponding to the terminal 1D. These user images 621c and 621d are images captured by the cameras 16c and 16d, images obtained by processing the images captured by the cameras 16c and 16d, or images unrelated to the images captured by the cameras 16c and 16d.

For example, the server 50 stores in the memory 53 dummy images and icon images used as the user images 621a, 621b, 621c, and 621d. The display processing unit 525 uses dummy images and icon images stored in the memory 53 as the user images 621a, 621b, 621c, and 621d when instructed by the operation of the terminal 1 or when set in advance. do.

The display frame 611a is an area in which the user image 621a is arranged. A display frame 611b is an area in which a user image 621b is arranged, and a display frame 611c is an area in which a user image 621c is arranged. The display frame 611d is an area in which the user image 621d is arranged.

The display processing unit 525 generates display data in which the sizes of the user images 621a, 621b, 621c, and 621d are uniform at the start of the conference. The size of the user image 621 a is, for example, the area of the user image 621 a or the ratio of the area of the user image 621 a to the conference screen 61 . The size of the user image 621 a may be the size of the user image 621 a in the vertical direction of the conference screen 61 and the size of the user image 621 a in the horizontal direction of the conference screen 61 . The same applies to the sizes of the user images 621b, 621c, and 621d. The sizes and aspect ratios of the display frames 611a, 611b, 611c, and 611d are appropriately determined according to the user images 621a, 621b, 621c, and 621d.

The display processing unit 525 determines the size of at least one of the user images 621 a , 621 b , 621 c , and 621 d based on the integrated value of the speech time of the user U that is integrated by the audio processing unit 524 . In the present embodiment, the relative sizes of the user images 621a, 621b, 621c, and 621d are determined in accordance with the relative relationship between the integrated values of the speech times of the users U. FIG.

For example, the display processing unit 525 generates display data so that the user image 621a is displayed in a size based on the speech volume of the user U1. In addition, the display processing unit 525 generates display data so that the user image 621b is displayed in a size based on the speech volume of the user U2. Then, when the amount of speech by user U1 is greater than the amount of speech by user U2, the size of user image 621a is made smaller than the size of user image 621b. In other words, the display processing unit 525 makes the size of the user image 621a larger than the size of the user image 621b when the amount of speech by the user U1 is smaller than the amount of speech by the user U2.

FIG. 5 is a diagram showing an example of a screen displayed on the terminal 1, and shows an example of changing the size of the user image 621 on the conference screen 61. As shown in FIG.
FIG. 5 shows an example of the conference screen 61 when the amount of speech is smaller in the order of users U1, U2, U4, and U3. User U1 and user U2 have the same amount of speech, and the amount of speech is the smallest among users U1, U2, U3, and U4. In this case, the sizes of the user image 621a corresponding to the user U1 and the user image 621b corresponding to the user U2 are larger than the sizes of the user images 621c and 621d. Also, assuming that user U3 speaks the most, the size of user image 621c corresponding to user U3 is smaller than user images 621a, 621b, and 621d. In addition, since user U1 and user U2 speak the same amount, the size of user image 621a and the size of user image 621b are almost the same.

FIG. 5 shows an example in which the display processing unit 525 maintains the aspect ratio of the user image 621. FIG. That is, when adopting the area of the user image 621 as the size of the user image 621, when adopting the size of the user image 621 in the vertical direction, and when adopting the size of the user image 621 in the horizontal direction , the display form shown in FIG. The same is true when using the ratio of the area of the user image 621 to the conference screen 61 as the size of the user image 621 . Also, the sizes and aspect ratios of the display frames 611a, 611b, 611c, and 611d are appropriately determined according to the user images 621a, 621b, 621c, and 621d. The display processing unit 525 does not have to maintain the aspect ratio of the user image 621. For example, the size of the user image 621 in the vertical direction may be used as a reference to determine the size of the user image 621 based on the amount of speech.

The display processing unit 525 makes the size of the user image 621 corresponding to the user U whose speech volume is large smaller than the size of the user images 621 corresponding to other users U. FIG. Therefore, the size of the user image 621 is made smaller for the user U who speaks more. The display processing unit 525 may have a lower limit for the size of the user image 621 . In this case, the display processing unit 525 sets the size of the user image 621 corresponding to the user U whose speech volume is greater than the threshold as the set lower limit. That is, the size of the user image 621 is equal to or larger than the lower limit. As a result, the size of the user image 621 does not fall below the lower limit even when the amount of speech by the user U is large. The lower limit value and the threshold value are stored in advance in the memory 53 by the server 50, for example. The lower limit value is, for example, a size that allows the user image 621 to be visually recognized when the user image 621 is displayed using the standard-sized display 14 . The lower limit may be defined by the vertical and horizontal lengths when the user image 621 is displayed using the standard size display 14, or by the display resolution or the number of display pixels of the user image 621. may be specified. Alternatively, the lower limit value may be defined by the ratio of the display resolution or the number of pixels of the user image 621 to the display resolution or the number of pixels of the display 14 . Also, the lower limit value may be defined by the ratio of the user image 621 to the area where the conference screen 61 is displayed on the display 14 .

In this way, the sizes of the user images 621a, 621b, 621c, and 621d on the conference screen 61 are determined according to the amount of speech of the users U1, U2, U3, and U4, and the user image 621 of the user U, whose amount of speech is small, is , is displayed relatively larger than at least one other user image 621 . In other words, the user image 621 of the user U whose speaking volume is large is displayed relatively small. As a result, the image of the user U whose speech volume is small in the conference is displayed more conspicuously, so that the corresponding user U can be encouraged to speak. In addition, since the image of the user U who speaks a lot in the conference is displayed in a relatively small size, it is possible to encourage the user U in question to refrain from speaking. Further, on the conference screen 61 displayed by all the terminals 1 participating in the conference in the conference system 100, user images 621a, 621b, 621c, and 621d are displayed in sizes based on the speech volume of the users U1, U2, U3, and U4. be done. Therefore, by displaying the user image 621 of the user U with a small amount of speech in a large size, it is possible to expect an effect of encouraging a plurality of users U participating in the conference to give the user U with a small amount of speech an opportunity to speak. .

[3. Operation of conference system]
6 and 7 are flowcharts showing an operation example of the server 50. FIG. The operation of the conference system 100 will be described with reference to these figures. In the operation of the conference system 100, any one of terminals 1A, 1B, 1C, and 1D corresponds to an example of a first terminal, and any one of the other terminals corresponds to an example of a second terminal. Steps ST11-ST16 are executed by the audio processing section 524, and steps ST17-ST19 are executed by the display processing section 525. FIG.

In step ST11, the server 50 starts detecting speech. The speech detection is a process of detecting that the user U has started speaking by analyzing the terminal data D1 received from the terminal 1 by the server 50 .

In step ST12, the server 50 determines whether or not speech has started. If the speech has not started (step ST12; NO), the server 50 repeats the determination of step ST12 at predetermined time intervals. If it is determined that speech has started (step ST12; YES), the server 50 proceeds to step ST13.

In step ST13, the server 50 identifies the user U who has spoken. Specifically, the server 50 identifies the terminal 1 that transmitted the terminal data D1 whose speech was detected in step ST12, and determines that the user U using this terminal 1 has spoken.

In step ST14, the server 50 determines whether or not the speech has ended. If the speech has not ended (step ST14; NO), the server 50 repeats the determination of step ST14 at predetermined time intervals. When determining that the speech has ended (step ST14; YES), the server 50 proceeds to step ST15.

In step ST15, the server 50 calculates the length of speech time. The server 50 takes the time from detecting the start of speech in step ST12 to detecting the end of speech in step ST14 as the length of speech time. Subsequently, in step ST16, the server 50 updates the integrated value of speech time stored in the memory 53, that is, the speech volume. The server 50 updates the speech volume stored in the memory 53 by adding the speech duration calculated in step ST15 to the speech volume stored in the memory 53 in association with the user U identified in step ST13. .

In step ST<b>17 , the server 50 generates server data D<b>2 by executing display data generation processing, and transmits the server data D<b>2 to the terminal 1 . Details of the display data generation process in step ST17 will be described later with reference to FIG.

At step ST18, the server 50 determines whether or not the conference has ended. If the conference has not ended (step ST18; NO), the server 50 returns to step ST12. If the conference has ended (step ST18; YES), the server 50 ends the processing of FIG.

FIG. 7 shows details of the display data generation process in step ST17.
In step ST21, the server 50 refers to the amount of speech stored in the memory 53 and determines whether or not there is a user U whose amount of speech is greater than the threshold. If there is no user U whose speech volume is greater than the threshold (step ST21; NO), the server 50 proceeds to step ST22. Also, if there is a user U whose speech volume is greater than the threshold (step ST21; YES), the server 50 proceeds to step ST27.

In step ST22, the server 50 calculates the speech volume ratio stored in the memory 53 in association with each user U. FIG. Subsequently, the server 50 calculates the inverse ratio of the amount of speech stored in the memory 53 in association with each user U in step ST23. The inverse ratio is the ratio of the reciprocal of the ratio calculated in step ST22. For example, if the speech volume ratio of users U1, U2, U3, and U4 calculated in step ST22 is 1:1:2:3, the inverse ratio calculated in step ST23 is 1:1:1/ 2:1/3.

At step ST24, the server 50 determines the size of the user image 621 according to the inverse ratio calculated at step ST23. Specifically, the server 50 adjusts the sizes of the user images 621a, 621b, 621c, and 621d so that the ratio of the sizes of the user images 621a, 621b, 621c, and 621d is equal to the inverse ratio calculated in step ST23. determine the For example, if the speech volume ratio of users U1, U2, U3, and U4 is 1:1:2:3, the sizes of user images 621a, 621b, 621c, and 621d are 1:1:1/2: 1/3.

At step ST25, the server 50 generates display data of the conference screen 61 including the user image 621 according to the size of the user image 621 determined at step ST24. The server 50 generates server data D2 including the display data of the conference screen 61 and transmits it to the terminal 1 in step ST26. As a result, the screen displayed on the display 14 by the terminal 1 is updated, and the size of the user image 621 displayed by each terminal 1 is changed to the size corresponding to the amount of speech.

Also, in step ST27, the server 50 sets the size of the user image 621 of the user U whose speech volume is greater than the threshold to the lower limit. Subsequently, in step ST28, the server 50 calculates the speech volume ratio stored in the memory 53 in association with each user U. FIG. The server 50 calculates the inverse ratio of the amount of speech stored in the memory 53 in association with each user U in step ST29. The inverse ratio is the ratio of the reciprocal of the ratio calculated in step ST28. For example, if the speech volume ratio of users U1, U2, U3, and U4 calculated in step ST28 is 1:1:2:5, the inverse ratio calculated in step ST23 is 1:1:1/ 2:1/5.

In step ST30, the server 50 determines the size of the user image 621 according to the lower limit of the size of the user image 621 and the inverse ratio calculated in step ST29. Specifically, the server 50 determines the sizes of the other user images 621, excluding the user image 621 set to the lower limit, so as to follow the inverse ratio calculated in step ST29. For example, if the speech volume ratio of users U1, U2, U3, and U4 is 1:1:2:5, the size of the user image 621d corresponding to user U4 is set to the lower limit. For example, if the lower limit is 1/4, the server 50 sets the sizes of the user images 621a, 621b, 621c, and 621d to 1, where the ratio of sizes is the inverse ratio of the amount of speech of users U1, U2, and U3. : 1:1/2:1/4.

In step ST30, the server 50 performs processing so that the size of the user image 621 determined to be the inverse ratio of the amount of speech does not fall below the lower limit. In the above example, the sizes of the user images 621a, 621b, and 621c are determined to be larger than the lower limit. After the processing of step ST30, the server 50 proceeds to step ST25.

In steps ST24 and ST30, the server 50 may allow multiple user images 621 to overlap on the conference screen 61. FIG. Specifically, the size of the user images 621 may be determined such that a plurality of user images 621 are displayed in an overlapping manner on the conference screen 61 . In this case, since restrictions on the size of the user image 621 with respect to the conference screen 61 are relaxed, the user image 621 can be displayed on the terminal 1 in a size that reflects the amount of speech.

Also, FIG. 7 shows an example in which the server 50 performs processing so that the size ratio of the user images 621a, 621b, 621c, and 621d is the inverse ratio of the amount of speech, but this is just an example. is. For example, the server 50 generates the ranking of the amount of speech by arranging the amounts of speech of the users U1, U2, U3, and U4 in descending order. In this case, the server 50 may determine the sizes of the user images 621a, 621b, 621c, and 621d so that the sizes of the user images 621a, 621b, 621c, and 621d are in reverse order of the order of speech volume. . In this example, the sizes of the user images 621a, 621b, 621c, and 621d can be set to reflect the amount of speech. Moreover, since the server 50 does not need to calculate the sizes of the user images 621a, 621b, 621c, and 621d in detail, the processing load can be reduced. In this example, the server 50 may set the size of the user image 621 in stages. That is, in the server 50, the size of the user image 621 may be set stepwise in advance. In this case, the server 50 can select the size of the user image 621 from preset sizes, so that the processing load on the server 50 can be further reduced.

[4. Action of Embodiment]
As described above, the control method of the conference system 100 is to set the first voice input from the first terminal to the first voice corresponding to the first terminal at the first magnitude based on the first speech volume indicating the time length of the first voice input from the first terminal. Including displaying images. In this control method, the second image corresponding to the second terminal is displayed side by side with the first image at a second size based on the second speech volume indicating the time length of the second voice input from the second terminal. including doing In this control method, the first magnitude is smaller than the second magnitude when the first speech volume is greater than the second speech volume. Here, the first terminal can be applied to any of the terminals 1A, 1B, 1C, and 1D in the conference system 100. FIG. The second terminal may be any one of the terminals 1A, 1B, 1C, 1D that is not the first terminal.

According to this method, when the utterance volume of the voice input from the first terminal is different from the utterance volume of the voice input from the second terminal, the second image corresponding to the second terminal with the smaller utterance volume is It is displayed larger than the first image. Therefore, the image corresponding to the user U whose speech volume is small is displayed larger than the image corresponding to the user U whose speech volume is large. As a result, the image of the user U whose speaking volume is small in the conference is displayed more conspicuously, so that the corresponding user U can be encouraged to speak. Therefore, it is possible to suppress bias in speaking opportunities in the conference and promote equalization of speaking opportunities.

In the above control method, when the first speech volume is smaller than the second speech volume, the first magnitude is greater than the second magnitude. Therefore, the image corresponding to the user U who speaks a lot is displayed in a relatively small size. As a result, the image of the user U who speaks a lot in the conference is displayed in a relatively small size, so that the corresponding user U can be urged to refrain from speaking. Therefore, it is possible to suppress bias in speaking opportunities in the conference and promote equalization of speaking opportunities.

In the above control method, the first magnitude is equal to or greater than the lower limit of the first magnitude. Thereby, the visibility of the image displayed relatively small can be ensured.

In the above control method, when the first speech volume is greater than the threshold, the first magnitude is the lower limit. Therefore, the image corresponding to the user U whose speech volume is larger than the threshold can be made relatively small by processing with a light load.

In the above control method, when the first speech volume is equal to the second speech volume, the first magnitude and the second magnitude are equal. As a result, when the amount of speech in a conference is nearly equal, the effect of not causing bias in speaking opportunities can be expected.

The control method includes calculating the first speech volume by accumulating the time length of the first voice after the conference using the first terminal and the second terminal is started. The above control method includes calculating the second speech volume by accumulating the time length of the second voice after the conference is started. As a result, the amount of speech can be obtained by accumulating the time length of speech from the start of the conference. According to this method, the size of the image can be determined according to the amount of speech after the conference is started, and the size of the image can be changed so as to reflect the change in the amount of speech. Therefore, it is possible to expect an effect of suppressing bias in speaking opportunities during a conference and promoting equalization of speaking opportunities.

A terminal 1 that constitutes the conference system 100 includes a control circuit 11 . The control circuit 11 displays the first image corresponding to the first terminal on the display device with a first size based on the first speech amount indicating the time length of the first voice input from the first terminal. The control circuit 11 displays the second image corresponding to the second terminal side by side with the first image in a second size based on the second speech volume indicating the time length of the second voice input from the second terminal. display on the device. The first magnitude is smaller than the second magnitude when the first speech volume is greater than the second speech volume. This configuration can be applied to any of terminal 1A having display 14a, terminal 1B having display 14b, terminal 1C having display 14c, and terminal 1D having display 14d, for example. Also, the first terminal can be applied to any of the terminals 1A, 1B, 1C, and 1D in the conference system 100. FIG. The second terminal may be any one of the terminals 1A, 1B, 1C, 1D that is not the first terminal.

According to this configuration, when the utterance volume of the voice input from the first terminal and the utterance volume of the voice input from the second terminal are different, the display 14 displays the number corresponding to the second terminal with the smaller utterance volume. Two images are displayed larger than the first image. Therefore, the image corresponding to the user U whose speech volume is small is displayed larger than the image corresponding to the user U whose speech volume is large. As a result, the image of the user U whose speaking volume is small in the conference is displayed more conspicuously, so that the corresponding user U can be encouraged to speak. Therefore, it is possible to suppress bias in speaking opportunities in the conference and promote equalization of speaking opportunities.

Conference system 100 includes a first terminal, a second terminal, and a third terminal. The conference system 100 causes the third terminal to display the first image corresponding to the first terminal in a first size based on the first speech volume indicating the time length of the first voice input from the first terminal. The conference system 100 arranges the second image corresponding to the second terminal side by side with the first image at a second size based on the second speech volume indicating the time length of the second voice input from the second terminal. 3 Display by terminal. In conference system 100, when the first speech volume is greater than the second speech volume, the first magnitude is smaller than the second magnitude.

The conference system 100 causes the third terminal to display an image corresponding to the first terminal and an image corresponding to the second terminal. In this configuration, when the utterance volume of the voice input from the first terminal is different from the utterance volume of the voice input from the second terminal, the image corresponding to the second terminal with the smaller utterance volume is displayed on the first terminal. It appears relatively larger than the corresponding image. As a result, the image of the user U whose speech volume is small in the conference is displayed more conspicuously, so that the corresponding user U can be encouraged to speak. Therefore, it is possible to suppress bias in speaking opportunities in the conference and promote equalization of speaking opportunities.

The control program 531 is a program executed by the server 50, which is a computer capable of communicating with the first terminal, second terminal, and third terminal. The control program 531 causes the server 50 to transmit the first image corresponding to the first terminal to the third terminal with the first size based on the first speech volume indicating the time length of the first voice input from the first terminal. causes the display to be performed. Also, a second image corresponding to the second terminal is displayed by the third terminal in a second size based on the second speech volume indicating the time length of the second voice input from the second terminal, side by side with the first image. Make it do what it displays. Here, when the first speech volume is larger than the second speech volume, the first magnitude is smaller than the second magnitude.

According to this program, the server 50 causes the third terminal to display an image corresponding to the first terminal and an image corresponding to the second terminal. When the utterance volume of the voice input from the first terminal and the utterance volume of the voice input from the second terminal are different from the third terminal, the server 50 displays an image corresponding to the second terminal with the smaller utterance volume. , is displayed relatively larger than the image corresponding to the first terminal. As a result, the image of the user U whose speech volume is small in the conference is displayed more conspicuously, so that the corresponding user U can be encouraged to speak. Therefore, it is possible to suppress bias in speaking opportunities in the conference and promote equalization of speaking opportunities.

[5. Other embodiments]
Each of the above embodiments shows a specific example to which the present invention is applied, and the present invention is not limited to this.

In the above embodiment, the terminal 1 includes the display 14 as a display device. However, the display device is not limited to the display 14 . For example, the terminal 1 may include, as a display device, a projector that projects an image onto a projection surface such as a screen.

In the conference system 100, the data formats of the terminal data D1 that the terminal 1 transmits to the server 50 and the server data D2 that the server 50 transmits to the terminal 1 are not limited. For example, the display data included in the server data D2 may be the data of the conference screen 61 displayed by the terminal 1, or the terminal 1 may generate the data of the conference screen 61 based on the display data. good.

The server data D21, D22, D23, and D24 generated by the server 50 may contain different display data or may contain common display data. When the server data D21, D22, D23, and D24 include common display data, all users U participating in the conference system 100 can be shown the user image 621 in a size based on the speaking volume. Further, the server 50 may display the user image 621 in a size based on the amount of speech on the terminal 1 used by the specific user U by changing the display data of the server data D21, D22, D23, and D24. good. For example, the user image 621 may be displayed in a size based on the amount of speech, targeting the terminal 1 used by the user U with the smallest amount of speech or the terminal 1 used by the user U with the largest amount of speech. In this case, it is possible to prompt the user U, who has a biased amount of speech, to promote speech or suppress speech.

Moreover, each functional unit shown in FIG. 3 shows a functional configuration, and does not limit a specific implementation form. For example, the server 50 does not need to implement hardware individually corresponding to each functional unit of the server control circuit 51, and the configuration is such that one processor executes a program to realize the functions of a plurality of functional units. Of course, it is also possible. Further, a part of the functions realized by software in the above embodiments may be realized by hardware, or a part of the functions realized by hardware may be realized by software. In addition, the specific detailed configuration of other units of the conference system 100 can be arbitrarily changed without departing from the scope.

Further, for example, the operation step units shown in FIGS. 6 and 7 are divided according to the main processing contents in order to facilitate understanding of the operation of the conference system 100. The disclosure is not limited by any name or name. It may be divided into many steps depending on the processing contents. Also, one step unit may be divided so as to include many processes. Also, the order of the steps may be changed as appropriate within the scope of the present disclosure.

1, 1A, 1B, 1C, 1D... terminal (terminal device), 7... communication network, 11... control circuit, 12... processor, 13... memory, 14, 14a, 14b, 14c, 14d... display, 16a, 16b, 16c, 16d... camera, 17a, 17b, 17c, 17d... microphone, 18a, 18b, 18c, 18d... speaker, 50... server, 51... server control circuit, 52... processor, 53... memory, 54... communication device, 100 Conference system 521 Communication control unit 522 Setting unit 523 Terminal identification unit 524 Audio processing unit 531 Control program (program) 532 Conference data 621, 621a, 621b, 621c, 621d User images, U, U1, U2, U3, U4 . . . users.

Claims (8)

displaying a first image corresponding to the first terminal at a first size based on a first speech volume indicating a duration of a first voice input from the first terminal;
Displaying a second image corresponding to the second terminal side by side with the first image at a second size based on a second amount of speech indicating a time length of a second voice input from the second terminal. , including
The method of controlling a conference system, wherein the first magnitude is smaller than the second magnitude when the first speech volume is greater than the second speech volume. 2. The method of controlling a conference system according to claim 1, wherein said first magnitude is greater than said second magnitude when said first speech volume is smaller than said second speech volume. 3. The conference system control method according to claim 1, wherein said first size is equal to or greater than a lower limit value of said first size. 4. The conference system control method according to claim 3, wherein said first magnitude is said lower limit value when said first speech volume is greater than a threshold. 4. The control of the conference system according to any one of claims 1 to 3, wherein when said first speech volume and said second speech volume are equal, said first magnitude and said second magnitude are equal. Method. calculating the first speech volume by accumulating the time length of the first voice after the conference using the first terminal and the second terminal is started;
5. The conference system according to any one of claims 1 to 4, comprising: calculating the second speech volume by accumulating the time length of the second voice after the conference is started. control method. including a control circuit,
The control circuit is
outputting a first image corresponding to the first terminal to a display device at a first magnitude based on a first speech volume indicating a duration of a first voice input from the first terminal;
displaying a second image corresponding to the second terminal on the display device in a second size based on a second amount of speech indicating a time length of the second voice input from the second terminal, side by side with the first image; and run the
The terminal device, wherein the first magnitude is smaller than the second magnitude when the first speech volume is greater than the second speech volume. A program executed by a computer capable of communicating with a first terminal, a second terminal, and a third terminal,
on said computer;
displaying, by the third terminal, a first image corresponding to the first terminal at a first size based on a first speech volume indicating a duration of a first voice input from the first terminal;
The second image corresponding to the second terminal is displayed in the third size side by side with the first image at a second size based on the second speech volume indicating the time length of the second voice input from the second terminal. display by the terminal and run
A program according to claim 1, wherein said first magnitude is smaller than said second magnitude when said first speech volume is greater than said second speech volume.

Images