US20170048283A1

US20170048283A1 - Non-transitory computer readable medium, information processing apparatus, and information processing system

Info

Publication number: US20170048283A1
Application number: US15/048,137
Authority: US
Inventors: Kohshiro Inomata; Takumi Kobayashi
Original assignee: Fuji Xerox Co Ltd
Current assignee: Fujifilm Business Innovation Corp
Priority date: 2015-08-12
Filing date: 2016-02-19
Publication date: 2017-02-16
Also published as: EP3131257B1; EP3131256B1; US20170048284A1; EP3131257A1; EP3131256A1; US20170046659A1; US10341397B2

Abstract

A non-transitory computer readable medium stores a program causing a computer to transmit and receive multimedia data, including at least one of sound data and video data, between a plurality of terminal devices connected to each other through a network. The program causes the computer to function as a transmission request receiving section, a reception request receiving section, a first communication channel establishment section, an establishment request receiving section, a permission/rejection receiving section, and a second communication channel establishment section. The transmission request receiving section receives a transmission request for first multimedia data from a terminal device. The reception request receiving section receives a reception request for the first multimedia data from another terminal device. The first communication channel establishment section establishes a first communication channel for transmitting and receiving the first multimedia data between the terminal device making the transmission request and the terminal device making the reception request.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2015-175948 filed Sep. 7, 2015, Japanese Patent Application No. 2015-160486 filed Aug. 17, 2015, and Japanese Patent Application No. 2015-159611 filed Aug. 12, 2015.

BACKGROUND

(i) Technical Field
The present invention relates to a non-transitory computer readable medium, an information processing apparatus, and an information processing system.
(ii) Related Art
In recent years, there has been proposed a system in which sound data and video data are transmitted and received between plural terminal devices connected to each other through a network. For example, the system is used in an electronic conference in which plural users in different locations simultaneously participate.
In the system of the related art, since sound data is generally transmitted and received bidirectionally between all terminal devices connected to each other through a network, it is not possible to exchange a sound only between specific terminal devices, that is, between specific users.

SUMMARY

According to an aspect of the invention, anon-transitory computer readable medium stores a program causing a computer to transmit and receive multimedia data, including at least one of sound data and video data, between plural terminal devices connected to each other through a network. The program causes the computer to function as a transmission request receiving section, a reception request receiving section, a first communication channel establishment section, an establishment request receiving section, a permission/rejection receiving section, and a second communication channel establishment section. The transmission request receiving section receives a transmission request for first multimedia data from at least one terminal device. The reception request receiving section receives a reception request for the first multimedia data from at least one another terminal device. The first communication channel establishment section establishes a first communication channel for transmitting and receiving the first multimedia data between the at least one terminal device making the transmission request and the at least one terminal device making the reception request. The establishment request receiving section receives, from a first terminal device, an establishment request for a second communication channel for limitedly transmitting and receiving second multimedia data between the first terminal device and a second terminal device. The plural terminal devices transmitting and receiving the first multimedia data through the first communication channel include the first terminal device and the second terminal device. The permission/rejection receiving section receives a reply, indicating one of permission and rejection, to the establishment request from the second terminal device. The second communication channel establishment section establishes the second communication channel between the first terminal device and the second terminal device when receiving a reply indicating the permission of the establishment request from the second terminal device.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described in detail based on the following figures, wherein:

FIG. 1 is a configuration diagram illustrating the entire electronic conference system according to an exemplary embodiment;

FIG. 2 is a block diagram illustrating hardware configurations of a terminal device and a server;

FIG. 3 is a functional block diagram illustrating a specific configuration of the electronic conference system;

FIG. 4 is a functional block diagram illustrating a specific configuration of the electronic conference system;

FIG. 5 is a diagram illustrating an example of a network configuration of the electronic conference system;

FIG. 6 is a diagram illustrating an example of a transmission terminal list;

FIG. 7 is a diagram illustrating an example of a reception terminal list;

FIG. 8 is a diagram illustrating an example of an updated reception terminal list;

FIG. 9 is a diagram illustrating an example of a network configuration showing first communication channels;

FIG. 10 is a diagram illustrating an example of a network configuration showing first communication channels;

FIG. 11 is a diagram illustrating an example of a network configuration showing communication channels;

FIG. 12 is a flow chart illustrating the operation of the electronic conference system;

FIG. 13 is a flow chart illustrating the operation of the electronic conference system;

FIG. 14 is a flow chart illustrating the operation of the electronic conference system;

FIG. 15 is a functional block diagram illustrating a specific configuration for executing a second communication channel establishment process;

FIG. 16 is a flow chart illustrating the operation of the second communication channel establishment process;

FIG. 17 is a diagram illustrating an example of a network configuration showing a limited communication channel;

FIG. 18 is a flow chart illustrating the operation of another second communication channel establishment process;

FIG. 19 is a diagram illustrating an example of a network configuration including another limited communication channel;

FIG. 20 is a flow chart illustrating the operation of another second communication channel establishment process;

FIG. 21 is a diagram illustrating an example of a network configuration including another limited communication channel;

FIG. 22 is a functional block diagram illustrating a specific configuration of a server according to Modification Example 1;

FIG. 23 is a diagram illustrating an example of a network configuration of the electronic conference system;

FIG. 24 is a diagram illustrating an example of a reception terminal list;

FIG. 25 is a diagram illustrating an example of a network configuration showing communication channels according to Modification Example 1; and

FIG. 26 is a diagram illustrating an example of a network configuration of the electronic conference system.

DETAILED DESCRIPTION

Hereinafter, an exemplary embodiment of the present invention will be described with reference to the accompanying drawings. In the exemplary embodiment, an electronic conference system that transmits and receives sound data between plural terminal devices connected to each other through a network will be described as an example of an information processing system.
FIG. 1 is a configuration diagram illustrating the entire electronic conference system according to the exemplary embodiment. An electronic conference system. 100 (information processing system) is configured to include plural terminal devices 10 that are used by plural users participating in a conference, and a server 20 that performs information communication with the terminal devices 10. FIG. 1 illustrates six terminal devices 10 a to 10 f and one server 20 connected to the terminal devices 10 a to 10 f through a communication network 30. Meanwhile, the number of terminal devices 10 constituting the electronic conference system 100 is not limited. Hereinafter, when components common to the terminal devices are described, the terminals will be collectively referred to as a terminal device 10.
As illustrated in FIG. 1, the terminal devices 10 and the server 20 are connected to each other through the communication network 30 such as a local area network (LAN) or the Internet. Meanwhile, in FIG. 1 and other drawings to be described later, a dotted line indicates a state of connection through a network. Control information and the like between the terminal devices are transmitted and received between the terminal devices through the communication network 30.
FIG. 2 is a block diagram illustrating hardware configurations of the terminal device 10 and the server 20. The terminal device 10 is configured with a computer including a CPU 101, a memory 102, a storage section 103, a communication section 104, an operation section 105, a display section 106, a sound input section 107, and a sound output section 108. The hardware elements are connected to each other by a bus so as to give and receive data to and from each other. The CPU 101 controls each section of the terminal device 10, and executes various types of information processing. The memory 102 stores various types of programs and data. A work area of the CPU 101 is also secured in the memory 102. The storage section 103 stores data such as various types of files or documents. The communication section 104 performs data communication with the server 20 and other terminal devices 10 through the communication network 30. The operation section 105 is an input device such as a keyboard, a mouse, a pen tool, or a touch panel. The display section 106 is a display screen that displays various types of files and documents through application software such as a Web browser. In addition, the display section 106 displays an operation screen for receiving a user's operation in an electronic conference. The sound input section 107 is a microphone to which a sound of a user speaking in the electronic conference is input. The sound output section 108 is a speaker that outputs sound data received from another terminal device 10 as a sound in the electronic conference. The storage section 103, the operation section 105, the display section 106, the sound input section 107, and the sound output section 108 may be provided outside the terminal device 10, and may be connected to the terminal device through the communication network 30. In addition, software such as a browser or an e-mail client is incorporated into the terminal device 10. The terminal device 10 is a portable information terminal such as, a personal computer, a personal digital assistant (PDA), or a smartphone.
The server 20 is configured with a computer including a CPU 201, a memory 202, a storage section 203, and a communication section 204. The hardware elements are connected to each other by a bus so as to give and receive data to and from each other. The CPU 201 controls each section of the server 20 and executes various pieces of information processing. The memory 202 stores various types of programs and data. A work area of the CPU 201 is also secured in the memory 202. The storage section 203 stores data such as various types of files or documents. The communication section 204 performs data communication with the terminal devices 10 through the communication network 30.
Here, each of the terminal devices 10 may have both a transmission function of transmitting sound data which is uttered from a user and a reception function of receiving the sound data, or may have either of the functions. The functions of each of the terminal devices 10 may be determined based on an instruction of a user using the terminal device 10, a mode of an electronic conference, or the like.
First Communication channel Establishment Process
Next, a function for realizing a process for establishing a communication channel (first communication channel) in the electronic conference system 100 will be described.
FIGS. 3 and 4 are functional block diagrams illustrating a specific configuration of the electronic conference system 100. For convenience of description, functional blocks corresponding to a transmission function are shown in the terminal device 10 a illustrated in FIG. 3, and functional blocks corresponding to a reception function are shown in the terminal device 10 b illustrated in FIG. 4. Meanwhile, as described above, the terminal device 10 a may further have a reception function (see FIG. 4), and the terminal device 10 b may further have a transmission function (see FIG. 3). Hereinafter, a case where each terminal device has both the functions will also be described as necessary.
As illustrated in FIGS. 3 and 4, the server 20 includes a transmission request receiving section 21, a reception request receiving section 22, a determination processing section 23, and a terminal information generation section 24. As illustrated in FIG. 3, the terminal device 10 a includes a transmission request transmitting section 11, a determination result acquisition section 12, a terminal information acquisition section 13, a communication channel establishment section 14 (first communication channel establishment section), a sound transmitting section 15, and a display control section 16. As illustrated in FIG. 4, the terminal device 10 b includes a reception request transmitting section 41, a determination result acquisition section 42, a display control section 43, and a sound receiving section 44.
The elements included in each of the terminal devices 10 and the server 20 are realized by the CPU 101 and the CPU 201 executing programs stored in the respective memories 102 and 202 (see FIG. 2).
The programs may be installed in the terminal device 10 and the server 20 from a computer-readable information storage medium such as a CD-ROM, s DVD-ROM, or a memory card, or may be downloaded through the communication network 30 such as the Internet. Alternatively, programs corresponding to predetermined elements may be downloaded to the terminal devices 10 from the server 20 in which the programs corresponding to the elements are stored in advance.
For example, as illustrated in FIG. 5, in a network configuration in which the terminal devices 10 a to 10 f are connected to the server 20 through the communication network 30, programs corresponding to predetermined elements may be downloaded to the memories 102 of the respective terminal devices 10 a to 10 f from the server 20 in which programs corresponding to the elements are stored in advance, at a point in time when the terminal devices 10 a to 10 f are connected to the server 20. The downloaded programs maybe constituted by, for example, JavaScript (registered trademark). The terminal devices 10 a to 10 f acquire the programs when connecting to the server 20.
Functions corresponding to the programs will be described below. Hereinafter, a flow of processing of the electronic conference system 100 according to the progress of an electronic conference will be described.
First, in an electronic conference, when a user desires to transmit his or her speaking (sound) to another terminal device 10, that is, when the user requests the establishment of a first communication channel for transmitting sound data (first sound data) between his or her terminal device and another terminal device, the user operates the operation section 105 of the terminal device 10 to give instruction in that effect. For example, in an electronic conference, a user A selects a selection screen (for example, a check box or an electronic button) of “sound transmission” displayed on the display section 106 of the terminal device 10 a (see FIG. 3) used by the user himself or herself, using a keyboard, a mouse, or the like. The transmission request transmitting section 11 of the terminal device 10 a transmits the received request of the user A (hereinafter, referred to as a transmission request) to the server 20. Meanwhile, for example, when the user A cancels the check box of “sound transmission” or when a predetermined condition is satisfied (described later), the transmission request transmitting section 11 transmits a cancellation request for transmission. For example, the transmission request transmitting section 11 transmits “+1” when transmitting a transmission request, and transmits “−1” when transmitting a cancellation request.
In addition, in an electronic conference, when a user desires to receive the speaking (sound) of another user, the user operates the operation section 105 of the terminal device 10 to give instruction in that effect. For example, in an electronic conference, a user B selects a selection screen (for example, a check box or an electronic button) of “sound reception” displayed on the display section 106 of the terminal device 10 b (see FIG. 4) used by the user himself or herself, using a keyboard, a mouse, or the like. The reception request transmitting section 41 of the terminal device 10 b transmits the received request of the user B (hereinafter, referred to as a reception request) to the server 20. Meanwhile, for example, when the user B cancels the check box of “sound reception” or when a predetermined condition is satisfied (described later), the reception request transmitting section transmits a cancellation request for reception. For example, the reception request transmitting section 41 transmits “+1” when transmitting a reception request, and transmits “−1” when transmitting a cancellation request.
Meanwhile, in the terminal devices 10, both selection screens of “sound transmission” and “sound reception” are displayed, and each user may individually select the selection screens. For example, in the terminal device 10 a, the user A may select both “sound transmission” and “sound reception”.
The transmission request receiving section 21 of the server 20 receives a transmission request which is transmitted from the transmission request transmitting section 11 of the terminal device 10. For example, when a transmission request is transmitted from each of the terminal devices 10 a and 10 f, the transmission request receiving section 21 receives both transmission requests of the terminal devices 10 a and 10 f.
The reception request receiving section 22 of the server 20 receives a reception request which is transmitted from the reception request transmitting section 41 of the terminal device 10. For example, when a reception request is transmitted from each of the terminal devices 10 a to 10 f, the reception request receiving section 22 receives a reception request of each of the terminal devices 10 a to 10 f.
The determination processing section 23 of the server 20 determines whether to permit sound transmission in the terminal device 10 that transmits a transmission request received by the transmission request receiving section 21, based on the transmission request. Specifically, the determination processing section 23 compares the total number of terminal devices making a transmission request received by the transmission request receiving section 21 with the predetermined number of terminal devices, and determines whether the total number is equal to or less than the set number. The determination processing section 23 permits sound transmission in the terminal device making a transmission request when the total number is equal to or less than the set number, and does not permit sound transmission when the total number exceeds the set number. For example, when the terminal devices 10 a and 10 f make a transmission request in a case where the set number is set to “3”, the total number (two) is equal to or less than the set number (three), and thus sound transmission in the terminal devices 10 a and 10 f is permitted.
Here, the determination processing section 23 calculates the total number of terminal devices 10 making a transmission request at the current point in time (at a point in time when a terminal device to be determined makes a transmission request). For example, when the terminal devices 10 a and 10 f has already made a transmission request and the terminal device 10 b has newly made a transmission request, the total number at the current point in time is three. Accordingly, for example, when the terminal devices 10 a and 10 f have already made a transmission request (sound transmission may be performed already) and the terminal device 10 b has newly made a transmission request in a case where the set number is set to the total number (three) exceeds the set number (two), and thus the sound transmission of the terminal device 10 b having newly made a transmission request is not permitted.
In this manner, sound transmission is permitted until the reaching of the set number in order of transmission requests made. In addition, when the number of terminal devices 10 permitted to transmit a sound has been permitted reaches the set number, the sound transmission of another terminal device 10 may be permitted in a case where sound transmission (first communication channel) is cancelled in any one of the terminal devices 10.
The determination processing section 23 transmits a result of the determination (permission notification, non-permission notification) to the terminal device 10 having transmitted a transmission request.
In addition, the determination processing section 23 may determine whether to permit sound reception in the terminal device 10 having transmitted a reception request received by the reception request receiving section 22, based on the reception request. For example, the determination processing section 23 may compare the total number of terminal devices 10 making a reception request received by the reception request receiving section 22 with the predetermined number of terminal devices, and may determine whether the total number is equal to or less than the set number. In addition, it may be determined whether to permit the sound reception based on other conditions to be described later (Modification Example 1 to be described later). In addition, the determination processing section 23 may permit sound reception with respect to all reception requests received by the reception request receiving section 22. The determination processing section transmits a result of the determination (permission notification, non-permission notification) to the terminal device 10 having transmitted a reception request.
Further, when the total number of terminal devices 10 capable of making a transmission request reaches the set number, the determination processing section 23 may transmit a notification (message) indicating that the number of transmission terminals reaches a specified value (set number) and a transmission request cannot be made any more, to all of the terminal devices 10 connected to the server 20. Similarly, when a transmission cancellation request is received and the number of terminal devices capable of making a transmission request falls below the set number, a notification indicating that a transmission request may be made may be transmitted to all of the terminal devices 10. In addition, a notification indicating the number of remaining terminal devices itself until the reaching of the set number may be transmitted. For example, the situation thereof is as follows. When the set number is two, the terminal device 10 a is notified of transmission permission when a transmission request is made from the terminal device 10 a, and “1” which is the number of remaining terminal devices until the reaching of the set number is transmitted to all of the terminal devices 10. Next, when a transmission request is made from the terminal device 10 f, the terminal device 10 f is notified of transmission permission, and “0” is transmitted to all of the terminal devices 10. Further, when a transmission cancellation request is made from the terminal device 10 a, “1” is transmitted to all of the terminal devices 10. Meanwhile, the determination processing section 23 may perform the same process as that mentioned above with respect to a reception request.
The terminal information generation section 24 of the server 20 generates transmission terminal information based on a transmission request received by the transmission request receiving section 21 and a result of the determination made by the determination processing section 23. Specifically, when the total number is equal to or less than the set number, the terminal information generation section 24 generates a transmission terminal list in which pieces of terminal information of the terminal device 10 permitted to transmit a sound are listed. For example, when transmission requests received from the terminal devices 10 a and 10 f are permitted, the terminal information generation section 24 generates a transmission terminal list in which pieces of terminal information such as names and pieces of identification information (terminal ID, IP address, and the like) of the terminal devices 10 a and 10 f are listed. FIG. 6 illustrates an example of a transmission terminal list. The terminal information generation section 24 updates the transmission terminal list whenever a received transmission request is permitted or a transmission request is cancelled. The generated transmission terminal list is stored in a terminal list DB 203 a.
In addition, the terminal information generation section generates reception terminal information based on a reception request received by the reception request receiving section 22. Specifically, the terminal information generation section 24 generates a reception terminal list in which pieces of terminal information of the terminal devices 10 permitting a reception request are listed. For example, when reception requests received from the terminal devices 10 a to 10 f are permitted, the terminal information generation section 24 generates a reception terminal list in which pieces of terminal information such as names and pieces of identification information (terminal ID, IP address, and the like) of the terminal devices 10 a to 10 f are listed. FIG. 7 illustrates an example of a reception terminal list. The terminal information generation section 24 updates the reception terminal list whenever a received reception request is permitted or a reception request is cancelled. FIG. 8 illustrates an updated reception terminal list in a case where a new reception request received from a network-connected terminal device 10 g (not shown) is permitted in the state in FIG. 7. The generated reception terminal list is stored in the terminal list DB 203 a.
In addition, the terminal information generation section 24 transmits the reception terminal list to the terminal device permitted to transmit a sound in the determination processing section 23. Further, even when the reception terminal list is updated (changed), the terminal information generation section 24 transmits the updated reception terminal list to the terminal device 10 permitted to transmit a sound. For example, when the terminal devices 10 a and 10 f are permitted to transmit a sound, the terminal information generation section 24 transmits the reception terminal list (see FIG. 7) to the terminal devices 10 a and 10 f. In addition, even when the reception terminal list is updated, the terminal information generation section 24 transmits the updated reception terminal list to the terminal devices 10 a and 10 f. At this time, only the changed (added) reception terminal list may be transmitted.
The determination result acquisition section 12 of the terminal device 10 a receives (or acquires) a result of the determination made in the determination processing section 23 of the server 20. For example, the determination result acquisition sections 12 of the respective terminal devices 10 a and 10 f acquire a permission notification for sound transmission from the server 20.
The terminal information acquisition section 13 (terminal information reception section) of the terminal device 10 a receives (or acquires) a reception terminal list from the terminal information generation section 24 of the server 20. For example, the terminal information acquisition sections 13 of the respective terminal devices 10 a and 10 f acquire the reception terminal list illustrated in FIG. 7 from the server 20.
The communication channel establishment section 14 of the terminal device 10 a establishes a first communication channel for transmitting (transmitting and receiving) sound data between the communication channel establishment section and another terminal device 10 based on the reception terminal list acquired by the terminal information acquisition section 13. The acquired reception terminal list is stored in the storage section 103. For example, the communication channel establishment section 14 of the terminal device 10 a establishes a first communication channel between the terminal device 10 a and each of the terminal devices 10 b, 10 c, 10 d, 10 e, and 10 f. FIG. 9 illustrates a network configuration showing a first communication channel established between the terminal device 10 a and each of the terminal devices 10 b, 10 c, 10 d, 10 e, and 10 f. Similarly, the communication channel establishment section 14 of the terminal device 10 f establishes a first communication channel between the terminal device 10 f and each of the terminal devices 10 a, 10 b, 10 c, 10 d, and 10 e. FIG. 10 illustrates a network configuration showing a first communication channel established between the terminal device 10 f and each of the terminal devices 10 a, 10 b, 10 c, 10 d, and 10 e. FIG. 11 illustrates a network configuration showing all of the first communication channels established in the above-mentioned examples. In FIGS. 9 to 11, a direction of an arrow indicates a direction in which sound data is transmitted. Although FIGS. 9 to 11 illustrate a configuration in which sound data is transmitted and received without going through the server 20, the network configuration is not limited thereto, and a configuration in which sound data is transmitted and received using the server 20 as a repeater may be adopted.
In addition, for example, the establishment of a first communication channel is performed on a terminal device (reception terminal device) receiving a sound by a terminal device (transmission terminal device) transmitting a sound, based on an IP address and a port number which are pieces of reception terminal information. In addition, for example, when a user A cancels a check box of “sound transmission” displayed on the display section 106 of the terminal device 10 a, the communication channel establishment section 14 cancels the established first communication channel.
When the first communication channel is established by the communication channel establishment section 14, the sound transmitting section 15 of the terminal device 10 a transmits (distributes) sound data which is input to the sound input section 107 to each of the terminal devices 10 through the first communication channel. For example, the sound transmitting section 15 transmits sound data corresponding to the speaking (sound) of the user A which is input to the sound input section 107 of the terminal device 10 a to each of the terminal devices 10 b, 10 c, 10 d, 10 e, and 10 f through a first communication channel (single line in FIG. 11), and transmits sound data corresponding to the speaking (sound) of a user F which is input to the sound input section 107 of the terminal device 10 f to each of the terminal devices 10 a, 10 b, 10 c, 10 d, and 10 e through a first communication channel (double line in FIG. 11). Thereby, the speaking (sound data) of the user A and the user F may be transmitted to another terminal device 10, for example, at the same timing. Meanwhile, the sound transmitting section 15 may perform stream transmission (streaming) on each of the terminal devices 10, or may transmit a sound data file thereto. In addition, when both a transmission terminal device and a reception terminal device are incorporated with a web browser, a sound stream of the transmission terminal device may be connected to the reception terminal device using web real-time communication (web RTC).
The display control section 16 of the terminal device 10 a displays display contents according to the determination result acquired by the determination result acquisition section 12 on the display section 106. For example, when the determination result acquisition section 12 acquires a permission notification for sound transmission from the server 20, the display control section 16 displays a message indicating that sound transmission has been permitted, on the display section 106. In addition, when the determination result acquisition section 12 acquires a non-permission notification for sound transmission from the server 20, the display control section 16 displays an error message indicating the non-permission of sound transmission on the display section 106, and makes a change so that a selection screen (a check box or an electronic button) of “sound transmission” displayed on the display section 106 cannot be selected. For example, an electronic button for selecting sound transmission (transmission request) may be displayed in a grayed-out (inactivated) manner so as not be capable of being selected (pressed down), or a note indicating that “now, transmission request is not available” may be displayed on the display section 106. In this manner, the display control section 16 may control what is displayed by the display section 106 so that a user cannot perform new sound transmission (transmission request).
The determination result acquisition section 42 of the terminal device 10 b receives (or acquires) the determination result obtained by the determination processing section 23 of the server 20. For example, the determination result acquisition sections 42 of the respective terminal devices 10 a to 10 f acquire a permission notification for sound reception from the server 20.
When the permission notification is acquired by the determination result acquisition section 42, the display control section 43 of the terminal device 10 b displays a message indicating that sound reception has been permitted, on the display section 106. In addition, when the determination result acquisition section 42 acquires a non-permission notification for sound reception from the server 20, the display control section 43 displays an error message indicating the non-permission of sound reception on the display section 106 and makes a change so that a selection screen (a check box or an electronic button) of “sound reception” displayed on the display section 106 cannot be selected.
The sound receiving section 44 of the terminal device 10 b receives sound data which is transmitted through the first communication channel. The sound output section 108 converts the received sound data to output a sound. For example, the sound receiving sections 44 of the respective terminal devices 10 b, 10 c, 10 d, 10 e, and 10 f receive sound data corresponding to the speaking (sound) of a user A which is transmitted through the first communication channels (single lines in FIG. 11) from the terminal device 10 a, and the sound output sections 108 output a sound. In addition, the sound receiving sections 44 of the respective terminal devices 10 a, 10 b, 10 c, 10 d, and 10 e receive sound data corresponding to the speaking (sound) of a user F which is transmitted through the first communication channels (double lines in FIG. 11) from the terminal device 10 f, and the sound output sections 108 output a sound. Thereby, for example, users B, C, D, and E of the respective terminal devices 10 b, 10 c, 10 d, and 10 e hear what the user A says and what the user F says, at the same timing.
Next, a flow of a process in the server 20 will be described with reference to a flow chart. FIG. 12 is a flow chart in a case where the server 20 receives a transmission request from the terminal device 10.
First, when a transmission request is transmitted from the terminal device 10, the transmission request receiving section 21 receives the transmission request (S101). Next, the determination processing section 23 compares the total number of terminal devices 10 making a transmission request with the predetermined number of terminal devices, and determines whether the total number is equal to or less than the set number (S102).
When the total number is equal to or less than the set number, the determination processing section 23 transmits a permission notification for sound transmission to the terminal device 10 making a transmission request, and the terminal information generation section 24 generates or updates a transmission terminal list (see FIG. 6) (S103). Subsequently, the terminal information generation section 24 transmits a reception terminal list in which pieces of information of terminal devices making a reception request are listed, to the terminal device 10 registered in the transmission terminal list (S104).
In the process of S102, when the total number exceeds the set number, the determination processing section 23 transmits a non-permission notification for sound transmission to the terminal device 10 having transmitted a transmission request (S105). The processes of S101 to S105 are repeated until an electronic conference is terminated (S106).
FIG. 13 is a flow chart in a case where the server 20 has received a reception request from the terminal device 10.
First, when a reception request is transmitted from the terminal device 10, the reception request receiving section 22 receives the reception request (S201). Next, the terminal information generation section 24 generates or updates a reception terminal list (see FIG. 7) (S202). Next, the terminal information generation section 24 transmits a permission notification for sound reception to the terminal device 10 registered in the reception terminal list (S203). Meanwhile, here, a case where sound reception is permitted with respect to all reception requests received by the reception request receiving section 22 is described, but the determination processing section 23 may permit sound reception only in a case of being equal to or less than the predetermined number, and may transmit a non-permission notification for sound reception to the terminal device 10 in a case of non-permission. Next, the terminal information generation section 24 acquires a transmission terminal list, and transmits a reception terminal list to each of the terminal devices 10 registered in the transmission terminal list (S204). Meanwhile, when the reception terminal list is changed (updated), only a changed (added) reception terminal list may be transmitted. The processes of S201 to S204 are repeated until the electronic conference is terminated (S205).
FIG. 14 is a flow chart in a case where a user makes a transmission request in the terminal device 10.
First, in an electronic conference, the terminal device 10 receives a transmission request from a user (S301). Next, the transmission request transmitting section 11 transmits the received transmission request to the server 20 (S302). When the server 20 receives the transmission request, the processes of S101 to S105 illustrated in FIG. 12 are performed.
Next, the determination result acquisition section 12 of the terminal device 10 acquires a transmission permission notification or a transmission non-permission notification from the server 20 (S303). When the transmission permission notification is acquired, the terminal information acquisition section 13 acquires a reception terminal list from the server 20 (S304).
Next, the communication channel establishment section 14 acquires terminal information (terminal ID) of any one terminal device 10 from the reception terminal list (S305). Next, the communication channel establishment section 14 establishes a first communication channel for transmitting sound data to the terminal device 10 corresponding to the acquired terminal information (S306). The communication channel establishment section 14 performs a process of establishing the first communication channel on all of the terminal devices 10 registered in the reception terminal list (S307).
Next, the sound transmitting section 15 transmits sound data to the terminal devices 10 through the established first communication channel (S308).
In the process of S303, when the determination result acquisition section 12 acquires the transmission non-permission notification from the server 20, the display control section 16 of the terminal device 10 displays an error message on the display section 106 (S309). The processes of S301 to S309 are repeated until the electronic conference is terminated (S310).
As described above, the electronic conference system 100 is configured to set an upper limit for the number of transmission terminals having a right to speak (transmitting a sound) and to dynamically change a terminal device permitted to perform transmission in response to a user's instruction to thereby establish a first communication channel between the terminal device permitted to perform transmission and another terminal device.
Second Communication channel Establishment Process
The electronic conference system 100 according to the exemplary embodiment is configured to perform a process for establishing a communication channel (second communication channel) for transmitting and receiving multimedia data which is limited to between specific terminal devices (hereinafter, referred to as a limited communication channel), in addition to being configured to perform the “first communication channel establishment process”. For example, in a state where the communication channels (first communication channels) illustrated in FIG. 11 are established during a conference, a limited communication channel limited to between the terminal device 10 c and the terminal device 10 d is established so that the user C of the terminal device 10 c and the user D of the terminal device 10 d have a conversation with each other without being heard by other users. Meanwhile, data which is transmitted and received through the limited communication channel may be video data without being limited to sound data. Hereinafter, a description will be given by taking sound data (second sound data) as an example.
FIG. 15 is a functional block diagram illustrating a specific configuration for performing a second communication channel establishment process of the electronic conference system 100. As illustrated in FIG. 15, the server 20 further includes a limited call request receiving section 25 in addition to the components illustrated in FIGS. 3 and 4, and the terminal device 10 further includes a limited call request transmitting section 51 and a permission/rejection result transmitting section 52 in addition to the components illustrated in FIGS. 3 and 4. Meanwhile, for convenience of description, some of the components illustrated in FIGS. 3 and 4 are shown in the terminal device 10 illustrated in FIG. 15, and the other components are omitted.
Hereinafter, the second communication channel establishment process will be described with reference to a flow chart of FIG. 16 based on a flow of a process of the electronic conference system 100 according to the progress of an electronic conference.
First, when an electronic conference is performed in a state where the first communication channels illustrated in FIG. 11 are established, in a case where the user C desires to talk with the user D without being heard by other users, that is, in a case where the user C desires to establish a limited communication channel for transmitting and receiving sound data between the terminal device 10 c and the terminal device 10 d, the user C operates the operation section 105 of the terminal device 10 c used by himself or herself to give instruction in that effect. Specifically, for example, the user C selects “limited call” on a screen such as a check box displayed on the display section 106 of the terminal device 10 c using a keyboard, a mouse, or the like, and selects a counterpart (here, the user D) who the user desires to talk with. The limited call request transmitting section 51 of the terminal device 10 c transmits the received request of the user C (hereinafter, referred to as a limited call request) to the server 20 (S11). Meanwhile, for example, when the user C cancels the check box of “limited call” or a predetermined condition (described later) is satisfied, the limited call request transmitting section 51 transmits a cancellation request for the limited call. For example, the limited call request transmitting section 51 transmits “+1” when transmitting the limited call request and transmits “−1” when transmitting the cancellation request.
The limited call request receiving section 25 (establishment request receiving section) of the server 20 receives the limited call request transmitted from the limited call request transmitting section 51 of the terminal device 10. For example, when a limited call request indicating that a limited call between the terminal devices 10 c and 10 d is desired is transmitted from the terminal device 10 c, the limited call request receiving section 25 receives the limited call request from the terminal device 10 c.
When the determination processing section 23 of the server 20 receives the limited call request, a message for inquiring the permission or rejection of a limited call is transmitted to the counterpart terminal device 10 of the limited call. For example, the determination processing section 23 transmits the message to the terminal device 10 d (S12).
When the display control section 16 of the terminal device 10 receives the message, a screen for selecting whether to permit a limited call is displayed on the display section 106 of the terminal device 10. For example, the display control section 16 of the terminal device 10 d of the user D displays “ terminal devices 10 c and 10 d” which is information of the terminal device 10 as an object for a limited call and “users C and D” which is information of users as objects for a limited call on the display section 106, and displays a selection screen for selecting “permission” indicating the permission of a limited call or “rejection” indicating the rejection of a limited call.
The permission/rejection result transmitting section 52 of the terminal device 10 transmits an answer for the message to the server 20. For example, when the user D selects “permission”, the permission/rejection result transmitting section 52 of the terminal device 10 d transmits a reply indicating that a limited call between the users C and D is permitted, to the server 20 (S13). When the user D selects “rejection”, the permission/rejection result transmitting section 52 of the terminal device 10 d transmits a reply indicating that a limited call between the users C and D is rejected, to the server 20.
When the determination processing section 23 (permission/rejection receiving section) of the server 20 receives the reply from the permission/rejection result transmitting section 52 of the terminal device 10, the determination processing section transmits the reply to the terminal device 10 having transmitted the limited call request. For example, when the determination processing section 23 receives “permission” from the permission/rejection result transmitting section 52 of the terminal device 10 d, the determination processing section transmits “permission” to the terminal device 10 c (S14). When the determination processing section receives “rejection” from the permission/rejection result transmitting section 52 of the terminal device 10 d, determination processing section transmits “rejection” to the terminal device 10 c.
When the communication channel establishment section 14 (second communication channel establishment section) of the terminal device 10 receives “permission” from the determination processing section 23 of the server 20, the communication channel establishment section establishes a limited communication channel for transmitting sound data between the communication channel establishment section and another terminal device 10 as an object for a limited call. For example, when a limited call between the users C and D is permitted, the communication channel establishment section 14 of the terminal device 10 c acquires a sound stream and establishes a communication channel for the terminal device 10 d (transmits the sound stream) (S15). When the communication channel establishment section 14 of the terminal device 10 d receives a sound stream from the terminal device 10 c, the communication channel establishment section acquires the sound stream and establishes a communication channel for the terminal device 10 c (returns the sound stream) (S16). Thereby, the communication channel limited to between the terminal devices 10 c and 10 d is established. Meanwhile, when the terminal device 10 c receives “rejection” from the determination processing section 23 of the server 20, the display control section 16 of the terminal device 10 c may display an error message on the display section 106.
FIG. 17 illustrates a network configuration in which a limited communication channel is established by the process shown in FIG. 16. Arrows of dotted lines in FIG. 17 indicate a limited communication channel. According to the above-described configuration, a communication channel limited to between the terminal devices 10 c and 10 d is established, and thus a limited call is made between the users C and D without allowing other users to hear what the users C and D communicate. Meanwhile, in the above-described configuration, a communication channel established between terminal devices is a communication channel capable of transmitting data in one direction, but is not limited thereto. The communication channel may be one communication channel capable of bidirectionally transmitting data.
In the network configuration illustrated in FIG. 17, when a limited call is started between the users C and D by a limited communication channel being established, the sound output sections 108 of the respective terminal devices 10 c and 10 d may output sounds (sounds during broadcasting) of speakers (users A and F) which are received by the sound receiving section 44 (see FIG. 4) by lowering the volume of the sounds. Thereby, it is possible to easily perform a limited call between the users C and D while understanding the summary of the sounds (sounds of the users A and F) being broadcasted.
In addition, when a limited call is not made for a fixed period of time after the limited communication channel is established, the communication channel establishment section 14 may cancel the limited communication channel. In this case, the limited call request transmitting section 51 transmits a cancellation request for a limited call to the server 20.
In addition, the plural terminal devices 10 in which a limited communication channel is established may include terminal devices 10 used by a speaker (presenter in a conference). For example, in the network configuration illustrated in FIG. 11, a limited communication channel may be established between the terminal device 10 a used by the user A (presenter in a conference) who is a speaker and the terminal device 10 b used by the user B (participant in a conference) other than the speaker. For example, when the user B desires to transmit a comment to the user A without being heard by other users (desires to whisper), that is, when the user B desires to establish a limited communication channel for transmitting sound data between the terminal device 10 b and the terminal device 10 a, the establishment of the limited communication channel is realized by performing a second communication channel establishment process illustrated in FIG. 18. Processes illustrated in FIG. 18 are the same as the processes of S11 to S15 illustrated in FIG. 16. FIG. 19 illustrates a network configuration in which a limited communication channel is established by the process of FIG. 18. Meanwhile, the process illustrated in FIG. 18 may be used between terminal devices 10 used by a user (participant in a conference) other than a speaker. For example, the process may also be used when the user B desires to transmit a comment to the user C (desires to whisper) without being heard by other users.
Here, in the process illustrated in FIG. 18, a sound stream is not returned from the terminal device 10 a of the user A who is a speaker. However, a sound is not broadcasted from the terminal device 10 a, a limited communication channel from the terminal device 10 a to the terminal device 10 b may be established by returning the sound stream from the terminal device 10 a. In addition, in the process illustrated in FIG. 18, a sound of the user B is not transmitted to the terminal device 10 a in real time, and the sound of the user B may be temporarily stored and may be transmitted to the terminal device 10 a at a predetermined timing, for example, at a timing when the speaking of the user A is stopped for a fixed period of time.
The number of terminal devices 10 as objects for the establishment of a limited communication channel is not limited to two, and may be three or more. FIG. 20 is a flow chart illustrating a second communication channel establishment process among three terminal devices 10. Hereinafter, a description will be given focusing on differences from the second communication channel establishment process illustrated in FIG. 16.
For example, when the user C desires to have a conversation among three people including the users D and E without being heard by other users, that is, when the user C desires to establish a limited communication channel for transmitting and receiving sound data between the terminal device 10 c, the terminal device 10 d, and the terminal device 10 e, the user C selects “limited call” and selects parties (here, the users D and E) who the user C desires to talk with. The limited call request transmitting section 51 of the terminal device 10 c transmits a received limited call request of the user C to the server 20 (S21). Meanwhile, for example, when the user C cancels a check box of “limited call” or a predetermined condition is satisfied, the limited call request transmitting section 51 transmits a cancellation request for the limited call.
When a limited call request indicating that a limited call among the terminal devices 10 c, 10 d, and 10 e is desired is transmitted from, for example, the terminal device 10 c, the limited call request receiving section 25 of the server 20 receives the limited call request from the terminal device 10 c.
When the determination processing section 23 of the server 20 receives the limited call request, the determination processing section transmits the message to the terminal devices 10 d and 10 e (S22).
When the display control sections 16 of the respective terminal devices 10 d and 10 e of the users D and E receive the message, “ terminal devices 10 c, 10 d, and 10 e” which is information of the terminal device 10 as a target for a limited call and “user C, D, E” which is information of a user as an object for a limited call are displayed on each of the display sections 106, and a selection screen for selecting “permission” or “rejection” is displayed thereon.
When the user D selects “permission”, the permission/rejection result transmitting section 52 of the terminal device 10 d transmits a reply indicating that a limited call among the users C, D, and E is permitted, to the server (S23). When the user D select “rejection”, the permission/rejection result transmitting section 52 of the terminal device 10 d transmits a reply indicating that a limited call among the users C, D, and E is rejected, to the server 20. In addition, when the user E selects “permission”, the permission/rejection result transmitting section 52 of the terminal device 10 e transmits a reply indicating that a limited call among the users C, D, and E is permitted, to the server (S23). When the user E selects “rejection”, the permission/rejection result transmitting section 52 of the terminal device 10 e transmits a reply indicating that a limited call among the users C, D, and E is rejected, to the server 20.
When the determination processing section 23 of the server 20 receives “permission” from the permission/rejection result transmitting section 52 of the terminal device 10 d, the determination processing section transmits “permission between terminal devices 10 c and 10 d” and an instruction to establish a communication channel for the terminal device 10 d (instruction to transmit a sound stream) to the terminal device 10 c (S24). In addition, when the determination processing section 23 of the server 20 receives “permission” from the permission/rejection result transmitting section 52 of the terminal device 10 e, the determination processing section transmits “permission between terminal devices 10 c and 10 e” and an instruction to transmit a sound stream for the terminal device 10 e to the terminal device 10 c (S24). In addition, when the determination processing section 23 of the server 20 receives “permission” from the permission/rejection result transmitting sections 52 of the respective terminal devices 10 d and 10 e, the determination processing section transmits “permission between terminal devices 10 d and 10 e” and an instruction to transmit a sound stream for the terminal device 10 e to the terminal device 10 d (S24).
When “rejection” is received from the permission/rejection result transmitting section 52 of each of the terminal devices 10 d and 10 e, the determination processing section 23 transmits a reply indicating that a limited call among the users C, D, and E is rejected, to the terminal device 10 c. In addition, when “permission” is received from one of the permission/rejection result transmitting sections 52 of the respective terminal devices 10 d and 10 e and when “rejection” is received from the other permission/rejection result transmitting section 52, the determination processing section 23 transmits a reply in that effect to the terminal device 10 c.
In this manner, when the determination processing section 23 receives “permission” from the counterpart terminal device 10 of the limited call, the determination processing section gives instruction to transmit a sound stream for the counterpart terminal device 10 to the terminal device 10 desiring the limited call. In addition, when there are plural terminal devices 10 which are opposite parties permitting the limited call, the determination processing section 23 gives instruction to transmit a sound stream for a terminal device 10 (here, the terminal device 10 e, see FIG. 7) having a large terminal ID, to a terminal device 10 (here, the terminal device 10 d, see FIG. 7) having a small terminal ID.
When a limited call among the users C, D, and E is permitted, the communication channel establishment section 14 of the terminal device 10 c acquires a sound stream, and establishes a communication channel for each of the terminal devices 10 d and 10 e (transmits the sound stream) (S25). When the communication channel establishment section 14 of each of the terminal devices 10 d and 10 e receives a sound stream from the terminal device 10 c, the communication channel establishment section acquires the sound stream and establishes a communication channel for the terminal device 10 c (returns the sound stream) (S26). In addition, the communication channel establishment section 14 of the terminal device 10 d acquires the sound stream and establishes a communication channel for the terminal device 10 e (returns the sound stream) (S27). When the communication channel establishment section 14 of the terminal device 10 e receives a sound stream from the terminal device 10 d, the communication channel establishment section acquires the sound stream and establishes a communication channel for the terminal device 10 d (returns the sound stream) (S28). Thereby, the communication channel limited to among the terminal devices 10 c, 10 d, and 10 e is established.
Meanwhile, when a reply indicating a limited call among the users C, D, and E is rejected is received from the determination processing section 23 of the server 20, the display control section 16 of the terminal device 10 c may display an error message on the display section 106. In addition, for example, when “permission” is received from the permission/rejection result transmitting section 52 of the terminal device 10 d and “rejection” is received from the permission/rejection result transmitting section 52 of the terminal device 10 e, the communication channel establishment sections 14 of the respective terminal devices 10 c and 10 d establish a limited communication channel which is limited to only between the terminal devices 10 c and 10 d. Thereby, the limited communication channel is established between the terminal devices 10 c and 10 d, and a limited communication channel is not established between the terminal devices 10 d and 10 e and between the terminal devices 10 c and 10 e. In this manner, in the plural terminal devices 10, a limited communication channel is established only between the terminal devices 10 permitting a limited call.
FIG. 21 illustrates a network configuration in which a limited communication channel is established by the process illustrated in FIG. 20. According to the above-described configuration, a communication channel limited to among the terminal devices 10 c, 10 d, and 10 e is established, and thus a limited call is made among the users C, D, and E without allowing other users to hear what the users C, D, and E communicate.
In the electronic conference system 100, an upper limit may be set for the number of terminal devices as objects for a limited communication channel, that is, the number of connections of a limited call. For example, the determination processing section 23 of the server 20 determines whether to permit a limited call based on the total number of pieces of “permission” received from the terminal device 10. Specifically, the determination processing section 23 compares the total number of pieces of “permission” received from the terminal device 10 with the predetermined number of pieces of “permission”, and determines whether the total number is equal to or less than the set number. When the total number is equal to or less than the set number, the determination processing section 23 transmits “permission” to a terminal device 10 making a limited call request and a terminal device 10 having a small terminal ID among the plural terminal devices 10 as objects for a limited call. When the total number exceeds the set number, the determination processing section notifies the terminal device 10 making a limited call request to reduce parties desiring a limited call. A user having received the notification selects a counterpart desiring a call and makes a limited call request again. According to such a configuration, it is possible to suppress a band of a sound stream.
The electronic conference system 100 is not limited to the above-described configuration, and various configurations maybe adopted. Hereinafter, an electronic conference system 100 according to a modification example will be described focusing on differences from the above-described configuration. In the following electronic conference system 100 according to the modification example, elements having functions common to those in the above-described electronic conference system 100 will not be described.
In an electronic conference system 100 according to Modification Example 1, the server 20 may further include a current location acquisition section 26 that acquires the current location of each of the terminal devices 10. FIG. 22 is a functional block diagram illustrating a specific configuration of the server 20 according to Modification Example 1.
For example, the current location acquisition section 26 receives a detection value of a position detection section (not shown) (for example, a GPS or the like) which is provided in each of the terminal devices 10 to acquire the current location (location or the like) of each of the terminal devices 10. In addition, the current location acquisition section 26 may set positional information such as an IP address and a subnet of a network, and may acquire the current location of each of the terminal devices 10 by acquiring an IP address of each of the terminal devices 10.
The determination processing section 23 determines the permission or non-permission of sound reception in a terminal device 10 having transmitted a reception request received by the reception request receiving section 22, based on the received reception request and the current locations of the respective terminal devices 10 which are acquired by the current location acquisition section 26.
The terminal information generation section 24 generates reception terminal information based on the reception request received by the reception request receiving section 22 and a result of the determination performed by the determination processing section 23. Specifically, the terminal information generation section 24 generates a reception terminal list in which pieces of information of terminal devices 10 permitted by the determination processing section 23 among the terminal devices 10 making a reception request are listed. The terminal information generation section 24 transmits the reception terminal list to the terminal device permitted to transmit a sound in the determination processing section 23.
Hereinafter, a description will be given using a specific example. FIG. 23 illustrates a network configuration in a case where the terminal devices 10 a, 10 b, and 10 c are provided in the same conference room 101, the terminal device 10 d is provided at a remote location P1 from the conference room 101, the terminal device 10 e is provided at a remote location P2 from the conference room 101, and the terminal device 10 f is provided at a remote location P3 from the conference room 101.
In the network configuration of FIG. 23, it is assumed that the terminal devices 10 a and 10 f transmit a transmission request and the terminal devices 10 a to 10 f transmits a reception request. Since the total number of terminal devices (two) is equal to or less than the set number of terminal devices (three), the terminal devices 10 a and 10 f are permitted to transmit a sound.
The current location acquisition section 26 acquires the current location from each of the terminal devices 10 a to 10 f. That is, the current location acquisition section 26 acquires the “conference room 101” as the current location of each of the terminal devices 10 a, 10 b, and 10 c, acquires the “remote location P1” as the current location of the terminal device 10 d, acquires the “remote location P2” as the current location of the terminal device 10 e, and acquires the “remote location P3” as the current location of the terminal device 10 f.
The determination processing section 23 determines whether the current locations of the terminal devices 10 are the same location or a location (predetermined location) within a predetermined range. When the plural terminal devices 10 are present at the predetermined location, sound reception is permitted for one terminal device 10 of the plural terminal devices 10, and sound reception is not permitted for the other terminal devices 10. In the network configuration of FIG. 23, sound reception is permitted for the terminal device 10 a among the terminal devices 10 a, 10 b, and 10 c that are present at a predetermined location (conference room 101), and sound reception is not permitted for the terminal devices 10 b and 10 c. The determination processing section 23 transmits a determination result (permission notification, non-permission notification) to a terminal device having transmitted a reception request.
The terminal information generation section 24 generates a reception terminal list illustrated in FIG. 24 based on the determination result, and transmits the generated reception terminal list to the terminal devices 10 a and 10 f. The terminal devices 10 a and 10 f establish a first communication channel based on the acquired reception terminal list. FIG. 25 illustrates a network configuration showing first communication channels established in the electronic conference system 100 according to Modification Example 1.
Meanwhile, when plural terminal devices 10 are present at a predetermined location and two or more terminal devices 10 among the plural terminal devices have made a transmission request, the determination processing section 23 may permit sound transmission for one terminal device 10 of the plural terminal devices 10 making a transmission request and may not permit sound transmission for the other terminal devices 10. For example, when the terminal devices 10 a, 10 b, and 10 c that are present in the conference room 101 have made a transmission request, the sound transmission of the terminal device 10 a is permitted, and the sound transmission of the terminal devices 10 b and 10 c is not permitted. The terminal information generation section 24 transmits a reception terminal list to the terminal device 10 a permitted to transmit a sound, and the terminal device 10 a establishes a first communication channel based on the acquired reception terminal list.
In an electronic conference having the above-mentioned network configuration, plural users (for example, users B and C) of the plural terminal devices 10 who are present in the same conference room directly hear what one user (for example, a user A) among the users speaks, and hear what the other users (for example, users D, E, and F) who are present at remote locations speak by any one terminal device 10 (for example, the terminal device 10 a) of the plural terminal devices 10 receiving and outputting what the other users speak. Therefore, according to the electronic conference system 100 of Modification Example 1, it is possible to reduce the number of first communication channels required.
In an electronic conference system 100 according to Modification Example 2, the number of terminal devices which is set may be dynamically changed. Specifically, the number of terminal devices may be set in accordance with the total number of terminal devices 10 connected to each other through a communication network 30. For example, a configuration may be adopted in which as the number of terminal devices 10 increases, the set number decreases. This configuration is suitable for, for example, an electronic conference such as a seminar in which a speaker (presenter) is determined. In addition, the set number may be calculated and set in accordance with a ratio to the total number of terminal devices 10 connected to each other through a network. Further, the set number may be set in accordance with the number of terminal devices 10 making a reception request.
In an electronic conference system 100 according to Modification Example 3, when a time for which sound data is not transmitted from a terminal device 10 permitted to transmit a sound after the establishment of a first communication channel reaches a predetermined time, a communication channel establishment section 14 may cancel the first communication channel corresponding to the terminal device 10. For example, when a user A permitted to transmit a sound has not spoken for a fixed period of time, the communication channel establishment section 14 forcibly cancels a first communication channel established in a terminal device 10 a of the user A. In addition, in the above-mentioned configuration, a user using a transmission terminal device may be captured by a camera, and the presence or absence of the user's speaking and a period of time of a speechless state may be measured by analyzing a captured image. The camera may be built into the terminal device 10, or may be installed outside (for example, a conference room) .
Here, in the exemplary embodiment, as illustrated in FIG. 14, when a transmission non-permission notification is acquired, the display control section 16 of the terminal device 10 displays an error message on the display section 106, but a configuration in a case where sound transmission is not permitted is not limited thereto.
For example, in an electronic conference system 100 according to Modification Example 4, when the total number of transmission terminal devices has reached the set number at a point in time when terminal devices 10 are connected to a server 20, control maybe performed so as that a check box cannot be checked. In addition, the control of a check box may be performed by acquiring information from the server 20 in accordance with an increase and decrease in the total number of transmission terminal devices. The total number of transmission terminal devices may be acquired by performing polling with respect to the server 20. However, preferably, a connected state may be set using a web socket or the like, and notice maybe given to each terminal device from the server 20 side.
Meanwhile, the electronic conference system 100 may have the following configuration.
The electronic conference system 100 is configured to restrict the number of people (number of transmission terminal devices) capable of speaking at the same time because the number of people (speaker terminals) who speak at the same time is limited at a certain moment in spite of a large number of terminal device being connected to each other and to dynamically change an establishment state of a first communication channel while switching between transmission terminal devices. Thereby, it is possible to transmit and receive sound data between a large number of terminal devices in a pseudo manner.
In addition, the electronic conference system 100 is configured to be able to select sound transmission or sound reception in each of the terminal devices and to notify the server 20 of a change in the state of sound transmission or sound reception in each of the terminal devices.
In addition, the server 20 notifies a terminal device in which sound transmission is selected of a list of terminal devices in which sound reception is selected. Meanwhile, at this time, notice of information regarding a terminal device that has not changed in a state so far may not be given. A transmission terminal device having been given notice makes a call to a reception terminal device (makes a connection request) so as to receive its own stream information only with respect to a reception terminal device which is newly registered.
In addition, in the electronic conference system 100 has a configuration in which the server 20 manages the total number of terminal devices capable of simultaneously performing transmission and a transmission request cannot be selected in a terminal device when the total number reaches an upper limit (set number).
In addition, the electronic conference system 100 may be configured such that a transmission terminal device includes two types of transmission terminal devices of a normal transmission terminal device and a temporary transmission terminal device, and the temporary transmission terminal device may perform transmission only for a fixed period of time and sound transmission is cancelled (check box for a transmission request is cancelled) after the fixed period of time elapses. In addition, the temporary transmission terminal device may be configured such that transmission may be performed only while a user pushes a sound transmission button. In addition, the temporary transmission terminal device may be configured such that a transmission request is cancelled when a speechless state is continued for a fixed period of time.
In addition, the electronic conference system 100 may be configured such that a user desiring to speak makes a transmission request in a terminal device and a nominated user in a specific terminal device (master terminal) is given a right to speak (permitted as a transmission terminal device).
The network configuration in the electronic conference system 100 is not limited to the configuration illustrated in FIG. 5, and the server 20 and any one terminal device 10 of the plural terminal devices 10 may be the same terminal, and a configuration in which the server 20 is not provided (FIG. 26) may be adopted. The configuration illustrated in FIG. 26 is realized by a program, corresponding to each component included in the electronic conference system 100, being downloaded to the terminal devices 10 a to 10 f. Specifically, the terminal devices 10 a to 10 f include the components illustrated in FIGS. 3, 4, and 15 in common, and a program stored in a memory of each terminal device is executed by a CPU of the terminal device, thereby realizing the electronic conference system 100. In addition, in a configuration in which the server 20 is not provided, when a transmission request (sound transmission) is made in a certain terminal device 10, a transmission request may be transmitted to all of the terminal devices 10, and all of the terminal devices 10 may share a terminal list DB 203 a.
In addition, in a second communication channel establishment process, the server 20 may manage information (terminal information) regarding all of the terminal devices connected to each other through a network, and terminal information may be transmitted to a certain terminal device when the terminal device requests a limited call.
In addition, in the above-described configuration, a terminal device 10 in which a transmission request is made may be configured such that a process of determining whether to permit sound transmission is performed before a transmission request is transmitted to another terminal device 10 and such that a transmission request is transmitted to another terminal device 10 when it is determined that sound transmission is permitted. Meanwhile, a program to be downloaded may be constituted by, for example, JavaScript (registered trademark). In this manner, each terminal device may function as a transmission terminal device or a reception terminal device depending on conditions of a conference. Meanwhile, a terminal device 10 having the program downloaded thereto or recorded thereon functions as an information processing apparatus. In addition, a computer having the program downloaded thereto may be a terminal device 10 or may be a server 20.
In addition, the configuration illustrated in FIG. 5 may be realized by a program, corresponding to each component included in the electronic conference system 100, being downloaded to the server 20. The server 20 having the program downloaded thereto or recorded thereon functions as an information processing apparatus.
The electronic conference system 100 is not limited to the transmission and reception of sound data. That is, the electronic conference system 100 may be used for the transmission and reception of multimedia data including at least one of sound data and video data (movie data). In addition, the electronic conference system 100 may be used in various types of electronic conferences such as a television conference and a telephone conference. Meanwhile, in the television conference, communication of videos captured by cameras built into respective terminal devices is bidirectionally performed between the terminal devices through the first communication channel and the limited communication channel. In addition, the electronic conference system 100 may also be used in a remote conference in which conference participants are located at positions distant from each other.
According to the above-described electronic conference system. 100, even when the number of terminal devices connected to each other through a network is large, high-quality data communication is realized between the terminal devices. In addition, each user may select a case where the user desires to transmit speaking (sound) or a case where the user desires to receive a sound in accordance with his or her intention, and a first communication channel is dynamically changed and established between terminal devices on a network in response to the user's intention. In addition, when the “second communication channel establishment process” is performed, it is possible to transmit and receive sound data (second multimedia data) which is limited to between specific terminal devices while transmitting and receiving sound data (first multimedia data) between plural terminal devices.
In the above description, an electronic conference system has been described as an example of an information processing system. However, the information processing system is not limited to the electronic conference system, and includes various systems that share multimedia data between terminal devices connected to each other through a network.
The foregoing description of the exemplary embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to understand the invention for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.

Claims

What is claimed is:

1. A non-transitory computer readable medium storing a program causing a computer to transmit and receive multimedia data, including at least one of sound data and video data, between a plurality of terminal devices connected to each other through a network, the program causing the computer to function as:

a transmission request receiving section that receives a transmission request for first multimedia data from at least one terminal device;

a reception request receiving section that receives a reception request for the first multimedia data from at least one another terminal device;

a first communication channel establishment section that establishes a first communication channel for transmitting and receiving the first multimedia data between the at least one terminal device making the transmission request and the at least one terminal device making the reception request;

an establishment request receiving section that receives, from a first terminal device, an establishment request for a second communication channel for limitedly transmitting and receiving second multimedia data between the first terminal device and a second terminal device, wherein the plurality of terminal devices transmitting and receiving the first multimedia data through the first communication channel include the first terminal device and the second terminal device;

a permission/rejection receiving section that receives a reply, indicating one of permission and rejection, to the establishment request from the second terminal device; and

a second communication channel establishment section that establishes the second communication channel between the first terminal device and the second terminal device when receiving a reply indicating the permission of the establishment request from the second terminal device.

2. The non-transitory computer readable medium according to claim 1,

wherein the second communication channel establishment section establishes

the second communication channel for transmitting the second multimedia data from the first terminal device to the second terminal device, and

the second communication channel for transmitting the second multimedia data to from the second terminal device the first terminal device.

3. The non-transitory computer readable medium according to claim 1,

wherein the first terminal device and the second terminal device transmit and receive second sound data through the second communication channel which is established between the first terminal device and the second terminal device, while transmitting and receiving first sound data through the first communication channel.

4. The non-transitory computer readable medium according to claim 3, the program causing the computer to further function as:

a sound output section that outputs a sound from each of the terminal devices,

wherein the sound output section makes an output volume of a sound corresponding to the first sound data lower than an output volume of a sound corresponding to the second sound data.

5. The non-transitory computer readable medium according to claim 1,

wherein if the first terminal device makes the transmission request for the first multimedia data and the second terminal device makes the reception request for the first multimedia data, the second communication channel establishment section

establishes the second communication channel for transmitting the second multimedia data from the second terminal device to the first terminal device, and

does not establish the second communication channel for transmitting the second multimedia data from the first terminal device to the second terminal device.

6. The non-transitory computer readable medium according to claim 1,

wherein (i) if the establishment request receiving section receives, from the first terminal device, the establishment request for the second communication channel for limitedly transmitting and receiving the second multimedia data between the plurality of terminal devices including the second terminal device and (ii) if a number of terminal devices permitting the establishment request is equal to or less than a predetermined value, the second communication channel establishment section establishes the second communication channel between the first terminal device and the terminal device permitting the establishment request, and

if the number of terminal devices permitting the establishment request exceeds the predetermined value, the second communication channel establishment section does not establish the second communication channel between the first terminal device and the terminal device permitting the establishment request.

7. The non-transitory computer readable medium according to claim 6,

wherein if the number of terminal devices permitting the establishment request exceeds the predetermined value, the establishment request receiving section receives again another establishment request in which a counterpart terminal device transmitting and receiving the second multimedia data is changed, from the first terminal device.

8. An information processing apparatus that transmits and receives multimedia data including at least one of sound data and video data between a plurality of terminal devices connected to each other through a network, the information processing apparatus comprising:

9. An information processing system that transmits and receives multimedia data including at least one of sound data and video data between a plurality of terminal devices connected to each other through a network, the information processing system comprising:

10. The non-transitory computer readable medium according to claim 1, wherein each of the first multimedia data and the second multimedia data includes at least one of sound data and video data.

11. The non-transitory computer readable medium according to claim 1, wherein the first multimedia data and the second multimedia data are different from each other.

12. The non-transitory computer readable medium according to claim 1, the second communication channel is limited to communication between the first terminal device and the second terminal device.

13. The non-transitory computer readable medium according to claim 1, the second multimedia data transmitted through the second communication channel is not received by any of the other terminal devices than the first and second terminal devices.