JP7363421B2 - Programs, communication methods, communication terminals and communication systems - Google Patents

Description

The present invention relates to a program, a communication method, a communication terminal, and a communication system.

Traditionally, companies, educational institutions, government agencies, etc. input audio signals into microphones to transmit audio signals to other communication terminals, and also transmit audio signals and camera images received from the other communication terminals. Video conferencing systems that play back on speakers and displays are in use. ``Webinar'', which is a seminar held on a video conferencing system, is generally recognized, and one of the genres of seminar is panel discussion. In a panel discussion, participants listen to a panel of experts (presenters) discuss a specific theme, guided by a moderator. When implementing a panel discussion using a video conferencing system, the images of the panelists and moderators (hereinafter sometimes referred to as speakers) are output on the conference screen, the audio is output from communication terminals, and the audience can view it. It is assumed that the At this time, since it is assumed that there will be as many as 100 or more people in the audience, a unidirectional distribution style is generally used in which the video and audio of the audience are not output. However, in panel discussions, there are cases where famous experts, influencers, or people close to the panelists are in the audience, and the panelists identify such participants and decide that they would like to have them take the stage during the panel discussion. A scene exists.

In such a video conferencing system, in order to select a destination using a destination list, other terminals participating in the session are easily registered in the destination list, and transmission of candidate destinations for the session is performed. A destination list management section that manages a destination list that stores terminal information, a session management section that manages information about transmission terminals participating in a session, and terminals that are participating in a session but are not registered in the destination list. A configuration has been disclosed that includes a notification unit that notifies that there is an unregistered terminal, and a registration control unit that controls registration of information on the unregistered terminal (for example, Patent Document 1).

However, with the technology described in Patent Document 1, there is a problem in that it is difficult to find a specific audience member to take the podium from among an unspecified number of audience members in a panel discussion held on a video conferencing system. . Additionally, since users have to register in the destination list, it is difficult to deal with situations where there are a large number of participants such as in a panel discussion, or where there are many participants who join in the middle of the meeting. There is a problem.

The present invention has been made in view of the above, and allows specific audience members to be easily selected even when there are a large number of audience members without having to register in advance a list of audience members for speaking on a video conference. The purpose of the present invention is to provide a program, a communication method, a communication terminal, and a communication system that allow a person to speak on stage.

In order to solve the above-mentioned problems and achieve the objects, the present invention provides an SNS service for a computer, which is a communication terminal that can make a video conference call with another communication terminal by transmitting and receiving audio signals via a network. Participant information derived from SNS information related to the user who uses the communication terminal acquired from the server of (Social Networking Service) and the identification information of the user are associated and registered in the management system extracted. a first receiving unit that receives participant information corresponding to the participant information that satisfies a search condition; and a display that displays the list of participant information received by the first receiving unit on a display device. a control unit; and an invitation notification for inviting a participant corresponding to information selected by the user from among information on the participants included in the list to participate as a speaker in the video conference. The present invention is characterized in that it is a program for functioning as a first transmitter that transmits data to other communication terminals used by participants.

According to the present invention, it is possible to easily select a specific audience member to take the podium even when there are a large number of audience members, without registering in advance a list of audience members to take the podium in a video conference.

FIG. 1 is a schematic configuration diagram of a communication system according to an embodiment. FIG. 2 is a diagram illustrating an example of the hardware configuration of the communication terminal according to the embodiment. FIG. 3 is a diagram illustrating an example of the hardware configuration of a smartphone as a communication terminal according to the embodiment. FIG. 4 is a diagram illustrating an example of the hardware configuration of the management system and program providing system according to the embodiment. FIG. 5 is a diagram illustrating an example of the software configuration of the communication terminal according to the embodiment. FIG. 6 is a diagram illustrating an example of the configuration of functional blocks of the communication system according to the embodiment. FIG. 7 is a diagram showing an example of a user management table. FIG. 8 is a diagram showing an example of a terminal management table. FIG. 9 is a diagram showing an example of a group management table. FIG. 10 is a diagram showing an example of a session management table. FIG. 11 is a diagram showing sessions established for transmitting and receiving content data and various types of management information in the communication system according to the embodiment. FIG. 12 is a sequence diagram illustrating an example of a preparation stage process including an authentication process for the communication terminal according to the embodiment to start a call. FIG. 13 is a diagram showing an example of a destination list screen. FIG. 14 is a sequence diagram illustrating an example of a conference start process of the communication system according to the embodiment. FIG. 15 is a diagram showing an example of a start request dialog box. FIG. 16 is a sequence diagram illustrating an example of SNS information acquisition processing of the communication system according to the embodiment. FIG. 17 is a diagram showing an example of an SNS ID registration screen. FIG. 18 is a sequence diagram illustrating an example of a conference screen display process when participating in a conference in the communication system according to the embodiment. FIG. 19 is a diagram showing an example of a speaker conference screen. FIG. 20 is a diagram showing an example of an audience conference screen. FIG. 21 is a sequence diagram illustrating an example of presence update processing of the communication system according to the embodiment. FIG. 22 is a sequence diagram illustrating an example of a meeting invitation process of the communication system according to the embodiment. FIG. 23 is a diagram illustrating an example of an audience conference screen on which a list of inviteable audience members is displayed. FIG. 24 is a diagram illustrating an example of an audience conference screen on which another list of inviteable audience members is displayed. FIG. 25 is a diagram illustrating an example of a speaker conference screen with a specific audience member selected from the inviteable audience list. FIG. 26 is a diagram illustrating an example of an invitation confirmation dialog box. FIG. 27 is a diagram illustrating an example of an invitation reception dialog box. FIG. 28 is a diagram illustrating an example of a speaker conference screen after an audience member has taken the stage. FIG. 29 is a diagram illustrating an example of a conference screen on the audience side after an audience member takes the stage. FIG. 30 is a diagram illustrating an example of a software configuration when using a web application of a communication terminal according to an embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a program, a communication method, a communication terminal, and a communication system according to the present invention will be described in detail below with reference to the drawings. Further, the present invention is not limited to the following embodiments, and the constituent elements in the following embodiments include those that can be easily conceived by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. is included. Furthermore, various omissions, substitutions, changes, and combinations of constituent elements can be made without departing from the gist of the following embodiments.

Furthermore, computer software refers to programs related to computer operations and other information used for processing by a computer that is equivalent to a program (hereinafter, computer software will be referred to as software). Application software is a general term for software used to perform specific tasks among software categories. On the other hand, an operating system (OS) is software that controls a computer and allows application software and the like to use computer resources. The operating system performs basic management and control of a computer, such as controlling input/output, managing hardware such as memory and hard disks, and managing processes. Application software operates using functions provided by the operating system. A program is a set of instructions to a computer that are combined to produce a single result. Also, what is said to be similar to a program refers to something that cannot be called a program because it is not a direct command to a computer, but has properties similar to a program in that it specifies computer processing. For example, a data structure (a logical structure of data expressed by mutual relationships between data elements) corresponds to a program.

(Overall configuration of communication system)
FIG. 1 is a schematic configuration diagram of a communication system according to an embodiment. The overall configuration of a communication system 1 according to this embodiment will be outlined with reference to FIG. 1.

As shown in FIG. 1, the communication system 1 includes a plurality of communication terminals 10aa, 10ab, 10ba, 10bb, 10ca, 10cb, 10da, 10db, 10ea, 10eb, and displays 120aa, 120bb, 120ca, 120db for each communication terminal. , a management system 50 , and a program providing system 90 , and are constructed so as to be able to communicate with each other via the communication network 2 .

In addition, although communication terminals 10aa, 10ab, 10ba, 10bb, 10ca, 10cb, 10da, 10db, 10ea, and 10eb are shown in FIG. 1, when any communication terminal among these is shown or collectively referred to, it is simply It will be referred to as "communication terminal 10." Further, the plurality of communication terminals 10 included in the communication system 1 shown in FIG. 1 is an example, and the number of communication terminals 10 may be different.

Further, in FIG. 1, displays 120aa, 120bb, 120ca, and 120db are shown, but when any display among these is shown or collectively referred to, it is simply referred to as "display 120." Further, the plurality of displays 120 included in the communication system 1 shown in FIG.

Further, the communication terminal 10 may be any of a dedicated device connected to a display (the above-mentioned display 120), a desktop PC (Personal Computer), a notebook PC, a smartphone, a tablet terminal, or the like. In FIG. 1, communication terminals 10aa, 10bb, 10ca, and 10db are shown as dedicated devices, communication terminals 10ab, 10ba, 10cb, and 10da are shown as notebook PCs, and communication terminals 10ea and 10eb are shown as smartphones.

The communication terminal 10 is a terminal (an example of a computer) that transmits and receives various information to and from other devices. The communication terminal 10 establishes a session with another communication terminal 10, and performs a telephone call by transmitting and receiving content data including audio data and image data (video data) in the established session. Thereby, in the communication system 1, a video conference between the plurality of communication terminals 10 is realized. Note that a video conference may be simply referred to as a "conference."

The management system 50 is a computer that centrally manages the communication terminals 10. The management system 50 establishes a session between the communication terminals 10, thereby realizing a video conference using a telephone call or the like between the communication terminals 10. When the management system 50 receives a session start request from a predetermined communication terminal 10, the management system 50 establishes a session between the communication terminal 10 that sent the start request (start request terminal) and the destination terminal, and starts a video conference. let Therefore, the management system 50 relays content data between the plurality of communication terminals 10 using the established session. Although the management system 50 establishes sessions between the plurality of communication terminals 10, the relay of content data may actually be performed by another relay device. In this embodiment, in order to simplify the explanation, it will be assumed that the management system 50 establishes a session and relays content data.

The program providing system 90 includes an auxiliary storage device (HDD (Hard Disk Drive), etc.) that stores terminal programs for realizing various functions or means in the communication terminal 10, and provides terminal programs to the communication terminal 10. This is a computer that provides programs (such as communication application A to be described later). The program providing system 90 also stores programs in the auxiliary storage device for making the management system 50 etc. implement various functions or means, and can send the corresponding programs to the management system 50 etc.

As shown in FIG. 1, the communication network 2 is constructed to include, for example, LANs (Local Area Networks) 2a to 2d, private lines 2ab and 2cd, base stations 2e and 2f, and the Internet 2i. Note that the communication network 2 is not limited to the configuration shown in FIG. 1, and may include other network devices, and may have locations where wireless communication is performed in addition to wired communication. .

LANs 2a to 2d and private lines 2ab and 2cd include routers 70a to 70d, 70ab and 70cd, respectively. The routers 70a to 70d, 70ab, and 70cd are network devices that select optimal routes for communication data. Note that when any router among the routers 70a to 70d, 70ab, and 70cd is indicated or collectively referred to, it is simply referred to as "router 70."

The communication terminals 10 (10aa, 10ab, . . . ) and the router 70a are communicably connected via the LAN 2a. Furthermore, the communication terminals 10 (10ba, 10bb, . . . ) and the router 70b are communicably connected via the LAN 2b. Further, the LAN 2a, the LAN 2b, and the router 70ab are communicably connected by a dedicated line 2ab, and are constructed within the area A.

On the other hand, the communication terminals 10 (10ca, 10cb, . . . ) and the router 70c are communicably connected via the LAN 2c. Further, the communication terminals 10 (10da, 10db, . . . ) and the router 70d are communicably connected via the LAN 2d. Further, LAN2c, LAN2d, and router 70cd are communicably connected by dedicated line 2cd, and are constructed within area B.

The base stations 2e and 2f transmit and receive wireless signals to and from the communication terminal 10 (for example, the communication terminals 10ea and 10eb as smartphones shown in FIG. 1), and mediate data communication between the communication terminal 10 and the Internet 2i. It is a radio station that

The networks in region A and region B are communicably connected via the Internet 2i by routers 70ab and 70cd, respectively.

Furthermore, the management system 50 and the program providing system 90 are communicably connected to each communication terminal 10 via the Internet 2i. Note that the management system 50 and the program providing system 90 may be installed in area A or area B, or may be installed in an area other than these.

In addition, in FIG. 1, four sets of numbers shown near each communication terminal 10, management system 50, each router 70, and program providing system 90 are simplified representations of IP (Internet Protocol) addresses in general IPv4. It is shown in For example, it is assumed that the IP address of the communication terminal 10aa is "1.2.1.3". Note that IPv6 may be used instead of IPv4, but to simplify the explanation, IPv4 will be used in the explanation.

Note that the configuration of the communication system 1 shown in FIG. 1 is an example, and is not limited to this configuration. That is, the number of each device and system shown in FIG. 1 is not limited to the number shown in FIG. In addition, although FIG. 1 shows the network configuration of two regions, regions A and B, the network may be within the same region, or it may be a structure in which three or more regions are connected by a network. Good too.

(Hardware configuration of communication terminal)
FIG. 2 is a diagram illustrating an example of the hardware configuration of the communication terminal according to the embodiment. The details of the hardware configuration of the communication terminal 10 will be described with reference to FIG. 2.

As shown in FIG. 2, the communication terminal 10 according to the present embodiment includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an auxiliary storage device 105. It includes a media drive 107 and an input device 108.

CPU 101 is an arithmetic device that controls the overall operation of communication terminal 10 . The ROM 102 is a non-volatile storage device that stores programs for the communication terminal 10 (such as communication application A to be described later). The RAM 103 is a volatile storage device used as a work area for the CPU 101.

The auxiliary storage device 105 is a nonvolatile storage device such as an HDD or an SSD (Solid State Drive) that stores various data such as image data, audio data, and video data. The media drive 107 is a device that controls reading and writing of data to and from the medium 106, such as a flash memory, under the control of the CPU 101. Media 106 is a storage device that is detachable from communication terminal 10 . Note that the medium 106 is not limited to a flash memory as long as it is a nonvolatile memory that reads and writes data under the control of the CPU 101, and may also be an EEPROM (Electrically Erasable and Programmable ROM) or the like.

The input device 108 is a device for inputting various information such as a mouse or a keyboard, and a button such as a power button.

The communication terminal 10 also includes a network I/F 111, an image sensor I/F 113, an audio input/output I/F 116, a display I/F 119, a camera 112, a microphone 114, and a speaker 115. .

The network I/F 111 is an interface for communicating data using the communication network 2. The network I/F 111 is, for example, a NIC (Network Interface Card) that is compliant with TCP (Transmission Control Protocol)/IP (Internet Protocol).

The image sensor I/F 113 is an interface for transmitting image data to and from the camera 112 that images a subject and obtains image data under the control of the CPU 101. The camera 112 includes a lens and a solid-state image sensor that converts light into electric charge to digitize an image (video) of a subject. The camera 112 is connected to the image sensor I/F 113 by a cable 112c. As the solid-state image sensor, a CMOS (Complementary Metal Oxide Semiconductor), a CCD (Charge Coupled Device), or the like is used.

The audio input/output I/F 116 is an interface that processes input/output of audio signals (audio data) between the microphone 114 that inputs audio and the speaker 115 that outputs audio under the control of the CPU 101. Microphone 114 and speaker 115 are connected to audio input/output I/F 116 by cable 114c and cable 115c, respectively.

Display I/F 119 is an interface for transmitting image data to display 120 (an example of a display device) under the control of CPU 101. The display 120 is a display device configured with a liquid crystal, an organic EL (Electro-Luminescence), or the like that displays an image of a subject, operation icons, and the like. Display 120 is connected to display I/F 119 by cable 120c. The cable 120c may be a cable for analog RGB (VGA) signals, or may be a cable for component video, such as HDMI (registered trademark) (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signal. It may be a cable for

The above-mentioned CPU 101, ROM 102, RAM 103, auxiliary storage device 105, media drive 107, input device 108, network I/F 111, image sensor I/F 113, audio input/output I/F 116, and display I/F 119 are an address bus and a data bus. are communicably connected to each other by a bus line 110 such as.

Note that the hardware configuration of the communication terminal 10 shown in FIG. 2 is an example, and may include components other than those shown in FIG. 2. Further, at least one of the camera 112, the microphone 114, and the speaker 115 may be used as an external device separate from the communication terminal 10.

FIG. 3 is a diagram illustrating an example of the hardware configuration of a smartphone as a communication terminal according to the embodiment. The details of the hardware configuration of the smartphone as the communication terminal 10 will be described with reference to FIG. 3.

As shown in FIG. 3, the communication terminal 10 as a smartphone according to the present embodiment includes a CPU 301, a ROM 302, a RAM 303, an EEPROM 304, a media I/F 307, and a GPS (Global Positioning System) receiving section 308. We are prepared.

The CPU 301 is a calculation device that controls the operation of the communication terminal 10 as a whole. The ROM 302 is a nonvolatile storage device that stores programs used to drive the CPU 301, such as the CPU 301 and an IPL (Initial Program Loader). RAM 303 is a volatile storage device used as a work area for CPU 301. The EEPROM 304 is a nonvolatile storage device that reads or writes various data such as smartphone programs under the control of the CPU 301.

The media I/F 307 is an interface that controls reading or writing (storage) of data to the media 306 such as a flash memory. The GPS receiving unit 308 is a circuit that receives GPS signals from GPS satellites.

The communication terminal 10 includes a long-distance communication circuit 311, an antenna 311a, a CMOS 312, an image sensor I/F 313, a microphone 314, a speaker 315, a sound input/output I/F 316, and a display 317 (an example of a display device). and a touch panel 318.

The long-distance communication circuit 311 is a circuit that wirelessly communicates with other devices via an antenna 311a. The CMOS 312 is a type of built-in imaging means that images a subject and obtains image data under the control of the CPU 301. Note that instead of a CMOS sensor, an imaging means such as a CCD (Charge-Coupled Device) sensor may be used. The image sensor I/F 313 is an interface that controls driving of the CMOS 312.

Microphone 314 is a built-in circuit that converts sound into electrical signals. The speaker 315 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and voice. The sound input/output I/F 316 is an interface that processes input/output of sound signals between the microphone 314 and the speaker 315 under the control of the CPU 301 .

The display 317 is a display device such as a liquid crystal or an organic EL display that displays an image of a subject, various icons, and the like. The touch panel 318 is an input device that allows the user to operate the communication terminal 10 by pressing the display 317.

The communication terminal 10 also includes a bus line 310. The bus line 310 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 301 shown in FIG. 3.

Note that the hardware configuration of the communication terminal 10 shown in FIG. 3 is an example, and the hardware configuration of the communication terminal 10 shown in FIG. 3 may include components other than those shown in FIG.

(Hardware configuration of management system, etc.)
FIG. 4 is a diagram illustrating an example of the hardware configuration of the management system and program providing system according to the embodiment. The details of the hardware configuration of the management system 50 and the program providing system 90 will be described with reference to FIG. 4.

First, the hardware configuration of the management system 50 will be described with reference to FIG. 4. As shown in FIG. 4, the management system 50 includes a CPU 201, a ROM 202, a RAM 203, an auxiliary storage device 205, a media drive 207, a display 208, a network I/F 209, a keyboard 211, a mouse 212, A DVD (Digital Versatile Disc) drive 214 is provided.

The CPU 201 is a calculation device that controls the operation of the management system 50 as a whole. The ROM 202 is a nonvolatile storage device that stores programs for the management system 50. The RAM 203 is a volatile storage device used as a work area for the CPU 201.

The auxiliary storage device 205 is a storage device such as an HDD or SSD that stores various data such as a user management DB 5001, a terminal management DB 5002, a group management DB 5003, and a session management DB 5004, which will be described later. The media drive 207 is a device that controls reading and writing of data to and from the recording medium 206 such as a flash memory under the control of the CPU 201 .

The display 208 is a display device configured with a liquid crystal, an organic EL, or the like that displays various information such as a cursor, menu, window, characters, or images. The network I/F 209 is an interface for communicating data using the communication network 2. The network I/F 209 is, for example, a TCP/IP compliant NIC.

The keyboard 211 is an input device for selecting letters, numbers, various instructions, moving a cursor, and the like. The mouse 212 is an input device for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like.

The DVD drive 214 is a device that controls reading and writing of data to and from a DVD 213 such as a DVD-ROM or a DVD-R (DVD Recordable), which is an example of a removable storage medium. In addition, as another example of the above-mentioned removable recording medium, it may be configured to be recorded and provided on a computer-readable recording medium such as a CD-R (Compact Disc Recordable) or a Blu-ray disc.

The above-mentioned CPU 201, ROM 202, RAM 203, auxiliary storage device 205, media drive 207, display 208, network I/F 209, keyboard 211, mouse 212, and DVD drive 214 can communicate with each other via bus lines 210 such as an address bus and a data bus. It is connected to the.

Note that the hardware configuration of the management system 50 shown in FIG. 4 is an example, and it is not necessary to include all the components shown in FIG. 4, or it may include other components.

Further, the program for the management system 50 described above may be recorded on a computer-readable recording medium such as the recording medium 206 or the DVD 213 and distributed as a file in an installable format or an executable format. .

Furthermore, since the program providing system 90 has the same hardware configuration as the above-described management system 50, the description thereof will be omitted. However, a program for the program providing system 90 for controlling the program providing system 90 is recorded in the ROM 202 . In this case as well, the program for the program providing system 90 may be a file in an installable or executable format and may be distributed by being recorded on a computer-readable recording medium such as the recording medium 206 or DVD 213. good.

(Software configuration of communication terminal)
FIG. 5 is a diagram illustrating an example of the software configuration of the communication terminal according to the embodiment. The details of the software configuration of the communication terminal 10 will be described with reference to FIG. 5.

A communication application A is installed on the communication terminal 10 as a client application. Here, the application means application software. As shown in FIG. 5, an OS (Operating System) 1020 and a communication application A operate on a work area 1010 of the RAM 103 (or RAM 303) of the communication terminal 10.

The OS 1020 is basic software that provides basic functions and manages the entire communication terminal 10. The communication application A is an application that operates under the control of the OS 1020 and is used to communicate (call) with other communication terminals 10 .

Note that examples of the communication protocol of the communication application A include SIP (Session Initiation Protocol), H.323, IRC (Internet Relay Chat), Jingle, and the like.

(Configuration of functional blocks of communication system)
FIG. 6 is a diagram illustrating an example of the configuration of functional blocks of the communication system according to the embodiment. The configuration of functional blocks of the communication system 1 according to this embodiment will be described with reference to FIG. 6.

<Configuration of functional blocks of communication terminal>
As shown in FIG. 6, the communication terminal 10 includes a communication section 11 (first receiving section, first transmitting section, second receiving section, second transmitting section), an operation input receiving section 12 (receiving section), and an imaging 13, a display control section 14, an audio input section 15, an audio output section 16, a storage/readout section 17, a storage section 18, an authentication request section 19, and a state management section 20.

The communication unit 11 is a functional unit that transmits and receives various data to and from other communication terminals 10 or each system via the communication network 2. The communication unit 11 is realized by execution of a communication application A, which is software, by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3), and by the network I/F 111 (or the long distance communication circuit 311 shown in FIG. 3).

Before the communication terminal 10 establishes a session with another communication terminal 10 and starts a video conference by phone call, the communication unit 11 sends information from the management system 50 to each communication terminal 10 as a destination terminal candidate and the communication terminal concerned. The reception of each status information indicating the status of the user (participant) using 10 is started. Here, the term "destination terminal candidate" refers to other communications that the communication terminal 10 uses as a destination candidate for use by a video conference partner, that is, a user (participant) who is a video conference partner who can be specified as a session partner. This is the terminal 10. That is, the communication terminal 10 cannot establish a session and cannot conduct a video conference with a communication terminal used by a user who is not set in advance as a user (participant) participating in the video conference.

In addition, status information includes terminal status information indicating the status of each communication terminal 10 and user status information (sometimes referred to as presence) indicating the status of the user (participant) using each communication terminal 10. shall be indicated. In addition, the terminal status information includes the operating status of each communication terminal 10 (online or offline status), and when online, the detailed status such as whether it is on a call or on standby (hereinafter referred to as communication status). ). In addition, the terminal status information includes not only the operating status and communication status of each communication terminal 10, but also information such as whether the cable is disconnected from the communication terminal 10, whether audio can be output but not images, or whether the audio is set not to be output. It may indicate various states, such as MUTE, but below, as an example, a case in which the operating state and communication state are particularly indicated will be explained.

The communication unit 11 transmits a conference start request to the management system 50 when the communication terminal 10 operates as a start request terminal. Here, the conference start request is information requesting the start of a session used for a video conference. The communication unit 11 also manages the conference start request, the user ID of the participant who uses (logs in) the start request terminal that is the source of the conference start request, and the user ID of the participant who uses the destination terminal. system 50. The user ID is information that uniquely identifies the user, and may be freely set by the user, or may be an e-mail address used by the user.

The operation input receiving unit 12 is a functional unit that receives various inputs from the user. The operation input receiving unit 12 is realized by the input device 108 shown in FIG. 2 (or the touch panel 318 shown in FIG. 3).

For example, when the user turns on the power button serving as the operation input reception unit 12, the power of the communication terminal 10 is turned on. Further, when the user turns off the power from the on state, the communication unit 11 transmits terminal status information to the management system 50 indicating that the power of the communication terminal 10 has been turned off, and then The power is completely turned off. Thereby, the management system 50 can grasp that the communication terminal 10 has changed from being powered on to being powered off.

The image capturing unit 13 is a functional unit that captures an image of a subject and obtains image data obtained by capturing the image. The imaging unit 13 executes the communication application A by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3), and by the camera 112 and the imaging device I/F 113 (or the CMOS 312 and the imaging device I/F 313 shown in FIG. 3). Realized.

The display control unit 14 is a functional unit that controls the display of image data and the like on the display 120 (or the display 317 shown in FIG. 3). The display control unit 14 is realized by execution of the communication application A by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3).

For example, before the communication terminal 10 as a video conference request source starts a video conference call with the communication terminal 10 as a desired destination, the display control unit 14 displays the destination terminal candidates received by the communication unit 11. A destination list including the candidate name of each destination terminal and the user ID of the participant who will use the destination terminal is displayed on the display 120, reflecting the status information.

The audio input unit 15 is a functional unit that inputs an audio signal after the participant's (speaker's) audio is converted into an audio signal by the microphone 114 (or the microphone 314 shown in FIG. 3). The audio input unit 15 is realized by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3) executing the communication application A, and the audio input/output I/F 116 (or the sound input/output I/F 316 shown in FIG. 3). .

The audio output unit 16 is a functional unit that outputs an audio signal to the speaker 115 (or the speaker 315 shown in FIG. 3) and causes the speaker 115 (speaker 315) to output audio. The audio output unit 16 is realized by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3) executing the communication application A, and the audio input/output I/F 116 (or the sound input/output I/F 316 shown in FIG. 3). .

The storage/readout unit 17 is a functional unit that stores various data in the storage unit 18 and performs processing to read out various data stored in the storage unit 18. For example, content data received when making a call to a destination terminal is overwritten and stored in the storage unit 18 each time the content data is received. Among these, an image is displayed on the display 120 (display 317) using the image data before being overwritten, and sound is output from the speaker 115 (speaker 315) using the audio data before being overwritten. The storage/reading unit 17 is realized by execution of the communication application A by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3). The storage unit 18 is realized by the RAM 103 and auxiliary storage device 105 shown in FIG. 2 (or the RAM 303 and EEPROM 304 shown in FIG. 3).

The authentication request unit 19 causes the communication unit 11 to log in to the management system 50 via the communication network 2 when the communication terminal 10 is powered on or when the operation input reception unit 12 accepts an authentication request operation. This is a functional unit that transmits authentication request information indicating that authentication of the communication terminal 10 is requested, and the current IP address of the communication terminal 10. The authentication request unit 19 is realized by execution of the communication application A by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3).

The state management section 20 is a functional section that manages state information (terminal state information, user state information) of the communication terminal 10 . Specifically, the state management unit 20 acquires state information (terminal state information, user state information) of the communication terminal 10 at a predetermined timing, and transmits it to the management system 50. The state management unit 20 is realized by execution of the communication application A by the CPU 101 shown in FIG. 2 (or the CPU 301 shown in FIG. 3).

Note that at least one of the display control section 14, the authentication request section 19, and the state management section 20 is realized by a hardware circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). Good too.

Moreover, each functional unit of the communication terminal 10 shown in FIG. 6 shows the function conceptually, and is not limited to such a configuration. For example, a plurality of functional units illustrated as independent functional units in the communication terminal 10 shown in FIG. 6 may be configured as one functional unit. On the other hand, the function of one functional unit of the communication terminal 10 shown in FIG. 6 may be divided into a plurality of functions and configured as a plurality of functional units.

<Configuration of functional blocks of management system>
As shown in FIG. 6, the management system 50 includes a communication section 51 (acquisition section, third transmission section), an authentication section 52, a state management section 53, an extraction section 54, a state acquisition section 55, and a session control section 52. 56, a storage/readout section 57, a storage section 58, a participant information management section 59 (registration section), and an audience extraction section 60. Note that both the participant information management section 59 and the audience extraction section 60 collectively correspond to an "extraction section." The storage unit 58 is realized by the auxiliary storage device 205 shown in FIG. 4, and stores a user management DB 5001, a terminal management DB 5002, a group management DB 5003, and a session management DB 5004, as shown in FIG. Each table managed in each DB stored in the storage unit 58 will be described below.

<<User management table>>
FIG. 7 is a diagram showing an example of a user management table.

The storage unit 58 stores a user management DB 5001 including a user management table shown in FIG. In the user management table, display name, password, SNS (Social Networking Service) ID, Role, presence, following ID, number of posts, number of followers, and icon information are managed in association with each other. The display name (name) is a name displayed on a video conference screen or the like to identify the user. A password is information used for login authentication.

The SNS ID is the ID of the SNS account used by the user identified by the user ID. A role is a role or position assigned in a panel discussion as a video conference. Roles include, for example, a ``presenter'' who actually conducts the discussion in a panel discussion, a ``moderator'' who is in charge of moderating the panel discussion, an ``audience'' who listens to the discussion by the speakers in the panel discussion, and a ``listener'' who listens to the discussions by the speakers in the panel discussion. There are "presenters (with restrictions)" who participate in the discussion as an audience member and take the stage upon receiving an invitation from the presenter.

Presence indicates the status of participants (users) in a panel discussion as a video conference, and is the above-mentioned user status information. Presence includes, for example, "online" when you are logged in via the communication terminal 10 but have not yet participated in the panel discussion, and "presenting" when you are participating as a speaker in the panel discussion after logging in. , ``Attending'' means that you are participating in a panel discussion as an audience member after logging in, and ``Offline'' means that you are logged off.

The following ID is the SNS ID of the user of the SNS that the user identified by the user ID follows in the SNS that is used. Here, following means registering an account of a specific SNS user. The number of posts is the number of texts, images, videos, etc. posted on the SNS. The number of followers is the number of users who have registered (followed) the SNS account used by the user identified by the user ID. The icon information is information on the icon of the SNS account used by the user identified by the user ID, or path information indicating the storage area on the auxiliary storage device 205 where the information is stored. .

For example, in the user management table shown in FIG. 7, the display name associated with the user ID "A_10aa" is "Komai", the password is "aaaa", the SNS ID is "a", the role is "presenter", and the presence is "presenting", the following ID is "b, c", the number of posts is "100", the number of followers is "500", and the icon information is "c://...". ing.

Note that the user ID is identification information such as letters, symbols, numbers, or various marks used to uniquely identify the user who uses the communication terminal 10. For example, the user ID is identification information such as an e-mail address used by the user. You can.

<<Terminal management table>>
FIG. 8 is a diagram showing an example of a terminal management table.

The storage unit 58 stores a terminal management DB 5002 including a terminal management table shown in FIG. In the terminal management table, for each terminal ID of each communication terminal 10, the terminal name, the user ID of the user who logged in to each communication terminal 10, the operating status of each communication terminal 10, the communication status with other communication terminals 10, and each The IP address of the communication terminal 10 is associated and managed. The terminal ID is information for identifying the communication terminal 10, and may be stored in the communication terminal 10 in advance or may be determined by being input directly into the communication terminal 10 by the user.

Here, the operating state includes online, which is a state in which the power is turned on and communication is possible or in progress, and offline, which is a state in which the power is not turned on and communication is not possible. Furthermore, the communication status includes, for example, "Calling", "Ringing", "Accepted", "Busy", and "None". "Calling" indicates a state in which another communication terminal 10 is called, that is, a state in which a conference start request for establishing a session used for a video conference is sent to another communication terminal 10 and a response is waited for. show. “Ringing” indicates a state where the terminal is being called by another communication terminal 10, that is, a state where a conference start request has been received from the other communication terminal 10 and the response to the received conference start request has not been completed. "Accepted" indicates a state in which a permission response has been completed to a conference start request from another communication terminal 10, but session establishment has not been completed, and a state in which permission has been granted to a conference start request sent by the own terminal. Indicates that the reception of the response has been completed, but the session establishment has not been completed. “Busy” indicates a state in which a session with another communication terminal 10 has been established and a call is being made by communicating content data in a video conference. “None” indicates that the communication terminal 10 is not communicating with other communication terminals 10 and is on standby.

For example, in the terminal management table shown in FIG. 8, the communication terminal 10ad with the terminal ID "10ad" has the terminal name "AD terminal", the logged-in user's user ID "C_10ad", and the operating state " "Online", the communication status is "Busy" indicating a state where another communication terminal 10 is calling, and the IP address is "1.2.1.6".

<<Group management table>>
FIG. 9 is a diagram showing an example of a group management table.

The storage unit 58 stores a group management DB 5003 including a group management table shown in FIG. In the group management table, users participating in each video conference registered in advance in the management system 50 are divided into groups, and user IDs of users (participants) included in the groups are managed. That is, in the group management table, a group ID for identifying a group and a user ID of a user included in the group are managed in association with each other.

For example, in the group management table shown in FIG. 9, the group whose group ID is "G002" is shown to include user IDs of "B_10ac," "J_10ca," and "G_10cb."

<<Session management table>>
FIG. 10 is a diagram showing an example of a session management table.

The storage unit 58 stores a session management DB 5004 including a session management table shown in FIG. In the session management table, for each session ID for identifying the session in which content data is communicated between the communication terminals 10, the terminal ID of the session start requesting terminal and the other party in the conference start request for establishing the session are shown. The terminal ID of the designated destination terminal is associated and managed.

For example, in the session management table shown in FIG. 10, the session identified by the session ID "se1" is between the start requesting terminal (communication terminal 10aa) with the terminal ID "10aa" and the destination terminal (communication terminal 10aa) with the terminal ID "10db". This indicates that the connection has been established with the communication terminal 10db).

Although the information managed in each table shown in FIGS. 7 to 10 is in table format, the information is not limited to this, and any information other than table format may be used as long as the managed information can be associated. (for example, a file in CSV (Comma-Separated Values) format).

Returning to FIG. 6, the description of the functional blocks of the management system 50 will be returned.

The communication unit 51 is a functional unit that transmits and receives various data to and from the communication terminal 10 or each system via the communication network 2. The communication unit 51 is realized by the execution of a program by the CPU 201 and the network I/F 209 shown in FIG.

The authentication unit 52 searches the user management table (see FIG. 7) in the storage unit 58 using the user ID and password included in the authentication request information received via the communication unit 51 as a search key, and searches the user management table (see FIG. 7). This is a functional unit that performs user authentication by determining whether the same user ID and password are managed in the same user ID and password. The authentication unit 52 is realized by executing a program by the CPU 201 shown in FIG.

The status management unit 53 is a functional unit that manages the presence (user status information) of the user management table shown in FIG. 7 and the operating status and communication status of the terminal management table shown in FIG. 8. In order to manage the operating state of the communication terminal 10 that has requested login authentication, the state management unit 53 stores the terminal ID of this communication terminal 10 and the user of the participant who is logged in to the communication terminal 10 in the terminal management table. The ID, the operating state of the communication terminal 10, and the IP address of the communication terminal 10 are stored and managed in association with each other.

When the communication terminal 10 changes from the off state to the on state by operation of the input device 108 by the user of the communication terminal 10, the state management unit 53 updates the terminal management table based on the information sent from the communication terminal 10 to turn on the power. Update the operating state from offline to online. In addition, when the communication terminal 10 changes from the on state to the off state by operation of the input device 108 by the user of the communication terminal 10, the state management unit 53 manages the terminal based on information sent from the communication terminal 10 indicating that the power is to be turned off. Update the operational status of a table from online to offline.

Further, when the user logs in via the communication terminal 10, the state management unit 53 updates the presence in the user management table from offline to online. Furthermore, when the user logs off via the communication terminal 10, the state management unit 53 updates the presence in the user management table to offline.

The state management unit 53 is realized by executing a program by the CPU 201 shown in FIG.

The extraction unit 54 searches the group management table shown in FIG. 9 using the user ID of the participant who is logged in to the target terminal to be processed, such as the communication terminal 10 that has requested login authentication, as a search key, and searches the group management table shown in FIG. This is a functional unit that reads the user ID of a participant who uses a destination terminal candidate (communication terminal 10 for calling in the same conference) with which a call can be made, that is, a session can be established. The extraction unit 54 is realized by executing a program by the CPU 201 shown in FIG.

The status acquisition unit 55 is a functional unit that searches the terminal management table shown in FIG. 8 using the user ID as a search key, and reads out the operating status and communication status of the communication terminal 10 being used for each user ID. Thereby, the status acquisition unit 55 can acquire the operating status and communication status of the destination terminal candidate that can communicate with the communication terminal 10 that has made the login authentication request. The status acquisition unit 55 is realized by executing a program by the CPU 201 shown in FIG.

The session control unit 56 is a functional unit that stores and manages the generated session ID, the terminal ID of the start requesting terminal, and the terminal ID of the destination terminal in association with each other in the session management table shown in FIG. Session control unit 56 performs control for establishing a session between communication terminals 10. The session control unit 56 is realized by executing a program by the CPU 201 shown in FIG.

The storage/reading unit 57 is a functional unit that stores information in various tables in the storage unit 58 and performs processing to read information from various tables stored in the storage unit 58. The storage/reading unit 57 is realized by executing a program by the CPU 201 shown in FIG.

The participant information management unit 59 acquires information regarding a specific user (hereinafter sometimes referred to as SNS information) from an external server of the SNS via the communication unit 51, and acquires the SNS ID and follow information from the SNS information. This is a functional unit that derives information such as ID, number of posts, number of followers, and icon information, and registers the information in the user management table (see FIG. 7) as participant information. The participant information management unit 59 is realized by executing a program by the CPU 201 shown in FIG.

The audience extraction unit 60 selects participants whose role is “auditor” and whose presence is “attending” from among the participant information acquired from the user management table by the participant information management unit 59. This is a functional unit that extracts information. The audience extraction unit 60 is realized by executing a program by the CPU 201 shown in FIG.

Note that at least one of the authentication section 52, state management section 53, extraction section 54, state acquisition section 55, session control section 56, participant information management section 59, and audience extraction section 60 is implemented using an ASIC, FPGA, or the like. It may also be realized by a hardware circuit.

Furthermore, the functions of each functional unit of the management system 50 shown in FIG. 6 are conceptually shown, and the structure is not limited to this. For example, a plurality of functional units illustrated as independent functional units in the management system 50 shown in FIG. 6 may be configured as one functional unit. On the other hand, the function of one functional section of the management system 50 shown in FIG. 6 may be divided into a plurality of parts and configured as a plurality of functional parts.

(Status of sending and receiving content data and various management information)
FIG. 11 is a diagram showing sessions established for transmitting and receiving content data and various types of management information in the communication system according to the embodiment. A session established for transmitting and receiving content data and various management information in the communication system 1 will be described with reference to FIG. 11.

In the example shown in FIG. 11, the communication system 1 has a management information session sei for transmitting and receiving various types of management information between the start request terminal, destination terminal A, and destination terminal B via the management system 50. Indicates an established condition. In this case, a content data session sed for transmitting and receiving image data and audio data is further established between the start request terminal, destination terminal A, and destination terminal B via the management system 50. That is, the content data session sed is a session that is directly used in the video conference. Note that this session concept is just an example; for example, an image data session may be divided by resolution.

(Transmission and reception processing of each management information in the preparation stage before the communication terminal starts a call)
FIG. 12 is a sequence diagram illustrating an example of a preparation stage process including an authentication process for the communication terminal according to the embodiment to start a call. FIG. 13 is a diagram showing an example of a destination list screen. With reference to FIGS. 12 and 13, the transmission and reception process of each piece of information in the preparation stage before the communication terminal 10aa starts a call will be described. Note that FIG. 12 shows a process in which various types of management information are transmitted and received by the management information session sei. Further, it is assumed that the conference group that includes the communication terminal 10aa, which is the start requesting terminal, is registered in advance in the group management table, using the group ID as a key, along with the user IDs of users who have the right to participate in the conference.

<Step S21>
First, when the user of the communication terminal 10aa presses the power button of the input device 108 shown in FIG. 2 (or the touch panel 318 shown in FIG. 3), the operation input reception unit 12 of the communication terminal 10aa accepts the power on, Power on the communication terminal 10aa.

<Step S22>
Then, the authentication request unit 19 of the communication terminal 10aa sends authentication request information indicating a login authentication request from the communication unit 11 to the management system 50 via the communication network 2, triggered by the above-mentioned power-on of the communication terminal 10aa. The IP address of the communication terminal 10aa is transmitted. This authentication request information includes the terminal ID (10aa) for identifying the communication terminal 10aa, which is the own terminal as the start request terminal, the user ID (A_10aa) of the user logging into the communication terminal 10aa, and the password. It is. The terminal ID, user ID, and password are data read from the storage section 18 by the storage/readout section 17 of the communication terminal 10aa and sent to the communication section 11. Further, when authentication request information is transmitted from the communication terminal 10aa to the management system 50, the management system 50, which is the receiving side, can grasp the IP address of the communication terminal 10aa, which is the sending side.

<Step S23>
Next, the authentication unit 52 of the management system 50 searches the user management table (see FIG. 7) using the user ID and password included in the authentication request information received via the communication unit 51 as a search key, and User authentication is performed by determining whether the same user ID and password are managed in the management table.

<Step S24-1>
If the authentication unit 52 determines that the request is for login authentication from the communication terminal 10aa used by a user with legitimate usage authority, the state management unit 53 of the management system 50 updates the terminal management table (FIG. ), the user ID and the IP address of the communication terminal 10aa are stored in association with each record indicated by the terminal ID and terminal name of the communication terminal 10aa. As a result, the terminal ID "10aa" of the communication terminal 10aa is managed in association with the user ID "A_10aa" and the IP address "1.2.1.3" in the terminal management table. Hereinafter, the user (participant) with user ID "A_10aa" may be referred to as "user A_10aa" or "participant A_10aa."

<Step S24-2>
Subsequently, the state management unit 53 sets the operating state "online" and the communication state "None" of the communication terminal 10aa, and adds the operating state to the record indicated by the terminal ID and terminal name of the communication terminal 10aa in the terminal management table. and the communication status are stored in association with each other. As a result, the terminal ID "10aa" of the communication terminal 10aa is managed in association with the operating state "online" and the communication state "None" in the terminal management table.

Furthermore, since the user A_10aa has logged in via the communication terminal 10aa, the state management unit 53 refers to the user management table and updates the presence corresponding to the user ID "A_10aa" to "online".

<Step S25>
The communication unit 51 of the management system 50 transmits authentication result information indicating the authentication result obtained by the authentication unit 52 to the start requesting terminal (communication terminal) that has transmitted the authentication request information via the communication network 2. 10aa). In the present embodiment, the following description will be continued assuming that the user who uses the communication terminal 10aa is authenticated by the authentication unit 52 as having legitimate usage authority.

<Step S26>
When the communication terminal 10aa receives authentication result information indicating that the user is authenticated as having legitimate usage authority, the communication unit 11 requests the management system 50 via the communication network 2 for a destination list. Send destination list request information indicating that Thereby, the communication unit 51 of the management system 50 receives the destination list request information.

<Step S27>
Next, the extraction unit 54 of the management system 50 searches the group management table (see FIG. 9) using the user ID "A_10aa" of the user who uses the start request terminal (communication terminal 10aa) as a search key, and searches the group management table (see FIG. 9) to find the user A_10aa. The user ID of a user who can make a phone call, that is, who belongs to the same group as user A_10aa, is extracted. Further, the extraction unit 54 searches the terminal management table using the extracted user ID as a search key, and extracts the terminal ID and terminal name corresponding to this user ID, that is, the terminal ID and terminal name of the destination terminal candidate.

<Step S28>
Next, the communication unit 51 of the management system 50 transmits destination list information including the user ID, terminal ID, and terminal name of the destination terminal candidates extracted by the extraction unit 54 to the start request terminal (communication terminal 10aa). .

<Step S29>
As a result, in the start requesting terminal (communication terminal 10aa), the communication section 11 receives the destination list information, and the storage/reading section 17 stores the destination list information in the storage section 18.

In this manner, in this embodiment, the management system 50 centrally manages the destination list information of all communication terminals 10, instead of managing destination list information at each communication terminal 10. As a result, even if a new communication terminal 10 is included in the communication system 1 or a communication terminal 10 already included in the communication system 1 is excluded, each communication The effort of changing destination list information on the terminal 10 side can be saved.

<Step S30>
The status acquisition unit 55 of the management system 50 also retrieves the terminal IDs of the destination terminal candidates extracted by the extraction unit 54 (“10ac”, “10ad”, “10ae”, “10ba”, “10bb”, “10db”). ) as the search key to search the terminal management table. Then, the status acquisition unit 55 acquires the operating status and communication status of each destination terminal candidate by reading out the corresponding operating status and communication status for each terminal ID of the destination terminal candidate.

<Step S31>
Next, for each destination terminal candidate, the communication unit 51 sends the terminal ID which is the search key used in step S30, the user ID of the corresponding destination terminal candidate, the operating state and the communication state to the start request. Send to terminal. Note that when the operating state is "offline", the state information does not include the communication state.

<Step S32>
Next, the storage/reading unit 17 of the start request terminal (communication terminal 10aa) sequentially stores the terminal ID, user ID, operating status, and communication status received from the management system 50 in the storage unit 18. Therefore, by receiving the terminal status information (operating status and communication status) of the destination terminal candidates, the start requesting terminal (communication terminal 10aa) can determine the current operating status and communication status of each of the destination terminal candidates with which the call can be made. Communication status can be obtained.

<Step S33>
Next, the display control unit 14 of the start requesting terminal (communication terminal 10aa) determines the operation status of the destination terminal candidate based on the destination list information stored in the storage unit 18 and the terminal status information of the destination terminal candidate. and create a destination list that reflects the communication status. Then, the display control unit 14 displays the destination list on the display 120 (or display 317) of the communication terminal 10aa at a predetermined timing. An example of the displayed destination list (destination list screen 1100) is shown in FIG.

As shown in FIG. 13, the destination list screen 1100 includes a destination terminal candidate user ID 1100-1, a destination terminal candidate terminal ID 1100-2, a terminal name 1100-3, and an icon 1100 that reflects status information. -4a to 1100-4c, and a role assignment button 1100-5. The icons include an offline icon 1100-4a that indicates that a call cannot be made offline, a callable icon 1100-4b that indicates that a call is available online, and a busy icon 1100-4c that indicates that a call is in progress online. There is. In the example of the destination list screen 1100 shown in FIG. 13, the user IDs of the users using the communication terminal 10 are "J_10ab", "B_10ac", "I_10db", and "C_10ad".

If the operating state of the destination terminal candidate is "online" and the communication state is "None," the display control unit 14 assigns a call enabled icon 1100-4b to this destination terminal candidate. In addition, if the operation status of the destination terminal candidate is "online" and the communication status is other than "None", the display control unit 14 displays a calling icon 1100-4c for this destination terminal candidate. assign. Then, when the operating state of the destination terminal candidate is "offline", the display control unit 14 assigns an offline icon 1100-4a to this destination terminal candidate.

Note that in other communication terminals 10, when the user turns on the power button, which is the input device 108 shown in FIG. , processing similar to the processing of steps S22 to S33 described above is performed.

(Meeting start process)
FIG. 14 is a sequence diagram illustrating an example of a conference start process of the communication system according to the embodiment. FIG. 15 is a diagram showing an example of a start request dialog box. The conference starting process in the communication system 1 according to the present embodiment will be described with reference to FIGS. 14 and 15.

In FIG. 14, an example will be described in which the communication terminal 10ac, whose user has been authenticated by the authentication process shown in FIG. 12, sends a conference start request. Based on the terminal status information of the destination terminal candidates received in step S31 of FIG. A call can be made with at least one communication terminal 10 among the terminals 10. In the following, a case will be described in which the communication terminal 10ad as a destination terminal receives a conference start request transmitted by the communication terminal 10ac as a start request terminal. It is also assumed that participant B_10ac of communication terminal 10ac is assigned the role of "moderator" in the panel discussion, and participant C_10ad of communication terminal 10ad is assigned the role of "auditor." Note that in the state before the process shown in FIG. 14 is started, the destination list screen 1100 (see, for example, FIG. 13) is displayed on the display 120 (or the display 317 shown in FIG. 3) of the communication terminal 10ac. shall be. The participant at the start requesting terminal can then select a desired conversation partner (participant at the destination terminal) from the destination list.

<Step S41>
First, on the destination list screen 1100, the participant at the communication terminal 10ac selects the destination terminal (communication terminal 10ad) that will be the destination requesting the start of the conference via the input device 108 (or the touch panel 318 shown in FIG. 3). and perform operations to start a meeting. Note that the participant may perform an operation to start the conference by selecting all the communication terminals 10 displayed on the destination list screen 1100 as destination terminals that are requested to start the conference. .

<Step S42>
When an operation to request the start of a conference is performed on the start request terminal (communication terminal 10ac), the communication unit 11 of the start request terminal sends a conference start request indicating a request to start the conference to the user of the participant of the communication terminal 10ac. The ID “B_10ac”, the user ID “C_10ad” of the participant of the destination terminal, and the IP address of the start requesting terminal are transmitted to the management system 50. As a result, when the communication unit 51 receives the conference start request, the management system 50 knows the IP address "1.2.1.3" of the start request terminal that is the sender.

<Step S43>
Then, the state management unit 53 of the management system 50 stores the user ID "B_10ac" of the participant of the start request terminal received with the conference start request and the user ID of the participant of the destination terminal in the terminal management table (see FIG. 8). The field portion of the communication status of each record containing "C_10ad" is changed. Specifically, the state management unit 53 changes the communication state of the record including the user ID "B_10ac" in the terminal management table to "Calling". Similarly, the state management unit 53 changes the communication state of the record including the user ID "C_10ad" in the terminal management table to "Ringing".

<Step S44>
Then, the session control unit 56 of the management system 50 generates a session ID for identifying a session (content data session sed) for executing communication with the destination terminal requested by the start requesting terminal. , and is registered in the session management table (see FIG. 10).

<Step S45>
Subsequently, the session control unit 56 enters the terminal ID of the start requesting terminal "10ac" into the fields of the terminal ID of the start requesting terminal and the terminal ID of the destination terminal of the record that includes the generated session ID in the session management table. , stores and manages the terminal ID "10ad" of the destination terminal.

<Step S46>
Next, the communication unit 51 transmits the session ID generated by the session control unit 56 to the start request terminal (communication terminal 10ac).

<Step S47>
Further, the communication unit 51 transmits the conference start request as well as the user ID "B_10ac" of the participant of the start request terminal, the session ID, and the IP address of the management system 50 to the communication terminal 10ad, which is the destination terminal. As a result, the destination terminal receives the conference start request and also learns the IP address "1.1.1.2" of the management system 50.

<Step S48>
When the communication unit 11 of the destination terminal receives the conference start request, the display control unit 14 of the destination terminal displays a start request dialog on the display 120 (or display 317) indicating that the conference start request has been received as shown in FIG. Display box 1200-1. A start request dialog box 1200-1 shown in FIG. 15 indicates that a conference start request has been received, and whether to start communication with the start request terminal (establish a session) and permit participation in the conference. This is a user interface for operating the The user can confirm that the conference start request has been received by viewing the start request dialog box 1200-1. The start request dialog box 1200-1 has a "Yes" button 1200-2 for allowing participation in the conference, and a "No" button 1200-3 for selecting not to allow participation in the conference. include.

<Step S49>
When the "Yes" button 1200-2 is pressed by operating the input device 108 (or touch panel 318) of the destination terminal, the operation input reception unit 12 of the destination terminal starts communication (session Establishment) and accept an operation to participate in the meeting.

<Step S50>
Next, the communication unit 11 of the destination terminal sends the user ID "C_10ad" of the participant of the destination terminal, the user ID "B_10ac" of the participant of the start requesting terminal, and the session ID together with the start response indicating participation in the conference. , to the management system 50.

<Step S51>
When the communication unit 51 of the management system 50 receives the start response, the state management unit 53 records the user ID “B_10ac” of the participant of the start request terminal received along with the start response and the participant of the destination terminal in the terminal management table. The field portion of the communication status of each record including the user ID "C_10ad" is changed. Specifically, the state management unit 53 changes the communication state of the record including the user ID "B_10ac" in the terminal management table to "Accepted". Similarly, the state management unit 53 changes the communication state of the record including the user ID "C_10ad" in the terminal management table to "Accepted".

<Step S52>
Next, the communication unit 51 transmits the user ID "C_10ad" of the participant of the destination terminal and the session ID along with the start response to the start request terminal.

<Steps S53, S54>
Upon receiving this start response, the start request terminal establishes a session by transmitting the session ID to the management system 50 using the communication unit 11. On the other hand, the destination terminal establishes a session by transmitting the session ID to the management system 50 using the communication unit 11.

<Steps S55 to S58>
When a session is established between the start request terminal and the destination terminal in steps S53 and S54, video data and audio data are exchanged between each other via the management system 50, and the entire group including the start request terminal and each destination terminal , video conferencing becomes possible. However, among the participants participating in a panel discussion that is a video conference, the communication terminal 10 (for example, the communication terminal 10ad) used by the participant whose role is assigned to the audience is not directly used for the discussion at the meeting. Since the participant does not participate, the video data and audio data of the participant are not transmitted to other communication terminals 10. That is, only the video data and audio data of participants whose roles are assigned to speakers (presenters and moderators) will be shared throughout the video conference.

(SNS information acquisition processing)
FIG. 16 is a sequence diagram illustrating an example of SNS information acquisition processing of the communication system according to the embodiment. FIG. 17 is a diagram showing an example of an SNS ID registration screen. SNS information acquisition processing in the communication system 1 according to this embodiment will be described with reference to FIGS. 16 and 17. The SNS information acquisition process shown in FIG. 16 may be executed in parallel when the meeting start operation in step S41 of FIG. 14 described above is performed, or the SNS information acquisition process shown in FIG. After the acquisition process ends, the process may proceed to step S42.

<Step S61>
When the participant at the communication terminal 10ac performs an operation to start a conference (the operation at step S41 in FIG. 14 described above), the display control unit 14 of the communication terminal 10ac causes the display 120 (or display 317) to display the image shown in FIG. SNS ID registration screen 1400 shown in FIG.

The SNS ID registration screen 1400 shown in FIG. 17 is used to input the ID of the SNS account used by the conference participant, and to obtain the information of the participant from the SNS server (external server 80 described later) as SNS information. This is the screen. As shown in FIG. 17, the SNS ID registration screen 1400 includes an SNS name input area 1400-1, an SNS ID input area 1400-2, a register button 1400-3, a join button 1400-4, and a cancel button 1400-. 5 and includes.

The SNS name input area 1400-1 is an area for inputting the name of the SNS used by the participant (himself). In the SNS name input area 1400-1, for example, it is possible to select from existing SNSs (for example, Twitter, Facebook (registered trademark), Instagram (registered trademark), LINE (registered trademark), etc.) included in a list displayed by a pulldown. And it is sufficient.

The SNS ID input area 1400-2 is an area for inputting the SNS account ID entered in the SNS name input area 1400-1.

The registration button 1400-3 registers the ID of the SNS input in the SNS ID input area 1400-2 in the SNS name input area 1400-1 in the user management table (see FIG. 7), and This is a button for acquiring the SNS information of the participant from the SNS server (external server 80) using the SNS ID.

The participation button 1400-4 is a button for selecting participation in a conference. The cancel button 1400-5 is a button for canceling participation in a conference. Note that the participant can also participate in the conference without inputting information in the SNS name input area 1400-1 or the SNS ID input area 1400-2. In this case, the SNS information of the participant is not acquired from the SNS server (external server 80), and participant information (described later) is not derived from the SNS information. In this case, for example, the participant information registered in the user management table (see FIG. 7) will be a NULL value.

<Steps S62, S63>
Next, when the participant at the communication terminal 10ac presses the registration button 1400-3 on the SNS ID registration screen 1400, the communication unit 11 of the communication terminal 10ac inputs the name of the SNS entered in the SNS name input area 1400-1. The user ID "B_10ac" and the SNS ID "b" of the participant are sent to the management system 50 along with the SNS ID registration request including the SNS ID registration request. Thereby, the communication unit 51 of the management system 50 receives the SNS ID registration request, the user ID, and the SNS ID.

<Steps S64, S65>
The participant information management unit 59 of the management system 50 associates the user ID received by the communication unit 51 with the SNS ID. Then, the participant information management unit 59 searches the user management table (see FIG. 7) for a record corresponding to the user ID, and registers "b" in the SNS ID of the record.

<Step S66>
Then, the communication unit 51 of the management system 50 sends an SNS information request to the external server 80 specified by the SNS name included in the SNS ID registration request received in step S63, along with an SNS information request for requesting SNS information. Send the SNS ID of the participant. Then, the external server 80 receives the SNS information request and the SNS ID.

<Step S67>
In accordance with the received SNS information request, the external server 80 acquires SNS information corresponding to the SNS ID received with the SNS information request. Here, the SNS information includes, for example, profile information of a user identified by an SNS ID, the number of followers and follower information on Twitter, friend information on Facebook, and icon information used on the SNS.

<Step S68>
The external server 80 transmits the acquired SNS information to the management system 50 via the communication network 2. The communication unit 51 of the management system 50 then receives the SNS information.

<Step S69>
The participant information management unit 59 of the management system 50 receives information such as SNS ID, following ID, number of posts, number of followers, and icon information from the SNS information received by the communication unit 51 as participant information. Derive.

<Step S70>
The participant information management unit 59 of the management system 50 then assigns the derived participant information to the user ID (an example of identification information) of the participant received in step S63 in the user management table (see FIG. 7). Register in association. In the example of the user management table shown in FIG. 7, the participant information associated with the user ID "B_10ac" includes SNS ID "b", following IDs "a, c, d", number of posts "1000", and followers. The number "15000" and the icon information "c://..." are registered.

Note that the participant information management unit 59 does not need to derive all participant information such as SNS ID, following ID, number of posts, number of followers, and icon information from the SNS information received by the communication unit 51. Derivable participant information may be derived within the range of information that can be received as SNS information by the communication unit 51.

Through the above operations, when participating in a conference, SNS information related to the participant is obtained from the SNS server used by the participant, and the SNS information is used to create the inviteable audience list described later. participant information can be derived.

Note that the above-mentioned SNS information acquisition process is executed at the start of the meeting, but is not limited to this. It may be executed later.

Furthermore, in the above-described SNS information acquisition process, the SNS is selected using the SNS name input area 1400-1 of the SNS ID registration screen 1400 shown in FIG. The SNS information may be acquired from a server.

(Meeting screen display processing when participating in a meeting)
FIG. 18 is a sequence diagram illustrating an example of a conference screen display process when participating in a conference in the communication system according to the embodiment. FIG. 19 is a diagram showing an example of a speaker conference screen. FIG. 20 is a diagram showing an example of an audience conference screen. Conference screen display processing when participating in a conference in the communication system 1 according to the present embodiment will be described with reference to FIGS. 18 to 20. It should be noted that the processes of steps S81 to S85 in the communication terminal 10ac in the conference screen display process when participating in a conference shown in FIG. 18 are executed after the session is established in step S53 of FIG. 14 described above, for example. good. Further, if the processes of steps S86 to S90 in the communication terminal 10ad in the conference screen display process when participating in a conference shown in FIG. 18 are executed after the session is established in step S54 of FIG. 14 described above, for example, good.

<Steps S81, S82>
When the conference participation process (for example, the process up to establishing a session in step S53 in FIG. 14) is performed in the communication terminal 10ac, the communication unit 11 of the communication terminal 10ac performs the role assigned to the participant of the communication terminal 10ac. A role information request is sent to the management system 50 together with the user ID "B_10ac" of the participant. Thereby, the communication unit 51 of the management system 50 receives the role information request and the user ID.

<Step S83>
The storage/reading unit 57 of the management system 50 searches the user management table (see FIG. 7) using the user ID “B_10ac” received by the communication unit 51 as a search key, and searches the role associated with the user ID (here ``Moderator'').

<Step S84>
Then, the communication unit 51 of the management system 50 transmits the role information acquired by the storage/reading unit 57 to the communication terminal 10ac. Thereby, the communication unit 11 of the communication terminal 10ac receives the role information.

<Step S85>
Then, when the display control unit 14 of the communication terminal 10ac confirms that the role received by the communication unit 11 is “MC”, that is, the speaker, the display control unit 14 of the communication terminal 10ac displays the podium as shown in FIG. A speaker conference screen 1300S, which is a conference screen for speakers, is displayed.

The speaker conference screen 1300S shown in FIG. 19 includes a speaker video 1300S-1, a self-video 1300S-2, a video changeover switch 1300S-3, an audio changeover switch 1300S-4, a list display button 1300S-5, and an exit button 1300S-6.

The speaker video 1300S-1 is a display area for videos of speakers other than the speaker who are discussing in the panel discussion. In the example of the speaker conference screen 1300S shown in FIG. 19, the speaker video 1300S-1 shows a participant (name: Komai) with the user ID "A_10aa" whose role is assigned to the presenter, and a participant whose role is assigned to the presenter. Images of a participant (name: Shindo) with the user ID "D_10ae" who is assigned the role of presenter and a participant (name: Fukumori) with the user ID "E_10ba" whose role is assigned to the presenter are displayed. The speaker video 1300S-1 displays the speaker's video based on the video data transmitted and received in steps S55 to S58 shown in FIG. 14 described above.

The self-image 1300S-2 is a display area for the user's own image allocated as a speaker. In the example of the speaker conference screen 1300S shown in FIG. 19, an image of the user (name: Miyazawa) with the user ID "B_10ac" whose role is assigned to the moderator is displayed as the self-image 1300S-2.

The video changeover switch 1300S-3 is a switch for switching whether or not to transmit the self-image photographed by the imaging unit 13 to another communication terminal 10. The audio changeover switch 1300S-4 is a switch for switching whether or not to transmit one's own voice inputted by the audio input unit 15 to another communication terminal 10 (muting or not).

The list display button 1300S-5 is used to display a list of inviteable audience members who can be invited to participate as speakers by the process shown in FIG. 22, which will be described later, among the participants who are participating as audience members. button. The display operation of the inviteable audience list will be described in detail in FIG. 22, which will be described later.

The exit button 1300S-6 is a button for exiting the currently held conference.

<Steps S86, S87>
On the other hand, when conference participation processing (for example, processing up to session establishment in step S54 in FIG. 14) is performed in the communication terminal 10ad, the communication section 11 of the communication terminal 10ad is allocated to the participant of the communication terminal 10ad. A role information request for requesting a role is sent to the management system 50 together with the user ID "C_10ad" of the participant. Thereby, the communication unit 51 of the management system 50 receives the role information request and the user ID.

<Step S88>
The storage/reading unit 57 of the management system 50 searches the user management table (see FIG. 7) using the user ID “C_10ad” received by the communication unit 51 as a search key, and searches the role associated with the user ID (here Then, get the "audience").

<Step S89>
Then, the communication unit 51 of the management system 50 transmits the role information acquired by the storage/readout unit 57 to the communication terminal 10ad. Thereby, the communication unit 11 of the communication terminal 10ad receives the role information.

<Step S90>
Then, when the display control unit 14 of the communication terminal 10ad confirms that the role received by the communication unit 11 is “auditor”, the display control unit 14 of the communication terminal 10ad displays a message for the audience as shown in FIG. 20 on the display 120 (or display 317). An audience conference screen 1300A, which is a conference screen, is displayed.

Audience conference screen 1300A shown in FIG. 20 includes speaker video 1300A-1 and exit button 1300A-6.

The speaker video 1300A-1 is a display area for the video of the speaker who will be having a discussion in the panel discussion. In the example of the audience conference screen 1300A shown in FIG. 20, the speaker video 1300A-1 shows a participant (name: Komai) with the user ID "A_10aa" whose role is assigned to the presenter, and a participant whose role is assigned to the presenter. a participant with user ID "D_10ae" (name: Shindo), a participant with user ID "E_10ba" (name: Fukumori) whose role is assigned to presenter, and a user whose role is assigned to moderator. A video of a participant with ID "B_10ac" (name: Miyazawa) is displayed. The speaker video 1300A-1 displays the speaker's video based on the video data transmitted and received in steps S55 to S58 shown in FIG. 14 described above.

The exit button 1300A-6 is a button for exiting the currently held conference.

Through the above operations, the speaker (presenter, moderator) listens to the speech audio of other speakers while checking the video of other speakers and himself/herself, and the speaker (presenter, moderator) listens to the speech audio of other speakers, and shares the speech audio of the speaker with other participants ( (other speakers and audience members). In addition, audience members can listen to the audio of the speaker's discussion while checking the video of the speaker.

(Presence update process)
FIG. 21 is a sequence diagram illustrating an example of presence update processing of the communication system according to the embodiment. With reference to FIG. 21, a flow of presence update processing for the communication terminal 10 of a participant participating in a panel discussion, which is a video conference, to update the presence of the participant at a predetermined timing will be described.

<Steps S101, S102>
The user of the communication terminal 10 logs in (user authentication) according to the steps S21 to S25 in FIG. 12 described above. Then, the status management unit 20 of the communication terminal 10 transmits presence information including the fact that the presence is “online” and the user ID of the user of the communication terminal 10 to the management system 50 via the communication unit 11. The communication unit 51 of the management system 50 then receives the presence information.

<Step S103>
Then, the participant information management unit 59 of the management system 50 updates the presence corresponding to the user ID included in the presence information received by the communication unit 51 to “online” in the user management table (see FIG. 7).

<Steps S104, S105>
The communication terminal 10 performs the processing of steps S41 to S54 in FIG. 14 described above, or the processing of participating in a conference that is already being held. Then, the communication unit 11 of the communication terminal 10 transmits a role acquisition request for acquiring the role of the user (participant) of the communication terminal 10 to the management system 50 via the communication network 2 together with the user ID of the user. do. When the role acquisition request and the user ID are received by the communication unit 51, the storage/reading unit 57 of the management system 50 reads out the role corresponding to the user ID from the user management table. The communication unit 51 then transmits the role read out by the storage/readout unit 57 to the communication terminal 10 via the communication network 2 . Thereby, the communication unit 11 of the communication terminal 10 receives the role of the user. This process is similar to the process shown in FIG. 18 described above. The following description will be made assuming that the received role is "auditor".

<Steps S106, S107>
Furthermore, the communication terminal 10 checks the presence or absence of devices such as microphones, speakers, and cameras (for example, the microphone 114, the speaker 115, and the camera 112, or the microphone 314, the speaker 315, and the CMOS 312), and device information including the check result. get. Then, the status management unit 20 of the communication terminal 10 transmits the presence information including the "attending" presence indicating participation in the conference, the above device information, and the user ID of the user of the communication terminal 10 to the communication terminal 10 in step S104. The information is transmitted to the management system 50 via the unit 11. The communication unit 51 of the management system 50 then receives the presence information. Here, the device information will be explained as information indicating, for example, "no camera".

<Step S108>
Then, the participant information management unit 59 of the management system 50 changes the presence from “online” to “attending (no camera)” in the user management table as the presence corresponding to the user ID included in the presence information received by the communication unit 51. ”.

Note that the process of acquiring device information in step S106 described above is not an essential process and may be skipped.

<Steps S109, S110>
After that, it is assumed that a camera (camera 112 or CMOS 312) is attached to the communication terminal 10 as a device. Then, the status management unit 20 of the communication terminal 10 transmits the presence information including the fact that the device has been changed (that the camera has been attached) and the user ID of the user of the communication terminal 10 to the management system via the communication unit 11. Send to 50. The communication unit 51 of the management system 50 then receives the presence information.

<Step S111>
Then, the participant information management unit 59 of the management system 50 changes the presence from “attending (no camera)” to “attending” in the user management table as the presence corresponding to the user ID included in the presence information received by the communication unit 51. Update to "Medium".

<Steps S112, S113>
After that, the user of the communication terminal 10 presses the exit button 1300A-6 on the audience conference screen 1300A (see FIG. 20) displayed on the display 120 (or display 317) of the communication terminal 10, thereby starting the conference. Leave the room. Then, the status management unit 20 of the communication terminal 10 transmits presence information including the fact that the user has left the conference and the user ID of the user of the communication terminal 10 to the management system 50 via the communication unit 11. The communication unit 51 of the management system 50 then receives the presence information.

<Step S114>
Then, the participant information management unit 59 of the management system 50 updates the presence corresponding to the user ID included in the presence information received by the communication unit 51 from “attending” to “online” in the user management table. .

<Steps S115, S116>
Thereafter, the user of the communication terminal 10 logs out by performing a logout operation on the communication terminal 10. Then, the status management unit 20 of the communication terminal 10 transmits presence information including the fact that the user has logged out and the user ID of the user of the communication terminal 10 to the management system 50 via the communication unit 11. The communication unit 51 of the management system 50 then receives the presence information.

<Step S117>
Then, the participant information management unit 59 of the management system 50 updates the presence corresponding to the user ID included in the presence information received by the communication unit 51 from “online” to “offline” in the user management table.

In steps S101 to S117 above, the timing of updating the presence of the user of the communication terminal 10 has been described, but the timing of updating the presence and the content of the update are not limited to the above-mentioned content.

(Meeting invitation processing)
FIG. 22 is a sequence diagram illustrating an example of a meeting invitation process of the communication system according to the embodiment. FIG. 23 is a diagram illustrating an example of an audience conference screen on which a list of inviteable audience members is displayed. FIG. 24 is a diagram illustrating an example of an audience conference screen on which another list of inviteable audience members is displayed. FIG. 25 is a diagram illustrating an example of a speaker conference screen with a specific audience member selected from the inviteable audience list. FIG. 26 is a diagram illustrating an example of an invitation confirmation dialog box. FIG. 27 is a diagram illustrating an example of an invitation reception dialog box. FIG. 28 is a diagram illustrating an example of a speaker conference screen after an audience member has taken the stage. FIG. 29 is a diagram illustrating an example of a conference screen on the audience side after an audience member takes the stage. Referring to FIGS. 22 to 29, the invitation process for a panel discussion as a video conference in the communication system 1 according to the present embodiment will be described. In addition, in FIG. 22, the role of the participant using the communication terminal 10ac is assumed to be a "moderator" (speaker), and the role of the participant using the communication terminal 10ad is assumed to be an "audience". .

<Steps S121, S122>
First, a participant (speaker, moderator) at the communication terminal 10ac operates the input device 108 shown in FIG. 2 (or the touch panel 318 shown in FIG. 3), and a speaker conference screen is displayed on the display 120 (display 317). Press the list display button 1300S-5 of 1300S. Then, the communication unit 11 of the communication terminal 10ac sends an audience list request to request a list of inviteable audience members who can be invited to participate as speakers among the participants who are participating as audience members. , is transmitted to the management system 50 together with the user ID "B_10ac" of the participant. Furthermore, the audience list request shall include information on search conditions that specify what conditions are used to search for audiences. For example, here, it is assumed that the audience list request includes information on a first condition (an example of a predetermined search condition) that is a search condition for searching for audience members who may be acquaintances of the participant B_10ac. Then, the communication unit 51 of the management system 50 receives the audience list request and the user ID.

<Step S123>
The participant information management unit 59 of the management system 50 refers to the user management table (see FIG. 7) and selects the SNS ID “b” corresponding to the user ID “B_10ac” included in the audience list request received by the communication unit 51. ” is extracted. Then, the participant information management unit 59 extracts, for example, the user ID of the user who follows participant B_10ac on the SNS, according to the first condition included in the audience list request. For example, in the example of the user management table shown in FIG. 7, the user IDs of the users whose following IDs include the SNS ID "b" of participant B_10ac are "A_10aa", "C_10ad", and "E_10ba". , "F_10bb" are extracted.

<Step S124>
Next, the audience extraction unit 60 of the management system 50 refers to the user management table (see FIG. 7), and selects users whose role is “auditor” from among the user IDs extracted by the participant information management unit 59. Identify the ID. For example, in the example of the user management table shown in FIG. 7, among the user IDs extracted by the participant information management unit 59, user IDs "C_10ad" and "F_10bb" whose role is "auditor" are specified. Furthermore, the audience extraction unit 60 extracts user IDs whose presence is "attending" from among the identified user IDs whose role is "auditor." That is, the user ID of the user who is actually participating as an "audience" in a panel discussion that is a video conference is extracted. Here, the audience extraction unit 60 extracts user IDs "C_10ad", "F_10bb", and "H_10da" as user IDs of participants whose role is "auditor" and whose presence is "attending". This will be explained below.

<Step S125>
The audience extraction unit 60 of the management system 50 reads, for example, SNS icon information, name (display name), and SNS ID as information corresponding to the extracted user ID. Then, the communication unit 51 of the management system 50 converts each set of user ID, icon information, name (display name), and SNS ID extracted by the audience extraction unit 60 into audience list information (participant information). example) and transmits it to the communication terminal 10ac. Thereby, the communication unit 11 of the communication terminal 10ac receives the audience list information.

<Step S126>
The display control unit 14 of the communication terminal 10ac causes the display 120 (or display 317) to display a speaker conference screen 1300S as shown in FIG. 23, which includes the content of the audience list information received by the communication unit 11.

In addition to the contents included in the speaker conference screen 1300S shown in FIG. 19 described above, the speaker conference screen 1300S shown in FIG. , an inviteable audience list 1300S-8, a check box 1300S-9, and an invite button 1300S-10. Furthermore, on the speaker conference screen 1300S shown in FIG. 23, since the inviteable audience list 1300S-8 is displayed, the list hide button 1300S-5a is displayed instead of the list display button 1300S-5 shown in FIG. Displayed.

The "Maybe acquaintance" selection button 1300S-7a searches for and lists audience members using the first condition, which is a search condition for searching for audience members who may be acquaintances of the above-mentioned participant (participant B_10ac in this case). This is a button for displaying. Since the speaker conference screen 1300S shown in FIG. 23 displayed in step S126 displays a list of audience members searched based on the first condition, the "may be someone you know" selection button 1300S-7a is in the selected state (pressed state). ).

The "may be famous" selection button 1300S-7b is a button for searching for and displaying a list of audience members using the second condition (an example of a predetermined search condition), which is a search condition for searching for audience members who can be judged to be famous. It is.

The inviteable audience list 1300S-8 is an area that displays a list of audience information included in the audience list information received by the communication unit 11. Specifically, in the example shown in FIG. 23, the user ID, SNS icon, name (display name), and SNS ID extracted by the audience extraction unit 60 are displayed in a list. In this way, it is possible to display the audience members searched for using the first condition, which is the search condition for searching for audience members who may be acquaintances of participant B_10ac. Among them, acquaintances of participant B_10ac can be immediately confirmed, and audience members who are acquaintances can be invited to be speakers as necessary.

The check box 1300S-9 is a check box for selecting an audience member to be invited to participate as a speaker in the panel discussion from among the audience members displayed in the inviteable audience list 1300S-8. Furthermore, check box 1300S-9 allows selection of multiple audience members.

The list hide button 1300S-5a is a button that hides the inviteable audience list 1300S-8 to display the speaker conference screen 1300S shown in FIG. 19 described above.

Here, the operation when the "may be famous" selection button 1300S-7b is pressed from the state of the speaker conference screen 1300S shown in FIG. 23 will be described. When a participant (speaker, moderator) at the communication terminal 10ac operates the input device 108 shown in FIG. 2 (or the touch panel 318 shown in FIG. 3) and presses the "may be famous" selection button 1300S-7b, the communication terminal 10ac The communication unit 11 transmits an audience list request for requesting an inviteable audience list to the management system 50 together with the user ID "B_10ac" of the participant. In this case, the audience list request includes information on the second condition, which is a search condition for searching for audience members who can be determined to be famous. Then, through the same process as steps S122 to S125 described above, a process of searching for an audience member based on the second condition is executed. Specifically, the participant information management unit 59 of the management system 50 refers to the user management table (see FIG. 7) according to the second condition, and determines whether, for example, the number of posts is a predetermined number (for example, 1000) or more, or the number of followers. is a predetermined number (for example, 5000) or more, or the number of posts is a predetermined number (for example, 1000) or more, and the number of followers is a predetermined number (for example, 5000) or more. Then, the audience extractor 60 of the management system 50 extracts a user ID whose role is "auditor" and whose presence is "attending" from among the user IDs extracted by the participant information management section 59. do.

A screen displaying the contents of the audience list information including information on the audience searched under the second condition is a speaker conference screen 1300S shown in FIG. 24. In the example of the speaker conference screen 1300S shown in FIG. 24, the list of audience members searched using the second condition is displayed in the inviteable audience list 1300S-8, so the "may be famous" selection button 1300S-7b is It is in a selected state (pressed state). Furthermore, the example of the speaker conference screen 1300S shown in FIG. 24 shows a case where the user IDs "C_10ad", "J_10ca", and "K_10fa" are extracted as the user IDs of the audience members searched using the second condition. That is, by using the "may be someone you know" selection button 1300S-7a and the "may be famous" selection button 1300S-7b, a list of audience members searched based on the first condition is displayed, or a list of audience members searched based on the second condition is displayed. You can switch between displaying.

Further, the list of audience members displayed in the inviteable audience list 1300S-8 may be sorted and displayed in order of number of posts or number of followers. In other words, the audience may be sorted and displayed in order of degree of fame. In this way, it is possible to display the audience members searched for using the second condition, which is the search condition for searching for audience members who can be judged to be famous, so it is possible to display the audience members who have been searched for using the second condition, which is the search condition for searching for audience members who can be judged to be famous. Alternatively, you can invite an audience member who has an ability to communicate as a speaker, making it possible to have a meaningful panel discussion.

Note that in step S122 described above, it is assumed that the information on the first condition is included in the audience list request, but the present invention is not limited to this, and it is assumed that the information on the second condition is included in the audience list request. Good too.

<Step S127>
Then, the participant at the communication terminal 10ac checks the participant C_10ad as shown in FIG. Assume that box 1300S-9 has been selected. Then, the participant at the communication terminal 10ac presses the invite button 1300S-10 via the input device 108 (or touch panel 318) in order to invite participant C_10ad, who is an audience member, to the discussion.

<Step S128>
Then, the display control unit 14 of the communication terminal 10ac displays an invitation confirmation dialog box 1310S on the display 120 (or the display 317), as shown in FIG. 26, for confirming whether or not to invite the participant C_10ad. Display. The invitation confirmation dialog box 1310S includes a "Yes" button 1310S-1 for making an invitation, and a cancel button 1310S-2 for canceling the invitation. In the following description, it is assumed that the "Yes" button 1310S-1 has been pressed by the participant B_10ac.

<Step S129>
Then, the communication unit 11 of the communication terminal 10ac sends an invitation notification for inviting the discussion to the user ID "B_10ac" of itself (participant B_10ac) and the user ID "C_10ad" of the invitee participant C_10ad. It is transmitted to the management system 50 via the communication network 2. Then, the communication unit 51 of the management system 50 receives the invitation notification, the user ID "B_10ac", and the user ID "C_10ad" of the invitation destination.

<Steps S130, S131>
The storage/reading unit 57 of the management system 50 refers to the user management table (see FIG. 7) and acquires the display name corresponding to the invitation source user ID “B_10ac” received by the communication unit 51. Then, the communication unit 51 of the management system 50 sends the invitation notification for inviting the discussion, and the invitation source information including the display name of the participant B_10ac and the user ID “B_10ac” acquired by the storage/reading unit 57. It is transmitted to the communication terminal 10ad via the communication network 2. Then, the communication unit 11 of the communication terminal 10ad receives the invitation notification and the invitation source information.

<Step S132>
Then, the display control unit 14 of the communication terminal 10ad causes the display 120 (or the display 317) to display an invitation reception dialog box 1310A indicating that an invitation notification as shown in FIG. 27 has been received. The invitation reception dialog box 1310A shown in FIG. 27 indicates that an invitation notification has been received, and whether participant C_10ad agrees to participate as a speaker (presenter) in the panel discussion in which he is participating as an audience member. This is a user interface for operating whether or not. In the invitation reception dialog box 1310A, as shown in FIG. 27, the display name "Miyazawa" included in the invitation source information, for example, is displayed to clearly indicate from whom the invitation is received. By checking the invitation confirmation dialog box 1310A, participant C_10ad (auditor) can confirm that he has received the invitation notification to the discussion from participant B_10ac (inviter). Invitation confirmation dialog box 1310A includes a "Yes" button 1310A-1 for accepting the invitation and a "No" button 1310A-2 for rejecting the invitation. In the following description, it is assumed that the "Yes" button 1310A-1 has been pressed by the participant C_10ad.

<Step S133>
Then, the communication unit 11 of the communication terminal 10ad updates an invitation acceptance notification indicating that the invitation is accepted, the invitee's user ID "C_10ad", the role to "presenter", and the presence to "presenting". Presence information to be updated is transmitted to the management system 50 via the communication network 2. Then, the communication unit 51 of the management system 50 receives the invitation approval notification, the user ID "C_10ad", and the presence information.

<Step S134>
Then, the storage/reading unit 57 of the management system 50 changes the role corresponding to the user ID “C_10ad” received by the communication unit 51 from “audience” to “presenter (limited) in the user management table (see FIG. 7). ) and update the presence from "Attending" to "Presenting." The reason why the role to be updated is "Presenter (with restrictions)" instead of "Presenter" is because there are some restrictions on the display contents of the speaker conference screen displayed in step S137, which will be described later. be.

<Step S135>
Then, the communication unit 51 of the management system 50 transmits a speaker information update command including information on the role of the participant who has become a speaker (presenter), "presenter (with restrictions)" to the communication terminal 10ad. Then, the communication unit 11 of the communication terminal 10ad receives the speaker information update command.

<Step S136>
Further, the communication unit 51 of the management system 50 transmits a speaker information update command including information indicating that the invitation has been accepted and the user ID "C_10ad" of the invited participant to the communication terminal 10ac. Then, the communication unit 11 of the communication terminal 10ac receives the speaker information update command.

<Step S137>
In accordance with the speaker information update command received by the communication unit 11, the display control unit 14 of the communication terminal 10ad changes the speaker conference screen 1300A displayed on the display 120 (or display 317) to the speaker as shown in FIG. The display switches to the participant conference screen 1500S.

A speaker conference screen 1500S shown in FIG. 29 is a conference screen for speakers that is displayed when the speaker accepts an invitation to speak on the podium in a panel discussion in which he or she has participated as an audience member. As shown in FIG. 29, the speaker conference screen 1500S includes a speaker video 1500S-1, a self-video 1500S-2, a video changeover switch 1500S-3, an audio changeover switch 1500S-4, and an exit button 1500S-6. and, including.

The speaker video 1500S-1 is a display area for the video of the speaker who will be having a discussion in the panel discussion. In the example of the speaker conference screen 1500S shown in FIG. 29, the speaker video 1500S-1 shows a participant (name: Komai) with the user ID "A_10aa" whose role is assigned to the presenter, and a participant whose role is assigned to the presenter. a participant with user ID "D_10ae" (name: Shindo), a participant with user ID "E_10ba" (name: Fukumori) whose role is assigned to presenter, and a user whose role is assigned to moderator. A video of a participant with ID "B_10ac" (name: Miyazawa) is displayed.

Self-image 1500S-2 is an area where participant C_10ad displays his or her own image in accordance with the updated role of "presenter (with restrictions)" included in the speaker information update command.

The video changeover switch 1500S-3 is a switch for switching whether or not to transmit the self-image photographed by the imaging unit 13 to another communication terminal 10. The audio changeover switch 1500S-4 is a switch for switching whether or not to transmit one's own voice inputted by the audio input unit 15 to another communication terminal 10 (muting or not).

The exit button 1500S-6 is a button for exiting the currently held conference.

Furthermore, in the speaker conference screen 1500S shown in FIG. 29, the speaker conference screen 1500S shown in FIG. A button corresponding to list display button 1300S-5 included in 1300S is not included. As a result, participants who have changed from audience members to presenters (speakers) will be shown a list corresponding to the inviteable audience list 1300S-8, and will not be able to invite new audience members as speakers. It has become. As a result, the authority to invite audience members as speakers will be granted only to participants who have been participating as speakers since the panel discussion was held, and will prevent an excessive increase in the number of speakers due to a chain of invitations. Can be suppressed. Note that if there is no risk of confusion in the discussion, such as when the number of participants in the panel discussion is small, the authority to invite new speakers may also be granted.

<Step S138>
The display control unit 14 of the communication terminal 10ac causes the display 120 (or display 317) to display a speaker conference screen 1300S as shown in FIG. 28 in accordance with the speaker information update command received by the communication unit 11. In the speaker conference screen 1300S shown in FIG. 28, since participant C_10ad has become a new speaker, a video of participant C_10ad (name: Suzuki) is added as speaker video 1300S-1a.

<Steps S139, S140>
Since the participant C_10ad using the communication terminal 10ad has become a speaker (presenter), the audio input unit 15 of the communication terminal 10ad starts inputting the voice of the participant C_10ad, and the imaging unit 13 The communication unit 11 starts capturing the video of C_10ad, and starts transmitting the audio data and video data of the audio to the management system 50 via the communication network 2. Thereby, the audio data and video data are distributed to the communication terminals 10 participating in the video conference other than the communication terminal 10ad. This allows participant C_10ad to participate in the panel discussion from the standpoint of an audience member to a speaker (presenter).

Although the first condition and the second condition have been described as conditions for searching for audience members to be displayed on the inviteable audience list 1300S-8 of the speaker conference screen 1300S, the present invention is not limited to these conditions. For example, audience members who are interested in a specific topic estimated from a specific hashtag may be searched for and displayed on the inviteable audience list 1300S-8.

As described above, the communication system 1 according to the present embodiment uses the SNS account ID of the user who participated in the video conference as a speaker to acquire and register information about the user from the SNS server, A list of participants searched for registered information under predetermined conditions is displayed on the conference screen, and participants selected from the list can be invited as speakers for the video conference. As a result, even if there are a large number of audience members, it is possible to easily select a specific audience member to take the stage without registering in advance a list of audience members to take the stage in a video conference.

Furthermore, in the communication system 1 according to the present embodiment, the information registered as information regarding the user from the SNS server is used to search for audience members who may be acquaintances of the participant who is participating as a speaker. It is possible to display the audience searched based on the first condition, which is the search condition. As a result, it is possible to immediately check who the participant is acquainted with among the audience members participating in the panel discussion, and it is possible to invite audience members who are acquainted to the speaker as necessary.

In addition, in the communication system 1 according to the present embodiment, the information registered as information regarding the user from the SNS server is used to perform a search using the second condition, which is a search condition for searching for an audience member who can be judged to be famous. audience can be displayed. As a result, among the audience members participating in the panel discussion, those who have influence or communication power can be invited as speakers, and a meaningful panel discussion can be realized.

(Another form of software configuration of communication terminal)
FIG. 30 is a diagram illustrating an example of a software configuration when using a web application of a communication terminal according to an embodiment. Another form of the software configuration of the communication terminal 10 will be described with reference to FIG. 30.

Although the above-described FIG. 5 describes the operation of the communication application A executed by the communication terminal 10, similar processing can also be realized by a Web application. A Web application operates by a program based on, for example, JavaScript (registered trademark) that runs on a browser, and a program on the Web server side, and is used by a user on the browser. That is, as shown in FIG. 30, the communication terminal 10 downloads a program WA (HTML (HyperText Markup Language) + JavaScript + CSS (Cascading Style Sheets), etc.) from the management system 50 and executes it on the browser 1040. The browser 1040 operates under the control of the OS 1020.

The communication terminal 10 can utilize the services provided by the management system 50 by exchanging data with the management system 50 using a protocol such as HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol Secure). In such a usage pattern, there is no need to download the communication application A to the communication terminal 10 in advance.

Note that each function of the embodiments described above can be realized by one or more processing circuits. Here, the term "processing circuit" refers to a processor that is programmed to execute each function by software, such as a processor implemented by an electronic circuit, an ASIC designed to execute each of the above-mentioned functions, and a DSP (Digital Signal It includes devices such as processors, FPGAs, SoCs (Systems on a Chip), GPUs (Graphics Processing Units), and conventional circuit modules.

Further, in the above-described embodiment, when at least one of the functional units of the communication terminal 10 and the management system 50 is realized by executing a program, the program is provided by being incorporated in a ROM or the like in advance. Furthermore, the programs executed by the communication terminal 10 and the management system 50 according to the above-described embodiments can be stored as files in an installable or executable format on a CD-ROM (Compact Disc Read Only Memory) or a flexible disk (FD). , a CD-R (Compact Disk-Recordable), a DVD, or an SD card. Further, the program executed by the communication terminal 10 and the management system 50 according to the above-described embodiment is stored on a computer connected to a network such as the Internet, and is provided by being downloaded via the network. Good too. Furthermore, the programs executed by the communication terminal 10 and the management system 50 according to the above-described embodiments may be configured to be provided or distributed via a network such as the Internet. Further, the programs executed by the communication terminal 10 and the management system 50 according to the above-described embodiment have a module configuration including at least one of the above-mentioned functional units, and the actual hardware includes the CPU described above. By reading a program from the storage device and executing it, each of the above-mentioned functional units is loaded onto the main storage device and generated.

1 Communication system 2 Communication network 2a to 2d LAN
2ab, 2cd Leased line 2e, 2f Base station 2i Internet 10, 10aa, 10ab, 10ac, 10ad, 10ba, 10bb, 10ca, 10cb, 10da, 10db, 10ea, 10eb Communication terminal 11 Communication unit 12 Operation input reception unit 13 Imaging unit 14 Display control unit 15 Audio input unit 16 Audio output unit 17 Storage/reading unit 18 Storage unit 19 Authentication request unit 20 Status management unit 50 Management system 51 Communication unit 52 Authentication unit 53 Status management unit 54 Extraction unit 55 Status acquisition unit 56 Session Control unit 57 Storage/readout unit 58 Storage unit 59 Participant information management unit 60 Audience extraction unit 70, 70a to 70d, 70ab, 70cd Router 80 External server 90 Program provision system 101 CPU
102 ROM
103 RAM
105 Auxiliary storage device 106 Media 107 Media drive 108 Input device 110 Bus line 111 Network I/F
112 Camera 112c Cable 113 Image sensor I/F
114 Microphone 114c cable 115 Speaker 115c cable 116 Audio input/output I/F
119 Display I/F
120, 120aa, 120bb, 120ca, 120db Display 120c Cable 201 CPU
202 ROM
203 RAM
205 Auxiliary storage device 206 Recording media 207 Media drive 208 Display 209 Network I/F
210 Bus line 211 Keyboard 212 Mouse 213 DVD
214 DVD drive 301 CPU
302 ROM
303 RAM
304 EEPROM
306 Media 307 Media I/F
308 GPS receiving section 310 Bus line 311 Long distance communication circuit 311a Antenna 312 CMOS
313 Image sensor I/F
314 Microphone 315 Speaker 316 Sound input/output I/F
317 Display 318 Touch panel 1010 Work area 1020 OS
1040 Browser 1100 Destination list screen 1100-1 User ID display area 1100-2 Terminal ID display area 1100-3 Terminal name display area 1100-4a Offline icon 1100-4b Call ready icon 1100-4c Calling icon 1200-1 Start request dialog Box 1200-2 "Yes" button 1200-3 "No" button 1300A Audience conference screen 1300A-1 Speaker video 1300A-6 Exit button 1300S Speaker conference screen 1300S-1 Speaker video 1300S-1a Speaker video 1300S- 2 Self-image 1300S-3 Video changeover switch 1300S-4 Audio changeover switch 1300S-5 List display button 1300S-5a List hide button 1300S-6 Exit button 1300S-7a "Maybe you know" selection button 1300S-7b "Maybe famous" selection Button 1300S-8 Invitable audience list 1300S-9 Check box 1300S-10 Invitation button 1310A Invitation reception dialog box 1310A-1 "Yes" button 1310A-2 "No" button 1310S Invitation confirmation dialog box 1310S-1 "Yes" button 1310S-2 Cancel button 1400 SNS ID registration screen 1400-1 SNS name input area 1400-2 SNS ID input area 1400-3 Register button 1400-4 Participation button 1400-5 Cancel button 1500S Speaker conference screen 1500S-1 Speaker video 1500S-2 Self-image 1500S-3 Video changeover switch 1500S-4 Audio changeover switch 1500S-6 Exit button 5001 User management DB
5002 Terminal management DB
5003 Group management DB
5004 Session management DB
A Communication app WA program

JP2017-208716A

Claims (12)

A computer, which is a communication terminal that can make video conference calls with other communication terminals by transmitting and receiving audio signals via a network,
Participant information derived from SNS information regarding the user using the communication terminal acquired from a SNS (Social Networking Service) server and identification information of the user are extracted in a registered management system in association with each other. a first receiving unit that receives participant information corresponding to the participant information that satisfies predetermined search conditions;
a display control unit that makes a list of the participant information received by the first reception unit and displays it on a display device;
Among the participants' information included in the list, the participant uses an invitation notification for inviting the participant corresponding to the information selected by the user to participate as a speaker in the video conference. a first transmitter that transmits to another communication terminal;
A program to make it work. 2. The program according to claim 1, wherein the participant information that is extracted in the management system and that satisfies the predetermined search condition is information on an audience participating in the video conference. The first receiving unit receives the SNS information from the management system that has acquired the SNS information from the server of the SNS identified by the identifying information received by the receiving unit that accepts operation input of information identifying the SNS. 3. The program according to claim 1 or 2, wherein the program receives information of. The first receiving unit receives, from the management system, participant information corresponding to the participant information that satisfies a first condition for searching for participants who may be acquaintances of the user as the predetermined search condition. 4. The program according to claim 1, wherein the program receives: The first receiving unit receives, from the management system, participant information corresponding to the participant information that satisfies a second condition for searching for participants who can be determined to be famous as the predetermined search condition. The program described in any one of 1 to 3. The display control unit displays participant information corresponding to the participant information that satisfies a first condition for searching for participants who may be acquaintances of the user as the predetermined search condition, according to an operation on the reception unit. on the list, or display on the list the information of the participant corresponding to the participant information that satisfies the second condition for searching for participants who can be determined to be famous as the predetermined search condition. The program according to any one of claims 1 to 3, which enables the program. 6. The display control unit is configured to sort information on the participants corresponding to the participant information extracted under the second condition in the management system in order of degree of fame and display the sorted information on the list. The program described in 6. further functioning as a second receiving unit that receives the invitation notification transmitted from another communication terminal,
8. The display control unit according to claim 1, wherein the display control unit does not display the list on the display device if the user accepts the invitation notification received by the second reception unit. program. a second receiving unit that receives the invitation notification sent from another communication terminal;
If the user consents to the invitation notification received by the second receiving unit, to distribute the video of the user captured by the imaging unit and the audio of the user input by the audio input unit. a second transmitter that starts transmitting to the management system;
The program according to any one of claims 1 to 7, further functioning as a program. A communication method for a communication terminal capable of making a video conference call with another communication terminal by transmitting and receiving audio signals via a network,
Participant information derived from SNS information regarding the user using the communication terminal acquired from a SNS (Social Networking Service) server and identification information of the user are extracted in a registered management system in association with each other. a receiving step of receiving participant information corresponding to the participant information satisfying predetermined search conditions;
a display control step of making a list of the received participant information and displaying it on a display device;
Among the participants' information included in the list, the participant uses an invitation notification for inviting the participant corresponding to the information selected by the user to participate as a speaker in the video conference. a sending step of sending the message to another communication terminal;
A communication method having A communication terminal capable of making a video conference call with another communication terminal by transmitting and receiving audio signals via a network,
Participant information derived from SNS information regarding the user using the communication terminal acquired from a SNS (Social Networking Service) server and identification information of the user are extracted in a registered management system in association with each other. a first receiving unit that receives participant information corresponding to the participant information that satisfies predetermined search conditions;
a display control unit that makes a list of the participant information received by the first reception unit and displays it on a display device;
Among the participants' information included in the list, the participant uses an invitation notification for inviting the participant corresponding to the information selected by the user to participate as a speaker in the video conference. a first transmitter that transmits to another communication terminal;
A communication terminal equipped with A communication system comprising: a communication terminal capable of making a videoconference call by transmitting and receiving audio signals to and from another communication terminal via a network; and a management system that manages the videoconference.
The management system includes:
an acquisition unit that acquires SNS information regarding a user who uses the communication terminal from an SNS (Social Networking Service) server;
a registration unit that associates and registers the user's identification information and participant information derived from the SNS information acquired by the acquisition unit in a storage unit;
an extraction unit that extracts the identification information of a participant corresponding to the participant information that satisfies a predetermined search condition from the storage unit;
a third transmitting unit that transmits information on the participant indicated by the identification information extracted by the extracting unit to the communication terminal;
Equipped with
The communication terminal is
a display control unit that makes a list of the participant information transmitted by the third transmission unit and displays it on a display device;
Among the participants' information included in the list, the participant uses an invitation notification for inviting the participant corresponding to the information selected by the user to participate as a speaker in the video conference. a first transmitter that transmits to another communication terminal;
communication system with.

Images