JP7138217B2 - TERMINAL DEVICE, COMMUNICATION SYSTEM, CONTROL METHOD AND PROGRAM THEREOF - Google Patents

Description

The present invention relates to a terminal device, a communication system, a control method thereof, and a program.

Electronic conferencing systems and services such as web conferencing are generally provided. A terminal that uses an electronic conference system uses a dedicated camera, speaker, and microphone that are directly connected to the terminal to transmit information input from each device to other devices, and output the received information to hold a conference. It can be carried out.

For example, Japanese Patent Application Laid-Open No. 2002-200003 provides a technology for holding a video conference by connecting a terminal in which a camera, a monitor, and a speaker are integrated, and a camera device via a multi-point video conference device.

JP 2013-26843 A JP 2016-122937 A

In recent years, PCs and smartphones with built-in microphones, speakers, and cameras have been distributed to employees for use. Therefore, each user can hold an electronic conference using a device such as a PC, a smartphone, or the like, which is installed in a terminal that the user already has.

However, there may be users who cannot use their own PCs for some reason. In other words, one user does not always have one terminal when there are multiple users in one location such as a conference room. When a user holding a terminal puts the terminal in front of himself/herself, it is difficult for users who are far away and do not hold the terminal to participate in the conference. For example, you have to speak loudly to input your voice into the microphone.

In addition, in order to prevent howling caused by using microphones and speakers of multiple terminals in the same room, and to prevent band pressure due to transmission and reception of images, it is sometimes intentionally not to use everyone's terminals.

For example, an electronic conference may be held by placing a terminal of a certain user in the center of a conference room as a substitute for a microphone and a speaker so that each user can easily input voice/hear the sound of the speaker. Also, when used as a camera, the terminal may be placed in a corner of the conference room to photograph the entire conference room so as to capture as many users as possible.

However, unlike a dedicated camera, microphone, speaker, etc., the terminal is counted as one of the users participating in the conference, so it is displayed to the other party of the electronic conference as one of the participants ( For example, Patent Document 2).

Also, when using it as a microphone or speaker, it is necessary to adjust the volume, and when using it as a camera, it is necessary to adjust the angle of view and zoom in / zoom out. , it is troublesome because it is necessary to directly touch the terminal to operate it.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a mechanism that enables the functions of terminal devices participating in a communication system to be controlled from other terminals associated with the same user .

The present invention provides a terminal device associated with a first user, comprising communication means for communicating with a communication system to which a plurality of information processing terminals are communicably connected, and connection with the communication system by the communication means. and when another terminal associated with the first user is connected to the communication system, a specific function of the terminal device based on an operation performed on the other terminal and a control means for controlling the

ADVANTAGE OF THE INVENTION According to this invention, the mechanism which can control the function of the terminal device which participates in a communication system from the other terminal associated with the same user can be provided.

It is a figure showing an example of composition of an information processing system in an embodiment of the present invention. It is a figure which shows an example of the hardware constitutions of various apparatuses in embodiment of this invention. It is a figure showing an example of functional composition of various devices in an embodiment of the present invention. 10 is a flow chart showing the flow of conventional log-in and room entry processing. 4 is a flow chart showing the flow of room entry processing in the embodiment of the present invention. 4 is a flow chart showing the flow of transmission/reception processing to/from a server of images and sounds in the embodiment of the present invention. 4 is a flow chart showing the flow of processing for acquiring an image and sound from a server in the embodiment of the present invention; It is a figure which shows an example of various data structures in embodiment of this invention. It is a figure which shows an example of a structure, such as a confirmation screen, in embodiment of this invention. 4 is a diagram showing an example of the configuration of a function selection screen in the embodiment of the invention; FIG. It is a figure which shows an example of a structure of the conference screen in embodiment of this invention. 4 is a diagram showing an example of the configuration of a function operation screen according to the embodiment of the invention; FIG. It is a figure which shows an example of a structure of the confirmation screen in embodiment of this invention. FIG. 11 is a diagram showing an example of the configuration of a conventional conference screen;

An example of an embodiment of the present invention will be described below with reference to the drawings.

First, with reference to FIG. 1, an example of the configuration of an information processing system according to an embodiment of the present invention will be described.

The information processing system of the present invention is configured by connecting a plurality of information processing apparatuses (a plurality of client terminals 100 and a server 200) through a network 101 so as to be able to communicate with each other.

The client terminal 100 (a generic term for the client terminals 100A to 100D) is a display terminal for receiving input of user information and allowing users to participate in an electronic conference, and includes a microphone, a speaker, and a camera.

A terminal that inputs voice with a microphone or captures images with a camera is called an input terminal, a terminal that plays voice with a speaker or plays back and outputs images taken with a camera is called an output terminal, and a terminal that performs both is called an input terminal. called a terminal.

The server 200 has a function of storing user information, camera images, and audio information received from the client terminal 100, and distributing them to necessary client terminals to hold an electronic conference.

In the description of the present invention, User001, User002, User00X, and User00Y are present in the actual conference room A, and User003 and User004 are present in the actual conference room B.

Further, each terminal of the client terminals 100A to 100D can hold an electronic conference by entering room001, which is a virtual conference room, on the server 200. FIG. The above is the description of FIG.

Next, with reference to FIG. 2, an example of hardware configuration of various devices in the embodiment of the present invention will be described.

In FIG. 2, reference numeral 201 denotes a CPU that controls devices and controllers connected to a system bus 204 in an integrated manner. The ROM 202 or the external memory 211 also stores a BIOS (Basic Input/Output System), which is a control program for the CPU 201, an operating system program (hereinafter referred to as OS), and functions necessary for realizing the functions executed by each server or each PC. Various programs, etc., which will be described later, are stored.

A RAM 203 functions as a main memory, a work area, and the like for the CPU 201 . The CPU 201 loads programs and the like necessary for executing processing from the ROM 202 or the external memory 211 to the RAM 203, and executes the loaded programs to realize various operations.

An input controller 205 controls input from an input device 209 such as a keyboard (KB), a pointing device such as a mouse (not shown), a touch panel, or a camera.

A video controller 206 controls display on a display device (external output device) such as a display device 210 (eg, CRT display). Note that the display device shown in the display device 210 is not limited to a CRT display, and may be a liquid crystal display, a multi-touch screen, or another display device. Alternatively, an external output device such as a projector may be used.

The video controller 206 can control a video memory (VRAM) for performing display control, and can use part of the RAM 203 as a video memory area, or provide a separate dedicated video memory. It is possible. In this embodiment, each client terminal 100 has a first video memory area used for display when the client terminal 100 is normally used, and a display of the first video memory area when a predetermined screen is displayed. It has a second video memory area which is used for another display overlaid on the content. The number of video memory areas is not limited to two, but may be as many as the resources of the information processing apparatus allow.

A voice input/output controller 212 controls input/output from the microphone/speaker 213 (102 and 103 in FIG. 1). It is possible to recognize the voice input from the microphone. It is also possible to connect earphones to the speaker.

A memory controller 207 is connected via an adapter to a hard disk (HD), a flexible disk (FD), or a PCMCIA card slot for storing boot programs, various applications, font data, user files, edit files, and various data. It controls access to an external memory 211 such as a compact flash (registered trademark) memory that is

A communication I/F controller 208 connects and communicates with an external device via a network (for example, the network 150 (eg, LAN, etc.) shown in FIG. 1), and executes communication control processing in the network. . For example, communication using TCP/IP, communication using telephone lines such as ISDN, and 3G lines for mobile phones are possible.

It should be noted that the CPU 201 enables display on the display device 210 by, for example, rasterizing outline fonts to a display information area in the RAM 203 . The CPU 201 also allows the user to issue instructions using a mouse cursor (not shown) displayed on the display device 210 .

Further, the storage device (storage medium) for permanently storing information such as the external memory 211 is not limited to a storage device such as a hard disk. For example, a medium such as an SSD (Solid State Drive) may be used.

Various programs described later for realizing the present invention are recorded in the external memory 211 and are executed by the CPU 201 by being loaded into the RAM 203 as necessary. Furthermore, definition files and various information tables used when executing the above programs are also stored in the external memory 211, and detailed descriptions of these will be given later. The above is the description of FIG.

Next, with reference to FIG. 3, an example of functional configurations of various devices in the embodiment of the present invention will be described.

The user information input reception unit 311 receives input of user information by user operation. User information is, for example, a user ID and password. An example of user information is shown in user information 800 in FIG.

In the description of the present embodiment, the user ID and password are used as user information, but only the user ID may be accepted as user information as long as the user can be identified.

The login request unit 312 makes a request (server connection request). The request reception unit 321 of the server 200 receives the login request.

The logged-in user determination unit 322 determines whether the user requested to log in has already logged in.

In the case of a user who has already logged in, the parent device determination unit 323 determines the client terminal that the logged-in user has already used for login as the parent device of the client terminal that has newly transmitted the login request. is determined to be the client terminal that has sent the

In other words, it determines which terminal is to be the master of the client terminal that has newly transmitted the login request.

The slave device participation control unit 324 causes the client terminal that has newly transmitted the login request to participate in the electronic conference in which the user of the master device is participating as a slave device of the terminal determined by the master device determination unit 323 .

The function specifying unit 325 specifies a function to be executed (operated) in the child device. The operation control unit 326 causes the child device to execute the function specified by the function specifying unit 325, and the server 200 controls the distribution of input/output information (transmission information) according to the function executed by the child device.

The input/output information (transmission information) referred to here is, for example, images and sounds, and is information input/output by functions such as a camera, a microphone, and a speaker, and transmitted between users.

The function setting unit 313 sets a function to be operated/executed by the self device according to an instruction from the operation control unit 326 and causes the self device to operate. The above is the description of FIG.

Next, with reference to FIG. 4, the flow of conventional login and room entry processing will be described. In the description of FIG. 4, the client terminal 100A is the terminal that newly logs in/enters the conference room, and the client terminal 100C is the terminal of the user who has already entered the conference room.

In order to enjoy the web conference service provided by the server 200, the CPU 201 of the client terminal 100A accesses a predetermined URL according to the user's operation on the browser software, and displays a login screen as indicated by 900 in FIG. .

Then, by accepting the input of the user ID and password on the login screen and accepting the pressing of the "login" button, a login request to the server 200 is transmitted (step S401).

The CPU 201 of the server 200 accepts the login request (step S402), compares it with user information (user ID and password/illustrated at 800 in FIG. 8) stored in advance in the external memory of its own device, and performs authentication processing (login processing). )I do.

If the authentication succeeds (if the login to the server 200 using the received user ID succeeds), the screen information (for example, html information) of the menu screen as shown in 910 in FIG. 9 is generated, and the client terminal 100A (step S403).

In the conventional processing, at the time of login processing, it is determined whether or not the user with the user ID is logging in from another terminal by referring to a login user list (not shown) stored in the server 200. does not include the user of the newly received login request, the login process is executed as a non-login user. That is, the user ID of the user is added to the logged-in user list and stored. After that, the process proceeds to step S403.

If the login user list contains a newly received login request user, that is, if the same user is already logged in on another terminal, forcibly log out the user who is logged in on another terminal. . Then, the user is logged in again, and the process of step S403 is executed for the terminal that has newly transmitted the request.

The CPU 201 of the client terminal 100A receives the screen information and displays the menu screen 910 on the display screen based on the received screen information (step S404).

The CPU 201 of the client terminal 100A receives an operation for selecting a room to be entered by the logged-in user in the room (terminal ID of the room) selection section that is a virtual conference room included in the menu screen 910 (step S405), and By accepting the pressing of the "enter room" button included in , the server 200 is sent to the server 200 a request to enter the room being selected by the user whose user ID has been entered by the device itself (step S406).

In response to the room entry request, the CPU 201 of the client terminal 100A receives the user ID of the logged-in user of the client terminal 100A that has accepted the room entry operation, the terminal ID of the room to be entered (room ID), the terminal ID of the client terminal 100A ( For example, the IP address) and the terminal type information of the client terminal 100A are included in the request to the server 200 .

The terminal type is, for example, information for identifying whether the client terminal 100A itself is a PC (desktop PC, notebook PC) or a mobile terminal (eg, tablet PC, mobile phone, etc.). If the own device is a PC, the value of terminal type=pc is transmitted, and if the own device is a portable terminal, the value of terminal type=mobile is transmitted.

Each client terminal 100 preliminarily holds and stores terminal type information of its own device in a storage area such as an external memory of its own device, acquires the information, and embeds it in the request.

Here, the server 200 acquires the room ID of the requested room and the requested room by including the room selected by the client terminal 100A in the room entry request. When the operation is accepted, the ID of the room is sent to the server 200 as the ID of the room being selected, and the server 200 accepts this. 100A may request the logged-in user to enter the room, and the server 200 may accept this request.

Upon receiving the room entry request (step S407), the CPU 201 of the server 200 executes room entry processing for allowing the user to enter the requested room (conference room) (step S408).

Here, in the room entry process, which user is entering which conference in which conference room, with which terminal of what terminal type, and with what communication method, is registered/stored.

Specifically, the room ID of the room entering the conference room 811 of the conference information 810 in FIG. 8 is added to the user ID of the user who enters the participant ID 813 .

In the description of this embodiment, it is assumed that entering a conference room=participating in a conference corresponding to the conference room.

If there is no user entering the room (if an unattended room is entered), a conference ID for identifying the conference is issued, inserted into the conference ID 812, and stored. For the second and subsequent users, the conference ID of the conference in which the user ID (participant) who is entering the same room (the conference ID corresponding to the room being entered) is stored in the conference ID 812 .

Also, the terminal ID of the client terminal 100A included in the room entry request accepted in step S407 is inserted into the terminal ID 814. FIG. In addition, the received terminal type information is stored in the terminal type 815 .

The room entry notification 816 is information indicating which user's terminal in the same conference room is to be notified that the user in the corresponding row has entered the conference room. and never use that value.

When the room entry process is completed, the CPU 201 of the server 200 enters the screen information (for example, html information) of the conference screen displaying the conference image captured by the camera of each client terminal 100 as shown in 1400 in FIG. The user's user ID is transmitted to the client terminal 100A, which is the transmission source (step S409).

A conference image is a camera image captured by a camera that is transmitted and received in an electronic conference, and is an image that mainly captures the appearance of a user.

The CPU 201 of the client terminal 100A receives the screen information of the conference screen (step S410), and displays the conference screen 1400 based on the received screen information (step S411).

At this point, as shown in the conference screen 1400, no image is displayed in the conference image display area.

The CPU 201 of the server 200 transmits the screen information of the conference screen to the client terminal 100A along with transmitting the screen information of the conference screen, and transmits the list of users participating in the conference of the room in which the user has entered (step S409). It receives (step S410), assigns which user's conference image is to be displayed in which display area, and stores it in the memory (step S412).

Further, when the camera function is set to ON, the CPU 201 of the server 200 captures an image of each frame of a moving image captured by a camera functioning on the own device (for example, a camera mounted on the own device). (frame image) is stored in the image queue on the memory. An example of an image queue on the client terminal is shown at 820 in FIG.

Also, the frame images are sequentially transmitted to the server 200 . The server 200 receives the frame images of the moving image and stores them in the image queue on the memory. An example image queue on the server is shown at 830 in FIG.

In addition, when the microphone function is set to ON, the CPU 201 of the server 200 receives voice information input to a functioning microphone (for example, a microphone installed in the own device) at predetermined intervals. is stored in the voice cue on the memory as data of The configuration of the audio queue on the client terminal is such that audio information is stored instead of the image at 820 in FIG. 8, for example.

Also, the voice information is sequentially transmitted to the server 200 . The server 200 receives the audio information and stores it in an audio queue on memory. The configuration of the audio queue on the server is such that audio information is stored instead of images at 830 in FIG. 8, for example.

In addition, the CPU 201 of the server 200 notifies the client terminal 100C, which is the client terminal of another user who has entered the same conference room, of the user (eg, User001) who has newly entered the conference room from the client terminal 100A ( step S413).

For example, data 811 to 815 of User001 in the conference information 810 are transmitted to the client terminal 100C.

In the description of FIG. 4, it is assumed that the user (User003) of the client terminal 100C participates in the same conference (enters the same conference room) as the user (User001) of the client terminal 100A. , as shown in 810 in FIG. 8, if there are users other than User003 who are participating in the same conference as User001, those users who are participating in the same conference room also newly enter the conference room using the client terminal 100A. Notify the user (eg User001).

The CPU 201 of the client terminal 100C receives the information of the user who has newly entered the conference room (step S414), and stores it in its own memory (step S415). For example, with the same configuration as the conference information 810, it stores and manages a list of users participating in the conference.

In practice, every time a new user enters or leaves the conference room, a notification is received from the server 200 and the list of users participating in the conference is updated. The above is the description of FIG.

The CPU 201 of each client terminal 100 requests the server 200 for the conference images and voices of the users of the other devices based on the list of users of the other devices participating in the same conference notified by the server. By transmitting images and voices in response to the request, the display of the conference screen is updated as shown in 1410 in FIG. 14 and the voices are reproduced.

Next, with reference to FIG. 5, the flow of room entry processing according to the embodiment of the present invention will be described.

In the description of this embodiment, the CPU 201 of the server 200 does not forcibly log out the user even if the same user has already logged in in step S402 of FIG. Move to S403.

The CPU 201 of the server 200 starts the process of FIG. 5 upon receiving the room entry request in step S407 of FIG.

The CPU 201 of the server 200 identifies the user ID included in the room entry request received in step S407 (step S501), and refers to the conference information 810 to determine whether the same user is entering the requested conference room. Determine (step S502).

If the same user is not entering the room, the process proceeds to step S408 in FIG. If the same user is entering the room, the process proceeds to step S503.

The CPU 201 of the server 200 generates screen information (html information) for a confirmation screen as indicated by 920 in FIG. 9, and transmits it to the client terminal 100B, which is the source of the entry request (step S503).

The confirmation screen 920 is a screen for confirming how to allow the user to enter the room based on the new room entry request when a new room entry request from a user who has already logged in and entered the conference room is received. In addition, for confirming how the client terminal that sent the new room entry request will function in the electronic conference in the conference room after entering the room (what function will be executed as a terminal to participate in the conference) is the screen.

A check box 922 is a selection acceptance section for allowing the client terminal 100B displaying the confirmation screen 920 to enter the conference room as a slave.

When entering the room as a child device, the parent device selection reception unit 923 is a selection reception unit that receives a selection of which terminal child device to use as a child device. This is a list display unit that logs in to the server 200 with the same user ID as the user ID requested to enter the room, and selectably displays client terminals that are entering the conference room requested to enter the room with the same user ID.

In the description of this embodiment, it is assumed that User001 has already logged in to the server 200 and is currently entering room 001 on the client terminal 100A, and that a new login request for User001 and a request to enter room 001 have been received from the client terminal 100B. do.

Therefore, the client terminal 100A is displayed in the parent device selection accepting portion 923 so as to be selectable.

If there are a plurality of client terminals that have already entered the room with the same ID, screen information is created so that a list of the plurality of client terminals is displayed on the base unit selection reception unit 923 .

A check box 921 is a selection acceptance section for entering the conference room as a general machine. A general machine is a terminal that is neither a parent machine nor a child machine. When entering the room as a general machine, the entry and login of the user from the client terminal 100A who has already entered the room with the same user ID are forcibly canceled, and the user of the client terminal 100B is allowed to newly enter the conference room.

The client terminal 100B receives the screen information of the confirmation screen 920 and displays it on the display screen (step S504). Then, user operation is accepted on the confirmation screen 920 (step S505).

For example, it is assumed that an operation of requesting entry as a general machine is accepted by accepting pressing of an "OK" button in a state where selection of "enter the room as a general machine" is accepted.

For example, in a state in which the selection of "enter the room as a child device" is accepted, the selection of the parent device is accepted, and the pressing of the "OK" button is accepted, so that the selected parent device can be operated as a child device to request entry into the room. shall be accepted.

The CPU 201 of the client terminal 100B turns off all of its own functions (camera, microphone, speaker) (step S506), and transmits the details of the operation received in step S505 to the server 200 (step S507).

Each client terminal 100 manages on/off of its own camera, microphone, and speaker functions using a setting table shown in function setting 850 in FIG. 8, which is stored in its own memory. When turning on the function, the ON/OFF 852 of the target function name 851 is set to ON. When turning off the function, the ON/OFF 852 of the target function name 851 is set to OFF.

The CPU 201 of the server 200 receives the information on the operation details accepted by the client terminal 100B in step S505, and determines whether or not the operation of entering the conference room (eg, room001) as a slave device has been accepted (step S508).

When the operation to enter the room as a general machine is accepted, the same user (User001 of the client terminal 100A) as the user of the client terminal 100B (example: User001) in the conference room to enter is logged out, and the process proceeds to step S408 in FIG. transition to

When the room entry process is performed as a general machine, the CPU 201 of the server 200 inserts "all machines" into the entry notice 816 of the data of the user who has entered the room in the conference information 810 and stores it.

Further, when the pressing of the “cancel” button is accepted, the user (User001 of the client terminal 100B) who has logged in at step S402 is logged out from the server 200. FIG.

The room entry notification 816 is data that determines which terminals participating in the conference are to be notified that the user of the data of the room entry notification 816 has entered the room.

The CPU 201 of the server 200 transmits a room entry notification of the client terminal 100 that has entered the room as a general machine to all machines participating in the same conference, according to the content set in the room entry notification 816 .

On the other hand, if it is determined that an operation to enter the room as a child device has been received, the process proceeds to step S509.

When the CPU 201 of the server 200 determines that the operation of entering the room as a child device has been accepted, the CPU 201 of the server 200 determines to allow the client terminal 100B to enter the target conference room as a child device, and holds the terminal ID of the client terminal 100B in the memory. (step S509).

Also, the master device selected on the confirmation screen 920 is determined to be the master device of the client terminal 100B, and the terminal ID of the master device is stored in the memory. Here, it is assumed that the client terminal 100A is selected as the parent device (step S510).

The CPU 201 of the server 200 associates the terminal IDs of the parent device and the child device and stores them in the memory (step S511). Specifically, the value of the client terminal 100A is recorded in the parent device ID 841 of the parent-child information 840 in FIG. 8, and the value of the client terminal 100B is recorded in the child device ID 842. The parent-child information 840 is a table created on the memory by the server 200 for each meeting ID at the time of creating the meeting ID.

The child device function 843 indicates what function the child device is designated as a terminal to execute in the relationship between the terminals of the parent device ID 841 and the child device ID 842 . No value is inserted in the slave function 843 at this time.

The CPU 201 of the server 200 allows User001 of the client terminal 100 to enter room001, which is the destination conference room (step S512). An example of conference information after entering the room is shown in 810 in FIG.

In the case of processing for entering a room as a child device, the CPU 201 of the server 200 inserts "parent device only" into the room entry notification 816 of the data of the user who has entered the room in the conference information 810 and stores it.

The CPU 201 of the server 200, in accordance with the content set in the room entry notification 816, transmits the room entry notification of the client terminal 100 that has entered the room as the child terminal of the client terminal 100A only to the client terminal 100A that is the parent terminal (step S513/non-display). control means).

In other words, the fact that the same user has entered the room from two terminals is not notified to anything other than the parent terminal.

Other terminals that have not been notified of the room entry do not know that User001 of the client terminal 100B has newly entered the room. Also, the display area for User001 on the client terminal 100A and the display area for User001 on the client terminal 100B are not generated and displayed separately on the conference screen.

It is assumed that the notification includes an instruction to display function selection screen 1000 shown in FIG. For example, the CPU 201 of the server 200 creates screen information (html) of the function selection screen 1000 and transmits it to the client terminal 100A.

The function selection screen 1000 is a screen that allows the user to select which function the client terminal 100B, which has entered the room as a child terminal, should operate as a terminal to execute.

Areas 1001, 1002, and 1003 are areas for selecting dedicated devices (camera, microphone, and speaker) mounted/connected to the terminal or the terminal to function as a camera, microphone, and speaker, respectively.

The server 200 generates screens so that each of 1001, 1002, and 1003 can be displayed in a pull-down manner so that a child machine and a dedicated device can be selected. 1010 and 1020 show the selection of the device to perform the function.

A display unit 1004 displays an image being captured by the child machine or device that functions as the camera selected in 1001 .

An “OK” button 1005 is a button for confirming the contents selected in 1001 , 1002 , and 1003 . In other words, it is a button for determining which terminal is to be the terminal that executes the camera, microphone, and speaker functions by operating the parent device.

Returning to the description of FIG. The CPU 201 of the client terminal 100A receives notification of the ID of the child device that has entered the room as the same user as the client terminal 100A, and receives an instruction to display the function selection screen 1000 (step S514).

CPU 201 of client terminal 100A displays function selection screen 1000 according to the screen information included in the received instruction (step S515).

The CPU 201 of the client terminal 100A accepts a selection operation for selecting which terminal/device to execute which function on the function selection screen 1000 (step S516).

For example, in the description of the present embodiment, as a terminal that functions as a camera, the client terminal 100B, which is a child device, is selected, and as a terminal that functions as a microphone and a speaker, the client terminal 100A itself (a microphone mounted on the client terminal 100A) is used. and speaker) is accepted.

The CPU 201 of the client terminal 100A transmits to the server 200 information indicating that "camera" has been specified as a function to be executed by the child device, in order to enable the camera function of the child device (step S517). In addition, the function of the camera installed in the machine is set to OFF.

CPU 201 of server 200 receives information indicating that “camera” has been specified as a function to be executed by the child device, and stores the value of “camera” in child device function 843 of parent-child information 840 . (Step S518).

The CPU 201 of the server 200 notifies the client terminal 100B, which is a child device, to enable the "camera" function (step S519).

The CPU 201 of the client terminal 100B receives the notification (step S520) and enables the notified function (step S521). Here, set the "Camera" function to ON.

Further, the CPU 201 of the server 200 transmits a screen display update instruction to the client terminal 100A, which is the parent device. For example, as indicated by 1100 in FIG. 11, the client terminal 100A generates screen information for a conference screen that identifies and displays the client terminal 100B that has become a child terminal and identifies and displays that the client terminal 100A has become the parent terminal. 100A (step S522).

The adjustment button 1101 in FIG. 11 is a button for displaying a function adjustment screen 1201 (shown in FIG. 12) for adjusting the quality of the camera, microphone, and speaker of the child device. This is identification information indicating that the terminal is the master of another terminal.

The client terminal 100A receives the conference screen 1100 and displays it on the display screen (step S523).

Further, the CPU 201 of the server 200 transmits a screen display update instruction to the client terminal 100B, which is a child device. For example, as indicated by 1120 in FIG. 11, the client terminal 100B generates screen information for a conference screen that identifies and displays the client terminal 100A that has become the parent terminal and identifies and displays that the client terminal 100B has become the child terminal. (step S524).

The client terminal 100B receives the conference screen 1120 and displays it on the display screen (step S525).

The other client terminals 100 (eg, 100C, 100D) that are neither parent nor child devices display conventional conference screens such as 1110 . However, if the camera for capturing the conference image of User001 is changed from the client terminal 100A to the client terminal 100B, the conference image captured by the camera of the client terminal 100B will be displayed in the display area of User001. . The above is the description of FIG.

Next, with reference to FIG. 6, the flow of processing for transmitting and receiving images and sounds to the server in the embodiment of the present invention will be described.

The process of FIG. 6 is started and executed at the timing when the user's entry into the conference room by the client terminal 100 is completed.

In the description of FIG. 6, it is assumed that the client terminal 100B is a child device of the client terminal 100A and enters the room as a terminal operating as a camera terminal.

The CPU 201 of the client terminal 100A identifies the functions enabled in its own device. Here, it is specified that the microphone function is enabled (step S601).

The CPU 201 of the client terminal 100A repeats the process of step S603 at predetermined time intervals until the end of the conference (step S602). For example, repeat every 0.5 seconds.

The CPU 201 of the client terminal 100A acquires voice information input from its own microphone and transmits it to the server 200 (step S603).

The CPU 201 of the server 200 receives the voice information (step S621), and identifies the terminal ID of the transmission source from the information of the session established when the voice was transmitted and received (step S622).

The CPU 201 of the server 200 refers to the parent-child information 840 and determines whether or not the terminal that has transmitted the specified audio is a child device (step S623). If it is not a child device, the process proceeds to step S626, and the received voice information is stored in a voice queue (not shown) of the user of the terminal of the voice transmission source (step S626).

The audio queue is information for managing audio information for each predetermined time interval, including a sequence number, a user ID, and a destination user to whom the audio is distributed. That is, it is information for storing and managing audio information instead of the image 833 of the image queue shown in 830 in FIG. The voice queue is created and managed on the memory by the server 200 for each terminal that has entered the conference room.

When the terminal of the transmission source of the voice is a child device, the CPU 201 of the server 200 determines whether the terminal of the transmission source is designated as a terminal that executes the microphone function by referring to the child device function 843 (step S624).

If the terminal is specified as a terminal that executes the microphone function, the process proceeds to step S625, and the received voice information is stored in the voice queue of the user of the client terminal 100A (master device) (step S625). If the terminal is not specified as a terminal that executes the microphone function, the processing ends without storing the voice.

The CPU 201 of the client terminal 100B identifies the functions enabled in its own device. Here, it is specified that the camera function is enabled (step S611).

Until the end of the conference, the CPU 201 of the client terminal 100B repeats the processing of steps S613 to S615 for each frame of the camera image captured by its own camera (step S612).

The CPU 201 of the client terminal 100A displays the input image (video) from the camera on the conference screen in step S613, while performing compression processing of the image (video) by the video codec, and compresses the compressed image frame by frame. Then, the images are sequentially saved and stored in the image queue on the memory of the device itself (step S614).

An example of an image queue is shown in image queue 820 in FIG. The image queue 820 contains an identification number (terminal ID) for each image, a sequence number 821 indicating the order of the images, and an image 822 that is the entity of the image (image data/image of one frame in video). Configured.

The CPU 201 of the client terminal 100</b>A assigns sequence numbers to the images of each frame of the video acquired from the camera of the client terminal 100</b>A in the order in which the images are acquired, and stores the sequence numbers in the image queue 820 .

The CPU 201 of the client terminal 100A sequentially transmits the images stored in the image queue 820 to the server 200 together with the sequence numbers (step S615). That is, the information shown in the image queue 820 is transmitted to the server 200 .

The CPU 201 of the server 200 receives the image (compressed data) (step S627), and identifies the terminal ID of the transmission source from the session information established when the image was transmitted/received (step S628).

The CPU 201 of the server 200 refers to the parent-child information 840 and determines whether the terminal that has transmitted the specified image is a child device (step S628).

The CPU 201 of the server 200 shifts the process to step S632 if it is not a child device. In step S632, the CPU 201 of the server 200 stores the image by adding it to the image queue 830 of the user entering the room at the client terminal 100A (terminal ID), which is the transmission source of the image, held in the memory of the server 200 (step S632). S632).

When the user enters the conference room, the CPU 201 of the server 200 generates a conference image queue for each terminal of the user and stores it in the memory.

The same information as the sequence number 821 and the image 822 of the image queue 820 received from the client terminal 100 is associated and stored in the image queue 830 on the memory of the server 200 .

When the terminal of the transmission source of the image is a child device, the CPU 201 of the server 200 determines whether the terminal of the transmission source is designated as a terminal that executes the camera function by referring to the child device function 843 (step S630).

If the terminal is specified as a terminal that executes the camera function, the process proceeds to step S631, and the received image is stored in the image queue 830 of the user of the parent device (step S631).

If the terminal is not specified as a terminal that executes the camera function, the process ends without storing the image. The above is the description of FIG.

Next, with reference to FIG. 7, the flow of processing for acquiring images and sounds from the server in the embodiment of the present invention will be described.

7 are repeatedly executed after the server 200 completes the user's room entry process and the client terminal 100 receives from the server 200 the room entry completion notification. The point in time when the notification of the completion of entering the room is received is, for example, the point in time when the conference image is received.

Based on the list of users who are in the same meeting room as the user of the client terminal 100, the process after step S711 is executed for the users in the list, which the client terminal 100 receives from the server 200. be.

The transmission/reception and reproduction processing of audio will be described below.

When the speaker function of the client terminal 100 is valid, the CPU 201 of the client terminal 100 repeatedly executes the processes of steps S701 to S706.

The CPU 201 of the client terminal 100 transmits to the server 200 a request to acquire voices of other conference participants (audio queues of user IDs of other participants in the same conference room) (step S701). The request includes the user ID currently logged into the server 200 with the client terminal 100 .

The CPU 201 of the server 200 receives the acquisition request (step S702), and refers to the conference information 810 for the user ID of the user who is in the same conference room as the logged-in user other than the logged-in user of the requesting client terminal 100. to specify.

Then, among the voices in the voice queue of the other specified occupants, the voices of the latest sequence number (the last number) that have not been transmitted to the client terminal 100 that requested the image are obtained and synthesized (step S703). ), and transmits the synthesized voice to the client terminal 100 that made the request (step S704). If the same user has voice cues for the child device and the parent device, the voice stored in the voice queue for the parent device is used for synthesis.

In addition, in association with each voice used for synthesis, the user ID who is in the room at the destination client terminal 100 to which the voice was transmitted is stored as information of the transmitted user.

The CPU 201 of the client terminal 100 receives the audio information transmitted in step S708 (step S705), and reproduces it through the speaker (step S706).

It is assumed that the CPU 201 of the client terminal 100 sequentially reproduces the received audio information.

Image transmission/reception and display processing will be described below.

The CPU 201 of the client terminal 100 identifies a predetermined user among other conference participants, and transmits a conference image acquisition request of the user to the server 200 (step S711). For example, the display area currently displayed on the conference screen is specified, and the conference image of the user corresponding to the display area is requested from the server.

It is assumed that the image acquisition request includes the user ID of the user logging into the server 200 with the client terminal 100 .

The CPU 201 of the server 200 accepts the acquisition request (step S712), and the user ID of a user who is in the same conference room as the logged-in user, other than the logged-in user of the client terminal 100 who made the request, receives the request. The conference information 810 is referenced to specify the user ID of the user who received the call. Then, the conference image of the user whose acquisition request is accepted is specified.

Among the images in the image queue of the identified other room occupants, the image of the latest sequence number (the last number) that has not been transmitted to the client terminal 100 that requested the image is acquired (step S713), and the acquired conference The image is transmitted to the requesting client terminal 100 together with the user ID corresponding to the image (step S714).

If the same user has image queues of the child device and the parent device, the image stored in the image queue of the parent device is acquired and transmitted.

Also, the user ID who is in the room at the client terminal 100 to which the image has been sent is stored in the transmitted user 834 (step S715).

The CPU 201 of the client terminal 100 receives the image (compressed data) transmitted in step S715 (step S716), performs decompression processing using the video codec to restore the video (image), and displays it on the conference screen being displayed. (step S717).

For example, each user's conference image is assigned to the number of the corresponding display area, stored in the memory, and displayed like the conference screen 1410 in FIG. It is assumed that the memory of the client terminal 100 stores information about which image of which user is to be displayed in which display area (information that associates the ID of the display area with the user ID).

The CPU 201 of the client terminal 100 sequentially displays/updates the received image of each user (conference image) in the display area corresponding to the user of each image, and reproduces/displays it as a moving image (video).

When a web conference is held, there may actually be a plurality of client terminals 100. Therefore, the CPU 201 of the server 200 performs steps S712 to S715 for the client terminals 100 with valid camera functions that are entering the conference room. Only the number will be executed. The above is the description of FIG.

Note that the client terminal 100A, which is the parent device, displays a function adjustment screen 1201 shown in 1200 in FIG. 12 by accepting the selection operation of 1101 in FIG. An instruction to change the volume or image quality is transmitted to the terminal executing the function via the server 200. - 特許庁A terminal (for example, the client terminal 100B) that receives the instruction can change the volume and image quality according to the instruction.

It is assumed that the function selection screen 1000 is displayed when the pressing of the "detailed setting" button 1202 is accepted.

In other words, an operable screen regarding the function of the child device is generated and transmitted to the parent device for display (corresponding to operation control means).

Further, in the above-described embodiment, the terminal to be used as the parent device is selected, but the server 200 may automatically select the parent device, for example. Specifically, when the user who has received the request to enter the room has entered the conference room to which the room is to be entered, among the terminals of the user with the same user ID who is entering the room, the user entered the conference room first (data In step S509, the client terminal 100 of the user with the oldest save date and time is determined as the parent device.

When a plurality of users with the same ID have already entered one meeting room, the parent-child information 840 is referred to, and the client terminal 100 serving as the parent device is changed to the client terminal of the user who has newly accepted the room entry request. may be determined as the parent machine. For example, when the client terminal 100A is the master device and the client terminal 100B is the slave device, and a room entry request for User001 is received from the client terminal 100C for a conference room in which User001 is entering, the client terminal 100C is transferred to the client terminal 100A. Decide as a child device.

In the above description, the client terminal 100B that has logged in/entered the room as a child device is allowed to select which function to execute. , the client terminal 100B may select which function to participate in the conference as a child device.

Specifically, in step S503 of FIG. 5, the CPU 201 of the server 200 generates screen information for the confirmation screen 1300 and transmits the screen information to the client terminal 100B.

The client terminal 100B displays the confirmation screen 1300 on the display screen, and in step S505, in addition to the above-described operation of entering the room as a child terminal, the client terminal 100B asks the function selection receiving unit 1301 which function the client terminal 100B has as a terminal in the conference room. It accepts the operation of selecting whether to enter the room. Then, the content of the operation is transmitted to the server 200 .

The CPU 201 of the server 200 executes the processes of steps S508 to S512, skips the processes of steps S513 and S518, and in step S519 selects the client terminal 100B that has entered the room as a child terminal by the function selection reception unit 1301. is notified (instructed) to enable the function that accepts

The client terminal 100B receives the notification and enables the function selected by the function selection receiving unit 1301 in response to the notification.

As a result, it is possible to provide a mechanism in which the terminal newly participating in the conference can be easily used as a device for executing a predetermined function by an operation from the terminal.

Further, on the function selection screen 1000, the client terminal 100B may be displayed so as to be selectable only as a child terminal of the terminal whose function has been selected by the function selection reception section 1301 of the confirmation screen 1300. FIG.

For example, according to the selection state of 1301, in step S513, the pulldown of 1001 of the function selection screen 1000 is set so that the client terminal 100B can be selected. Information is generated and transmitted to the client terminal 100A (master device).

On the client terminal 100A that receives and displays the function selection screen 1000, the client terminal 100B is selectable in the 1001 pull-down menu, but is not displayed in the 1002 and 1003 pull-down menus. In other words, functions other than those designated/selected by the child device cannot be selected.

Further, when the client terminal 100B serving as a child device does not have a camera function, the server 200 notifies the server 200 that the client terminal 100B does not have a camera function but has a microphone and a speaker function. In 1001 of 1000, the client terminal 100B cannot be selected, and in 1002 and 1003, the client terminal 100B may be set to be selectable in a pulldown.

Accordingly, by narrowing down the functions that can be executed by the terminal, it is possible to facilitate the selection of functions by the user, and to provide a mechanism in which the terminal can be easily used as a device that executes a predetermined function.

In the above-described embodiment, the server generates and manages the audio queue and the image queue for each client terminal 100 that transmits the audio and the image. Queues and image cues may be generated, stored and managed on a memory.

In this case, instead of steps S625 and S626 in FIG. 6, a process of storing the voice in the voice queue corresponding to the user of the terminal from which the voice was received is executed. Also, instead of steps S631 and S632 in FIG. 6, a process of storing the image in the image queue corresponding to the user of the terminal from which the conference image was received is executed.

Further, in the above-described embodiment, the same user is permitted to log in twice to the server 200 once, and the processing of FIG. can be run with

For example, when a login request with the same user ID as that of a logged-in user who has already logged into the server 200 is received, the conference in which the logged-in user is participating (the conference room that the login user is in) is specified, and the conference room of the conference is specified. It may be determined that the request to enter the room has been received, and the process of step S501 may be executed. For example, if the user presses the "cancel" button on the confirmation screen 920, the login is not performed and the request is discarded. As a result, it is possible to prevent consumption of processing resources due to one-time login.

In addition, for example, when the client terminal is placed in the center of the conference room as a dedicated microphone and speaker, or installed at the end of the conference room as a dedicated camera, it is easier to carry and install than a large PC. Mobile terminals that are easy to use are often used.

Therefore, for example, when a login request is received, an inquiry is made to the client terminal 100 as to whether the client terminal 100 that has issued the login request is a mobile terminal. You may make it perform the room entry process as machine.

As described above, according to the present invention, it is possible to provide a mechanism in which a terminal participating in a conference can be easily used as a device that executes a certain function.

The present invention can also be embodied as, for example, a system, apparatus, method, program, storage medium, etc. Specifically, it may be applied to a system composed of a plurality of devices, or , may be applied to a device consisting of one device.

It should be noted that the present invention includes those that directly or remotely supply a software program that implements the functions of the above-described embodiments to a system or apparatus. The present invention also includes a case where the computer of the system or apparatus reads and executes the supplied program code.

Therefore, the program code itself installed in the computer to implement the functional processing of the present invention also implements the present invention. That is, the present invention also includes the computer program itself for realizing the functional processing of the present invention.

Recording media for supplying programs include, for example, flexible disks, hard disks, optical disks, magneto-optical disks, MOs, CD-ROMs, CD-Rs, and CD-RWs. There are also magnetic tapes, non-volatile memory cards, ROMs, and DVDs (DVD-ROM, DVD-R).

Another method of supplying the program is to connect to a home page on the Internet using a browser on the client computer. It can also be supplied by downloading the computer program itself of the present invention or a compressed file including an automatic installation function from the home page to a recording medium such as a hard disk.

It is also possible to divide the program code constituting the program of the present invention into a plurality of files and download each file from a different home page. In other words, the present invention also includes a WWW server that allows a plurality of users to download program files for implementing the functional processing of the present invention on a computer.

In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. Let Then, by using the downloaded key information, the encrypted program can be executed and installed in the computer.

Also, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, the OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

Further, the program read from the recording medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, based on the instruction of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the processing also realizes the functions of the above-described embodiments.

It should be noted that the above-described embodiments merely show specific examples for carrying out the present invention, and the technical scope of the present invention should not be construed to be limited by these.
That is, the present invention can be embodied in various forms without departing from its technical concept or main features.

100A client terminal 100B client terminal 100C client terminal 100D client terminal 101 network 200 server

Claims (20)

A terminal associated with the first user,
a communication means for communicating with a communication system to which a plurality of information processing terminals are communicably connected;
When the communication means is connected to the communication system, and a first terminal associated with the first user and different from the terminal device is connected to the communication system, the first terminal a control means for controlling to enable a specific function of the terminal device from disabled based on an instruction operation performed on the
A terminal device comprising: 2. The terminal device according to claim 1, wherein said control means does not control a specific function of said first terminal based on an operation on said terminal device. When the terminal device is connected to the communication system and the first terminal associated with the first user is currently connected to the communication system, the terminal device is connected to the first terminal device. selection means for selecting whether to connect as a controlled terminal that controls the specific function or to connect as a terminal that is not the controlled terminal, based on an operation performed on the terminal The terminal device according to claim 1 or 2. 4. The controlling means further controls to adjust the extent to which the specific function is applied based on the operation performed on the first terminal . The terminal device described in . The specific function is a function selected from the plurality of functions by the instruction operation performed at the first terminal on a selection screen from which a function to be executed by the terminal device can be selected from among the plurality of functions. The terminal device according to any one of claims 1 to 4, characterized in that there is a A terminal associated with the first user,
a communication means for communicating with a communication system to which a plurality of information processing terminals are communicably connected;
When the communication means is connected to the communication system, and a second terminal associated with the first user and different from the terminal device is connected to the communication system, the second terminal a receiving means for receiving an instruction operation to enable a specific function of from disabled to enabled;
a control means for controlling to transmit information corresponding to the reception of the instruction operation by the reception means;
A terminal device comprising: 7. The terminal device according to claim 6, wherein said control means performs control so as to disable said specific function of said terminal device in response to receiving said instruction operation. 8. The terminal device according to claim 6, wherein said control means controls transmission of said information to said communication system. 9. The terminal device according to any one of claims 6 to 8, wherein the instruction operation is an operation on a selection screen that allows selection of a function to be executed by the second terminal from among a plurality of functions. The accepting means further accepts an adjusting operation for adjusting the degree of the specific function of the second terminal enabled based on the instruction operation,
10. The terminal device according to any one of claims 6 to 9, wherein said control means performs control so as to transmit an instruction corresponding to said adjustment operation. The specific function is a terminal connected to the communication system and is a function of acquiring information to be transmitted to a third terminal which is a terminal of another user different from the first user. The terminal device according to any one of claims 1 to 10. 12. The terminal device according to any one of claims 1 to 11 , wherein said specific function is a microphone function. 11. The terminal device according to any one of claims 1 to 10 , wherein said specific function is a speaker function. 12. The terminal device according to any one of claims 1 to 11 , wherein said specific function is a camera function. A program for causing at least one computer to function as each means of the terminal device according to any one of claims 1 to 14 . A method of controlling a terminal device associated with a first user, comprising:
a communication step of communicating with a communication system to which a plurality of information processing terminals are communicably connected;
When the first terminal is connected to the communication system by the communication step, and a first terminal associated with the first user and different from the terminal device is connected to the communication system, the first terminal a control step of controlling to enable a specific function of the terminal device from disabled based on an instruction operation performed on the
A control method for a terminal device, comprising: A method of controlling a terminal device associated with a first user, comprising:
a communication step of communicating with a communication system to which a plurality of information processing terminals are communicably connected;
When a second terminal different from the terminal device is connected to the communication system by the communication step and is associated with the first user and is connected to the communication system, the second terminal a receiving step of receiving an instruction operation to enable a specific function of from disabled;
a control step of controlling to transmit information in response to receiving the instruction operation in the receiving step;
A method of controlling a terminal device, comprising: A communication system configured to enable communication between a first terminal associated with a first user, a second terminal associated with the first user, and a plurality of information processing terminals,
receiving means for receiving, at the first terminal, an instruction operation to enable a specific function of the second terminal from disabled ;
a control means for controlling to enable the specific function of the second terminal from disabled based on the instruction operation received by the receiving means ;
A communication system comprising: A program for causing at least one computer to function as each means of a communication system according to claim 18. A method of controlling a communication system configured to enable communication between a first terminal associated with a first user, a second terminal associated with the first user, and a plurality of information processing terminals There is
a receiving step of receiving, at the first terminal, an instruction operation to enable a specific function of the second terminal from disabled ;
a control step of controlling to enable the specific function of the second terminal from disabled based on the instruction operation received in the receiving step ;
A control method for a communication system, comprising:

Images