JP2018036688A - Information processing device, server device, information processing system, control method thereof, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mechanism that easily informs a device of a user focused of that the user is focused.SOLUTION: A convention image, a user of the convention image, and an information processing device of the user are managed correspondingly, selection of the convention image focused from among the plurality of convention images displayed on a display screen of the information processing device is accepted, the information processing device of the user of the convention image, in which the selection is accepted, is identified, the information processing device which has accepted the selection of the convention image is identified, and it is informed to the identified information processing device of the user of the convention image that the convention image of the user has been selected.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an information processing device, a server device, an information processing system, a control method thereof, and a program.

  In recent years, web conferencing systems have become popular. A web conference is a video conference which is one of electronic conferences.

  For example, Patent Document 1 describes a mechanism of a web conference system by a plurality of users using a portable terminal such as a tablet. In a web conference, for example, an image of a user who operates the terminal is captured using a camera mounted on the client terminal or a camera separately installed and connected on the client terminal, and the captured image that is the captured image Is transmitted to other terminals participating in the same conference as the own device, and the image of the user of the conference partner terminal is displayed on the display screen.

  Further, in Patent Document 2, for example, when a voice conference is performed with three or more terminals, a voice relay server is assigned to the designated destination device by accepting designation of a destination terminal to which voice is to be transmitted. On the other hand, a technique for relaying and transmitting uttered voice only is described.

Japanese Patent Laid-Open No. 2015-82207 JP 2008-124597 A

  The number of people when conducting an electronic conference such as a web conference varies. As the number of participants in the conference increases, the number of images of the conference partner users displayed on the display screen also increases, so that each user participating in the conference has difficulty in confirming the detailed state of the conference partner.

  For example, it becomes difficult to confirm which user in the conference partner is paying attention to him or how he / she is listening to his / her state.

  An object of the present invention is to provide a mechanism for easily notifying a noticed user's device that the user has been noticed.

  The present invention includes the conference image including a plurality of information processing devices having a function of transmitting a conference image obtained by imaging a user of the information processing device to another information processing device and displaying the received conference image on a display screen; An information processing system including a management unit that associates and manages a user of the conference image and an information processing device of the user of the conference image, the plurality of conference images displayed on a display screen of the information processing device A selection receiving unit that receives a selection of a conference image of interest from among the first and an information processing device of a user of the conference image that has received a selection by the selection receiving unit using information managed by the management unit Specifying means, a second specifying means for specifying an information processing apparatus that has received the selection of the conference image by the selection receiving means, and a specification by the first specifying means Notification means for notifying the user information processing apparatus of the conference image that has been selected in the information processing apparatus specified by the second specifying means. Features.

  According to the present invention, it is possible to provide a mechanism for easily notifying a noticed user's device that the user is noticed.

It is a figure which shows an example of a structure of the information processing system in embodiment of this invention. It is a figure which shows an example of the hardware constitutions of various apparatuses in embodiment of this invention. It is a figure which shows an example of a function structure of the various apparatuses in embodiment of this invention. It is a flowchart which shows the flow of the meeting image display process in the 1st Embodiment of this invention. It is a flowchart which shows the flow of the notification process to the attention user in the 1st Embodiment of this invention. It is a figure which shows an example of the various data structure in the 1st Embodiment of this invention. It is a figure which shows an example of a structure of the meeting selection screen in the 1st Embodiment of this invention. It is a figure which shows an example of a structure of the meeting image display screen in the 1st Embodiment of this invention, and an example of the mode of notice of a user of interest. It is a figure which shows an example of a structure of the meeting image display screen in the 1st Embodiment of this invention, and an example of the mode of notice of a user of interest. It is a flowchart which shows the flow of a process in the 2nd Embodiment of this invention.

  Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

  First, an example of the configuration of an information processing system in the embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 1, in the information processing system of the present invention, a server 200 and a client terminal 100 (a generic term for a client terminal 100A, a client terminal 100B, and a client terminal 100C) are communicably connected via a network 101. It is configured.

  The server 200 has various functions for executing a web conference using the client terminal 100. For example, creation and storage of a room for a web conference, identification information of the client terminal 100, and a conference image received from each client terminal (for example, a captured image (video) that is image data obtained by capturing a user who is a conference participant). Storage, storage of association information between the conference image and the terminal that transmitted the conference image, and control of transmission processing of the conference image to each form terminal are performed.

  The client terminal 100 is a terminal provided with a camera. For example, there are a client terminal with a built-in camera device and a client terminal to which an external camera device is connected.

  In the description of the embodiment of the present invention, the client terminal 100 includes an in-camera (a camera installed on the side where the display is installed) and an out-camera (the back side of the client terminal / the side where the display is installed). And a terminal provided with a touch panel function (operation unit).

  The client terminal 100 logs in to the server 200 in response to a user operation using the function of the conference application, and the same image (conference image / video) captured by the camera is acquired via the server 200. Are sequentially transmitted to other client terminals 100 entering the room (participating in the conference).

  In addition to the image data acquired from the camera, data stored in the client terminal 100, for example, static data such as photographic images and document data such as PDF files, can be transferred to other clients via the server 200. A file sharing function for transmitting and sharing to the terminal 100 is provided.

  In addition, the conference application has an application image sharing function for transmitting and sharing an image of a screen of an application other than the conference application running on the client terminal 100 to another client terminal. The above is the description of an example of the configuration of the information processing system in the embodiment of the present invention shown in FIG.

  Next, an example of the hardware configuration of various devices in the embodiment of the present invention will be described with reference to FIG.

  In FIG. 2, reference numeral 201 denotes a CPU that comprehensively controls each device and controller connected to the system bus 204. Further, the ROM 202 or the external memory 211 is necessary to realize a BIOS (Basic Input / Output System) or an operating system program (hereinafter referred to as an OS), which is a control program of the CPU 201, or a function executed by each server or each PC. Various programs to be described later are stored.

  A RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for execution of processing from the ROM 202 or the external memory 211 into the RAM 203 and executing the loaded program.

  An input controller 205 controls input from a keyboard (KB) 209 or a pointing device such as a mouse (not shown). A video controller 206 controls display on a display device such as a CRT display (CRT) 210. In FIG. 2, although described as CRT 210, the display device is not limited to the CRT, but may be another display device such as a liquid crystal display. These are used by the administrator as needed.

  In the embodiment of the present invention, the CRT 210 of the client terminal 100 has a touch panel function. When the CRT 210 has a touch panel function, the CRT 210 also functions as an input device, and the touch operation information detected by the touch panel function (received by the display) is transmitted to the input controller as input information. That is, the input controller 205 performs input control from the CRT 210.

  A memory controller 207 is connected to the hard disk (HD), flexible disk (FD), or PCMCIA card slot for storing a boot program, various applications, font data, user files, editing files, various data, etc. via an adapter. The access to the external memory 211 such as a compact flash (registered trademark) memory is controlled.

  A communication I / F controller 208 connects and communicates with an external device via a network (for example, the network 101 shown in FIG. 1), and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the CRT 210 by executing outline font rasterization processing on a display information area in the RAM 203, for example. In addition, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the CRT 210.

  Various programs to be described later for realizing the present invention are recorded in the external memory 211 and executed by the CPU 201 by being loaded into the RAM 203 as necessary. Furthermore, definition files and various information tables used when executing the program are also stored in the external memory 211, and a detailed description thereof will be described later.

  An acceleration sensor 212 detects, for example, the acceleration of the client terminal 100. When the acceleration sensor is attached to an apparatus such as the client terminal 100 as an external apparatus, for example, the acceleration sensor 212 communicates the detected acceleration information to the client terminal 100 by communicating with the input controller 205. The acceleration information is accumulated and stored as a history of the operation of the client terminal on the RAM 203, for example. The above is an explanation of an example of the hardware configuration of various apparatuses in the embodiment of the present invention shown in FIG.

  Next, with reference to FIG. 3, an example of a functional configuration of various apparatuses in the embodiment of the present invention will be described. FIG. 3 is a diagram illustrating an example of a functional configuration of various apparatuses according to the embodiment of the present invention.

  3 are functional units included in a conference application (client application) installed in the client terminal 100. Each functional unit 322 to 328 is a functional unit included in a conference application (server application) installed in the server 200.

  The captured image storage control unit 311 is a conference image of its own device that is an image to be transmitted to other terminals participating in the conference (for example, the client terminals 100B to 100C for the client terminal 100A). As a control unit that performs control to be stored in the external memory.

  The conference image transmission unit 312 transmits the conference image of the host device stored in the host device to the server 200 (other terminals participating in the same conference as the host device (the destination terminal) via the server 200). To the transmitter). When an unsent conference image is transmitted, information indicating that it has been transmitted is stored in association with the conference image, so that the transmitted conference image and the unsent conference image are managed separately.

  The conference information storage unit 328 stores and manages which user is participating in which conference.

  The conference image reception unit 321 receives the conference image transmitted from each client terminal 100, and the conference image storage unit 322 stores the received conference image for each terminal and user of the conference image transmission source. The conference image transmission unit 323 displays each conference image stored in the conference image storage unit 322 of a user who is participating in the same conference as the user of the terminal that is the transmission source of each conference image other than the transmission source of each conference image. It is a transmission part which transmits to a terminal. For example, assuming that the client terminals 100A to 100C are participating in the same conference, the conference images received from the client terminal 100A are transmitted to the client terminals 100B to 100C, respectively.

  The conference image receiving unit 313 receives the conference image of the other device transmitted from the server 200, and the conference image display unit 314 displays the received conference image of the other device and the conference image of the own device on the display screen. Each area is stored and displayed in the display area.

  The conference image selection receiving unit 315 receives the selection of the conference image displayed on the conference image display unit 314.

  The user specifying unit 316 specifies the user of the conference image whose selection has been received by the conference image selection receiving unit 315 as the target user who is the focused (attention) user, and stores it in the memory. Further, the user of the client terminal 100 that has selected the target user's image in the conference image selection receiving unit 315 is specified as the target user who has focused on the target user, and is stored in the memory.

  The notification unit 317 pays attention to the target user by the user of the user's own device at the client terminal of the user specified by the user specifying unit 316 (the terminal of the conference image transmission source that has received selection by the conference image selection receiving unit 315). In order to notify that the server 200 is informed, the server 200 is notified.

  The notification reception unit 324 receives the notification, and the notification source identification unit 325 refers to the notification source information included in the notification and identifies the notification source of the notification. For example, the notification source client terminal 100 and the user are specified.

  The notification destination specifying unit 326 specifies the notification destination of the notification with reference to the notification destination information included in the notification received by the notification receiving unit 324. For example, the client terminal 100 to be notified and the target user are specified.

  The notification unit 327 notifies the client terminal 100 of the notification destination specified by the notification destination specifying unit 326 that the target user has focused on the terminal of the target user.

  The notification reception unit 318 receives the notification, and the notification display unit 319 displays the content of the received notification on the display screen. The above is the description of FIG.

  In the following description, it is assumed that the processing of each step in each flowchart is executed by a CPU of each device using a function included in an application installed in each device.

<First Embodiment>
Hereinafter, the first embodiment of the present invention will be described with reference to FIGS.

  First, the flow of the conference image display process in the first embodiment of the present invention will be described with reference to FIG.

  In step S401, the client terminal 100 activates the conference application in response to a user operation.

  In step S <b> 402, the client terminal 100 displays a login screen (not shown), receives input of a user ID and password, and transmits the input user ID and password to the server 200, whereby a conference application (client application and server). Login processing of the user with the user ID to the application) is performed. Strictly speaking, when the server 200 that has received the user ID and password determines that the user ID and password are stored in the own machine (authentication has been successful), the user is logged in to the server application, By notifying the client application of the client terminal 100 that the authentication is successful, the client application performs a user login process to itself. An example of the user ID and password is shown in the user information 600 of FIG.

  In step S403, the client terminal 100 requests the server 200 for a conference selection screen 700 (illustrated in FIG. 7) to be displayed after login. The conference selection screen is a screen for selecting a conference to participate by selecting a “participation” button 702 from a list 701 of conference rooms where the conference is held.

  In step S404, the server 200 receives the request, and in step S405, generates the conference selection screen 700 and transmits it to the client terminal 100. Specifically, html information (screen information) of the generated conference selection screen 700 is transmitted. In the screen information, the selection unit of each conference room and the conference room ID are stored in association with each other.

  In step S406, the client terminal 100 receives the screen information, analyzes html, and displays it on the display screen.

  In step S <b> 407, the client terminal 100 accepts a conference room selection operation (selection operation for a conference to join) from the user. Then, the conference room ID of the selected conference room is specified from the screen information, and the entry request of the login user to the conference room is transmitted to the server 200. The milk request includes the user ID of the login user of the client terminal 100 and the terminal ID of the client terminal 100 itself.

  In step S408, the server 200 accepts the request, and in step S409, the server 200 performs a process of entering the requested conference room.

  Specifically, the user ID of the logged-in user is stored in association with the conference room that has been requested to enter the room and is stored in advance in the external memory of the server 200. An example of information indicating the association is shown in conference information 610 in FIG. In step S409, the server 200 adds and stores the user ID of the logged-in user who is logged in at the client terminal 100 that is the entry request source, in the field of the participant ID 613 corresponding to the conference room ID 611 of the conference room requested to enter. . At this time, the terminal ID of the client terminal 100 to which the user is logged in is stored in the terminal ID 614. The conference ID 612 is an ID of a conference held in the conference room with the conference room ID 611, and is deleted when the conference ends.

  The right to speak 615 indicates whether the user indicated by the participant ID 613 has the right to speak in the conference with the conference ID 612. For example, a predetermined user who participates in the conference is given the right to set a speaking right of another user who is participating in the conference. For example, according to an operation received at the client terminal 100 of the predetermined user immediately after the conference starts, It is assumed that the server 200 that has received the operation information determines whether or not each person has the right to speak and stores it in the right to speak 615.

  Returning to the description of FIG. In step S410, the server 200 acquires the screen information of the conference screen shown in the conference screen 800 of FIG. 8 from the external memory of the own device, and transmits it to the client terminal 100. The screen information is, for example, html information indicating the layout of the screen, and includes display areas such as 801 and 802 and IDs of the respective display areas. That is, it is template information of the conference screen. At this point, no conference image is displayed in each display area.

  In step S411, the client terminal 100 receives the information on the conference screen and stores it in the memory. Further, the conference screen 800 is displayed on the display screen.

  In step S412, the client terminal 100 activates the camera. Specifically, the OS of the client terminal 100 is instructed to start a predetermined camera application installed in the camera application and start imaging with the camera.

  When the client terminal 100 is a mobile terminal, it is assumed here that the camera application is first instructed to start imaging using the in-camera.

  The client terminal 100 repeats the processes in steps S413 to S414 until the logged-in user leaves the conference room that is entering the room.

  In step S413, the client terminal 100 acquires a captured image captured by the camera activated by the camera application from the camera application, and stores it in the memory. Specifically, the captured images are sequentially stored in the conference image table 620 (FIG. 6) that stores images to be transmitted to the server 200 in step S414 described later (sequentially stored in the order of acquisition).

  The conference image table 620 is a table that accumulates and sequentially stores conference images for each conference image transmission source terminal. The transmission source terminal here is one of the client terminals 100A to 100C that transmits the conference image.

  The terminal ID 621 is a terminal ID of a transmission source terminal that transmits the conference image. At the time of step S413, the terminal ID of the own device is stored. The sequence number 622 indicates the stored order of the conference images. The conference image 623 is entity data (image data) of the conference image. In the conference image 623, images for each frame (one frame) of a moving image (moving image) captured by the camera are sequentially stored. “A005.jpg” or the like is the name of the image. The sequence number and the name of the image are given a unique number and name by the conference application when the conference application acquires a captured image from the camera application. The sequence number is assigned so that the next number to the number assigned immediately before is sequentially numbered each time a new image is acquired.

  The transmitted flag 624 is a flag indicating an image that has been transmitted to the server 200. The transmitted flag 624 = 1 indicates that transmission has been completed, and the transmitted flag 624 = 0 indicates that transmission has not been performed.

  In step S414, the client terminal 100 displays an undisplayed image in a predetermined display area of the display screen among the conference images of the own device (the conference image in which the terminal ID of the own device is stored in the terminal ID 621). For example, the conference image of the own device is displayed in the display area 802. Also, the oldest unsent conference image is acquired and transmitted to the server 200. After transmission, the transmitted flag 624 is changed to 1 to manage the conference image as transmitted on the conference image table.

  In step S415, the server 200 receives the conference image transmitted from the client terminal 100, and in step S416, stores it in the memory in association with the terminal ID of the transmission source. Specifically, the conference image transmission source terminal ID (621) and the conference image (623) are stored in association with each other in the conference image table managed by the server application having the same configuration as the conference image table 620.

  On the other hand, the processing in steps S417 to S418 is repeated until the logged-in user leaves the conference room that is entering the room. The processes in steps S417 to S418 are started after the process in step S411.

  In step S <b> 417, the client terminal 100 transmits a request for acquiring a conference image of another device to the server 200.

  In step S420, the server 200 receives the request. In step S421, the server 200 acquires the conference image of the other device from the conference image table in its own memory in response to the request and transmits it to the client terminal 100. In the conference image table of the server 200, the terminal ID of the client terminal 100 that has already been transmitted is inserted and stored in the field corresponding to the transmitted flag 624 in place of the transmitted flag. That is, which conference image is transmitted to which terminal is stored. In step S421, a conference image that has not yet been transmitted to the requesting client terminal 100 is identified and acquired, and is transmitted together with the identification information of the client terminal 100 that is the transmission source of the conference image.

  In step S418, the client terminal 100 sequentially receives the conference images and stores them in the RAM 203. The user corresponding to the client terminal 100 that is the source of the conference image is identified with the identification information of the client terminal 100 and the conference in FIG. A display area for displaying the conference image of the user is determined by referring to the information 610.

  Specifically, the list of display areas as shown in the display area / user association information 630 in FIG. 6 is referred to, and the smallest number among the first display areas in the display areas to which the user is not yet associated. The display area ID 631 is specified, and the user ID of the user of the conference image received is stored as the user ID 633. That is, the display area and the user are stored in association with each other.

  The display area / user association information 630 is a list of display areas (for example, 801 and 802) in the conference screen displayed on the client terminal 100. A type 632 is a type of the display area. Only 801 in FIG. 8 is a “large” region (main image display region), and other display regions such as 802 other than 801 are “small” regions (sub image display regions).

  The selection flag 634 is an identification flag for a display area that receives a selection on the conference screen and is in a selected state. A selection flag 634 = 1 indicates that it is in a selected state. A selection flag 634 = 0 indicates that it is in a non-selected state.

  In step S418, the client terminal 100 sequentially displays the received conference image in the display area that stores the conference image associated with the user. Specifically, in the display area / image association information 640 for storing the latest conference image 642 to be displayed in each display area (display area ID 641), the latest image received in the conference image 642 in the display area to be displayed is received. The conference image 642 is written and stored, the conference image 642 is read out on the memory, and displayed / updated in the display area of the corresponding display area ID.

  The information of the display area / user association information 630 and the display area / image association information 640 is transmitted to the server 200 whenever it is updated, and the server 200 transmits the terminal ID of the transmission source client terminal 100 that has transmitted the information. It is assumed that they are stored and managed in association with each other. That is, which image of which user is displayed in which display area of which client terminal 100 is managed. The above is the description of FIG.

  Next, with reference to FIG. 5, the flow of the notification process to the user of interest in the first embodiment of the present invention will be described.

  In step S501, the client terminal 100A accepts a user operation on the conference screen 800. For example, a selection operation for the display area 801 or the display area 802 is accepted.

  The display area selection operation is, for example, a double tap operation on the display area. Further, it is a selection operation of a conference image displayed in the display region corresponding to the selected display region, and a user's selection operation corresponding to the terminal that is the transmission source of the conference image.

  The fact that the user has been selected means that the user who selected the user has focused on the selected user. For example, it can be considered that the user who has selected the user is trying to talk to the selected user.

  In step S502, the client terminal 100A determines whether the operation received in step S501 is a selection operation for the display area 801 or a selection operation for the display area 802. In the case of an operation on the display area 802, the process proceeds to step S503. In the case of an operation on the display area 801, the process proceeds to step S512.

  In step S503, the client terminal 100A specifies the ID of the selected display area 802 based on the screen information of the conference screen 800, and displays the record having the ID in the display area ID 631 as the display area / user association information. 630.

  Then, the user ID 633 of the specified record is specified and stored on the RAM 203 as the user ID of the target user.

  Specifically, notification information 650 (illustrated in FIG. 6) is generated on the RAM 203, and the target user ID 652 is inserted into the target user ID 652 and stored. Also, refer to the display area ID 651 of the display area 802 that received the selection, the latest conference image of the selected user as the conference image 653, and the conference information 610 as the terminal ID of the client terminal of the selected user. And is inserted into the notification destination terminal ID 654 and stored. Further, the terminal ID of the client terminal 100 that has received the selection of the display area is inserted into the notification source terminal ID 656, the user of the client terminal 100 is identified by referring to the conference information 610, and is inserted into the user ID 655 of interest. Remember.

  Here, it is assumed that user002 is selected and stored as the target user.

  In step S <b> 504, the client terminal 100 </ b> A displays the video image of the user 002 being the target user in the display area 801, and displays the video image of the user displayed in the display area 801 in the selected display area 802.

  Specifically, the conference image 642 of user002 and the conference image of the display area 801 (conference image 642 = A004.jpg stored and displayed in association with display area ID = 1 in FIG. 6) are exchanged. The state of replacement of the conference image is shown in 810 to 820 in FIG.

  In the present embodiment, in the client terminal 100A, the conference image of user002 displayed in the display area with the display area ID = 2, which is the selected display area 802, and the display area with the display area ID = 1 are displayed. Assume that the conference image of user003 that has been used is replaced. Thereafter, the conference image of user003 is continuously displayed in the display area of display area ID = 2 until the selection operation for the display area of display area ID = 2 is accepted again.

  In step S505, the client terminal 100A notifies the server 200 that the user 002 has become the target user, thereby requesting the client terminal 100 of the target user to notify the user 200. Notification information 650 is included in the notification.

  In step S506, the server 200 receives the notification and stores the notification information 650 included in the notification in the RAM 203. In step S507, the user of the client terminal 100 that is the notification source of the notification is identified as the focused user who is paying attention to the user 002 (corresponding to focused user identifying means). Specifically, the value of the user ID of the target user ID 655 is specified.

  Here, the notification source terminal is the client terminal 100A, and the client terminal 100A is a user001 terminal. In other words, it is assumed that the target user = user001 who focuses on the target user = user002.

  In step S508, the server 200 specifies a notification destination for notifying that the user 002 has been selected and focused in accordance with the notification received in step S506. Specifically, the notification destination terminal ID 654 of the notification information 650 received in step S506 is referred to, and the client terminal 100 of the noticed user (user002) notified is specified as the notification destination. Here, it is assumed that the terminal corresponding to user002 is the client terminal 100B.

  In step S509, the server 200 notifies the client terminal 100B corresponding to user002 that the user001 has focused on user002. For example, a message character string indicating that user001 has focused on user002 is generated and transmitted to the client terminal 100B. Further, the user ID of the user of interest is notified to the client terminal 100B.

  That is, it notifies that the conference image of user002 has been selected on the client terminal 100A.

  In step S510, the client terminal 100B accepts the notification, and in step S511, displays a message included in the notification on the display screen. An example of the conference screen of the client terminal 100B is shown at 830 in FIG. 8, and an example of a message display is shown at 831.

  In addition, the client terminal 100B specifies a display area corresponding to the user 001 who is the target user with reference to the display area / user association information 630 in the own device, and the display area corresponding to the user 001 (the conference image of the target user is displayed). The displayed display area) is identified and displayed (identification display control means). For example, as indicated by 832 in FIG. 8, a frame is displayed and highlighted in the display area in which the conference image of the user of interest is displayed.

  With the identification display, the target user can easily confirm which image user is paying attention to himself / herself. The above is the description of FIG.

  According to the first embodiment of the present invention, it is possible to easily notify a focused user's device that the user has focused.

  That is, the user using the notified apparatus can easily know who is paying attention among the users participating in the same conference.

  In the above-described embodiment, the noticed user is notified of the attention when it is noticed. For example, even after a predetermined time has elapsed since the attention was given, the same noticed user is sent to the same noticeable user. You may make it notify the said when paying attention.

  For example, when an unintended user conference image is selected due to an erroneous operation or the like, it is conceivable that the desired user conference image is immediately selected again. On the other hand, when a desired user is selected and focused on, it is conceivable that the focused on the user continues for a predetermined time or more.

  In other words, even if a predetermined time has passed since the focus of attention, the notification is given when the same target user is paying attention to the same target user. The noticeable user can be notified.

  Moreover, in embodiment mentioned above, although 800-830 of FIG. 8 was used as an example of a meeting screen, the structure of a meeting screen is not restricted to this. For example, 900 to 930 in FIG. 9 may be used as the conference screen. In FIG. 9, reference numeral 903 denotes a desktop screen called a PC, and the conference screen 900 is displayed as a window on the desktop. The client terminal 100 displays a dialog 911 shown in 910 by accepting pressing of the “layout” button 902 on the conference screen 900. The full screen display target user selection accepting unit 912 is a selection accepting unit for a user (target user) who displays a conference image in full screen. In the pull-down, which is the selection receiving unit, the user IDs of other users participating in the same conference as the client terminal 100 displaying the dialog are displayed so as to be selectable. The client terminal 100 identifies the user with the user ID selected by the full screen display target user selection accepting unit 912 as the target user, stores it in the notification information, and transmits it to the server 200. Further, the conference image of the user with the selected user ID is displayed on the full screen as indicated by 921 of the conference screen 920. The client terminal 100B that has received the message and the notice of the user of interest via the server displays the conference screen 930. For example, the OS is instructed to pop up the received notification message, and the message is displayed as indicated at 931. Also, as shown at 932, the user of interest is identified and displayed.

<Second Embodiment>
Hereinafter, a second embodiment of the present invention will be described with reference to FIG. Note that a description of processing common to the first embodiment is omitted.

  When a certain user is focused, it is possible that the focused user is going to talk to the focused user.

  However, for example, when a user is talking, the user may be focused on simply looking at the user's face and listening to the story.

  In the second embodiment, it is determined whether or not the focused user is talking to the focused user. When the focused user is trying to speak, the noticed user is notified of the fact.

  For example, it is possible to easily notify the noticed user's device to the attention of the user according to whether or not the function of transmitting the voice to the other device is effective in the client terminal.

  With this notification, it is possible to make preparations such as an answer when the user on the side to be spoken talks to.

  With reference to FIG. 10, the flow of processing in the second embodiment of the present invention will be described.

  The client terminal 100A starts the process of FIG. 10 immediately after step S502 of FIG.

  In step S <b> 1001, the client terminal 100 </ b> A reads the device setting 660 stored in its own external memory onto the RAM 203.

  The device setting 660 is information stored in the external memory 211 of the client terminal 100, and is information indicating which device (device indicated by the device name 661) is ON or OFF (ON / OFF 662). is there.

  Assume that there are three devices that are set to ON / OFF in the device setting 660 in the present embodiment: a camera, a microphone, and a speaker.

  The client terminal 100 sets ON / OFF 662 of the connected device to ON when a device corresponding to a camera, a microphone, or a speaker is connected to the client terminal 100, and sets it to OFF when the connection is released. Also, ON / OFF setting for each device can be performed on a device ON / OFF setting screen (not shown). The client terminal 100 changes the ON / OFF 662 setting corresponding to each device name 661 from ON to OFF / OFF according to the ON / OFF switching operation of each device received on the ON / OFF setting screen of the device. Switch to ON.

  In the present embodiment, the state where ON / OFF 662 is ON indicates that the function of the device is effective. A state where the ON / OFF 662 is OFF indicates that each function is disabled.

  For example, the state where the camera is ON is a state where the function of transmitting and acquiring a captured image by the camera installed in the client terminal 100 and the function of transmitting the conference image to the other client terminal 100 via the server 200 are valid. In the conference, the camera function is basically ON. For example, when the client terminal 100 is installed in a room that handles confidential information, the room image is not transmitted as a conference image. The camera function may be turned off.

  The state in which the microphone is ON is a state in which the voice acquisition function by the microphone installed in the client terminal 100 and the voice transmission function via the server 200 to the other client terminal 100 are valid.

  Further, the state where the speaker is ON is a state where the function of reproducing / sounding the voice received from the other client terminal 100 by the speaker installed in the client terminal 100 is valid.

  When talking in a conference, the microphone and speaker functions must be set to ON. However, for example, when conducting training or lectures using the web conferencing mechanism, the listener basically does not speak. Therefore, in the client terminal 100 of the user who is a listener, the microphone function may be turned off so as not to transmit extra noise to other terminals.

  In other words, if the user intends to talk to the target user, the microphone function should be ON in the client terminal 100 of the user who wants to talk to the target user.

  In addition, when the user is forced to respond to a call while participating in the conference, the client terminal 100 is operated to turn off the speaker function so that the conference sound does not enter the phone, and the speaker function may be turned off. is there.

  That is, when the user is ready to listen to stories from others, the speaker function of the user's client terminal 100 should be on.

  Returning to the description of FIG. In step S <b> 1002, the client terminal 100 </ b> A refers to the device setting 660 of the own device, and determines whether the voice transmission function to the other device client terminal is valid or invalid. (Applicable to valid judgment means)

  If the voice transmission function of the own device to the other device client terminal is valid, the process proceeds to step S503. That is, it is determined that the notification is necessary. If the voice transmission function of the own device to the client device of the other device is invalid, the process proceeds to step S512. That is, it is determined that the notification is unnecessary.

  As described above, in the second embodiment, it is possible to determine whether the focused user is talking to the focused user, and to notify the focused user when trying to speak.

  Therefore, even if a plurality of users are paying attention to themselves, the user who is paying attention is not simply a user who is paying attention to himself, but easily checks the user who is talking to the paying user. be able to.

  In the above description, it is determined that the voice transmission function is valid as the setting of the client terminal 100, and the necessity of notification is determined / determined. For example, the right to speak is set for the user participating in the conference. -It may be managed to determine / determine whether notification is necessary or not according to the right to speak.

  For example, referring to the user's right to speak instead of step S1001, the presence / absence of the right to speak is determined instead of step S1002, or to referring to the user's right to speak immediately after step S1002. Is determined (corresponding to authority determination means). If there is a right to speak, the process proceeds to step S503. If there is no right to speak, the process proceeds to step S512.

<Third Embodiment>
Hereinafter, a third embodiment of the present invention will be described. Note that description of processing that is common to the first embodiment and the second embodiment is omitted.

  In the third embodiment, when the focused user talks to the focused user, it is determined whether the focused user can respond to this, and the focused user is not merely the focused user but the focused focused user. It is possible to easily confirm the users who can talk with the selected users.

  Specifically, instead of step S1002 in FIG. 10 or immediately after YES is determined in step S1002, the client terminal 100 determines whether the speaker function of the own device is ON or OFF.

  When the speaker function is ON, the user of interest (notice of interest) is notified to the user of interest, and when the user of interest speaks, the user of the own device listens to the story The process proceeds to step S503 in FIG. That is, it is determined that the notification is necessary.

  On the other hand, when the speaker function is OFF, even if the target user is notified that the user (target user) is paying attention to the target user, the user of the target device may be notified when the target user speaks. Determines that it cannot hear the story, and the process proceeds to step S512 in FIG. That is, it is determined that the notification is unnecessary.

  As described above, in the third embodiment, when the focused user is spoken to the focused user, if the user can hear the story, the fact that the focused attention has been paid The user can be notified.

  Therefore, even if a plurality of users are paying attention to themselves, the user who is paying attention is not simply a user paying attention to himself / herself, but easily confirming a user who can have a conversation with the paying user. Can do.

  As described above, according to the present invention, it is possible to provide a mechanism for easily notifying the focused user's device that the user has focused.

  In the description of the above-described embodiment, each screen information is generated by the server 200 and the screen is displayed by the client terminal 100 receiving the screen information. In this case, the screen information request, generation, and transmission / reception processes are omitted, and the screen information stored in its own memory is read and displayed on the display screen.

  In the above-described embodiment, the client terminal 100A generates the notification information 650 and transmits it to the server 200 in step S503. For example, the client terminal 100A is the user of the user of interest selected by the own device. The server 200 that notifies the server 200 of the ID and the terminal ID of the device itself may generate the notification information 650.

  Specifically, the server 200 that has received the notification generates notification information 650 on the RAM 203. Then, the terminal ID of the notification source terminal 100 is stored in the notification source terminal ID 656. Also, referring to the conference information 610 in the own device, the record in which the same value as the notification source terminal ID 656 is stored in the ID 614 is referred to, and the participant ID 613 of the record is specified as the target user, and the target user ID 655 is specified. Remember.

  Further, the user ID of the target user included in the received notification is stored in the target user ID 652, and the record of the conference information 610 having the same value as the target user ID 652 in the participant ID 613 is referred to. Terminal ID 614 is specified as the notification destination terminal and stored in notification destination terminal ID 654.

  Even when the notification of the display area ID 651 selected by the client terminal 100 is received instead of the notification of the user ID of the target user, the server 200 of the client terminal 100 of the notification source in its own device is received. With reference to the display area / user association information 630 corresponding to the terminal ID, it is possible to identify the user (target user) of the conference image displayed at the time of the selection operation in the selected display area. By referring to the conference information 610, the client terminal of the user (target user) can be specified. The notification information 650 can be generated in the server 200 that has received the notification through the above processing.

  In the above-described embodiment, the message is transmitted to the notification destination terminal in step S509. However, for example, notification may be performed by transmitting notification information 650. In this case, the client terminal 100B that has received the notification information 650 generates a message indicating that the user with the target user ID 655 has focused on the user with the target user ID 652, and displays the message on the display screen in step S511.

  It should be noted that the present invention can be implemented as, for example, a system, apparatus, method, program, or storage medium, and can be applied to a system composed of a plurality of devices. You may apply to the apparatus which consists of one apparatus.

  Specifically, the client terminal 100 and the server 200 may be integrated, and the processing of each step of the above-described embodiment may be executed as one housing.

  Note that the present invention includes a software program that implements the functions of the above-described embodiments directly or remotely from a system or apparatus. The present invention also includes a case where the system or the computer of the apparatus is achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  Examples of the recording medium for supplying the program include a flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, and CD-RW. In addition, there are magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R), and the like.

  As another program supply method, a browser on a client computer is used to connect to an Internet home page. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. Let It is also possible to execute the encrypted program by using the downloaded key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto.
That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

100A client terminal 100B client terminal 100C client terminal 101 network 200 server

Claims (15)

The conference image and the conference image including a plurality of information processing devices having a function of transmitting a conference image obtained by imaging a user of the information processing device to another information processing device and displaying the received conference image on a display screen. An information processing system comprising management means for managing a user and the information processing device of the user of the conference image in association with each other,
Selection accepting means for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
First identifying means for identifying an information processing device of a user of a conference image whose selection is received by the selection receiving means, using information managed by the management means;
Second specifying means for specifying an information processing apparatus that has received selection of the conference image by the selection receiving means;
Notifying the user information processing apparatus of the conference image specified by the first specifying means that the user's meeting image has been selected in the information processing apparatus specified by the second specifying means. Notification means;
An information processing system comprising: Focused user identifying means for identifying a user of the information processing apparatus identified by the second identifying means as a focused user focused on a user of a conference image whose selection has been received by the selection receiving means;
With
The notifying unit pays attention to the user of the conference image for which the focused user identified by the focused user identifying unit has received selection by the selection receiving unit with respect to the information processing apparatus identified by the first identifying unit. The information processing system according to claim 1, wherein the information processing system notifies that it is in progress. In the information processing apparatus specified by the first specifying means as a notification destination in response to the notification by the notification means, the information processing apparatus includes an identification display control means for controlling the target user to be identified and displayed. The information processing system according to claim 2.   The information processing system according to any one of claims 1 to 3, wherein the notification unit performs the notification after a predetermined time has elapsed after the meeting image is selected by the selection reception unit. Validity determination means for determining whether the function of transmitting voice to another information processing apparatus is valid;
The notifying means receives the selection of the conference image by the selection accepting means in the information processing apparatus that has been judged to be effective by the validity judging means as a function of transmitting voice to another information processing apparatus. Notifying the user's information processing apparatus of the conference image specified by one specifying means that the user's meeting image has been selected in the information processing apparatus specified by the second specifying means. The information processing system according to any one of claims 1 to 4, wherein the information processing system is characterized in that: First authority presence / absence determining means for determining whether a user of the information processing apparatus that has received a selection of a conference image by the selection receiving means is a user having authority to transmit audio to another information processing apparatus;
When the first authority presence / absence determining means determines that the user of the information processing apparatus is a user who has the authority to transmit voice to the other information processing apparatus, the notification means Informing the user information processing apparatus of the conference image specified by the specifying means that the user's meeting image is selected in the information processing apparatus specified by the second specifying means. The information processing system according to any one of claims 1 to 5. Second authority presence / absence determining means for determining whether the user of the conference image whose selection has been received by the selection receiving means is a user having the authority to transmit audio to another information processing apparatus;
The notification means is determined by the second authority presence / absence determining means that the user of the conference image whose selection is received by the selection receiving means is a user who has the authority to transmit audio to the other information processing apparatus. In this case, the user's conference image is selected in the information processing device specified by the second specification means for the user's information processing device of the conference image specified by the first specification means. The information processing system according to any one of claims 1 to 6, wherein the information processing system is notified. The conference image and the conference image including a plurality of information processing devices having a function of transmitting a conference image obtained by imaging a user of the information processing device to another information processing device and displaying the received conference image on a display screen. A control method of an information processing system comprising a management unit that manages a user and an information processing apparatus of the user of the conference image in association with each other,
A selection accepting step for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
A first specifying step of specifying an information processing device of a user of a conference image that has received a selection by the selection receiving step, using information managed by the management unit;
A second specifying step of specifying an information processing device that has received the selection of the conference image in the selection receiving step;
Notifying the user's information processing device of the conference image specified in the first specifying step that the user's conference image has been selected in the information processing device specified in the second specifying step. A notification process;
A method for controlling an information processing system comprising: The conference image and the conference image including a plurality of information processing devices having a function of transmitting a conference image obtained by imaging a user of the information processing device to another information processing device and displaying the received conference image on a display screen. A program for controlling an information processing system including a management unit that manages a user and an information processing apparatus of a user of the conference image in association with each other,
The information processing system;
Selection accepting means for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
First identifying means for identifying an information processing device of a user of a conference image whose selection is received by the selection receiving means, using information managed by the management means;
Second specifying means for specifying an information processing apparatus that has received selection of the conference image by the selection receiving means;
Notifying the user information processing apparatus of the conference image specified by the first specifying means that the user's meeting image has been selected in the information processing apparatus specified by the second specifying means. An information processing system program which functions as a notification means. A function of transmitting a conference image obtained by imaging a user of the information processing device to a plurality of other information processing devices and displaying the received conference image on a display screen; the conference image, the user of the conference image, and the user of the conference image An information processing apparatus comprising a management means for managing the information processing apparatus in association with each other,
Selection accepting means for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
First identifying means for identifying an information processing device of a user of a conference image whose selection is received by the selection receiving means, using information managed by the management means;
Second specifying means for specifying an information processing apparatus that has received selection of the conference image by the selection receiving means;
Notifying the user information processing apparatus of the conference image specified by the first specifying means that the user's meeting image has been selected in the information processing apparatus specified by the second specifying means. Notification means;
An information processing apparatus comprising: A function of transmitting a conference image obtained by imaging a user of the information processing device to a plurality of other information processing devices and displaying the received conference image on a display screen; the conference image, the user of the conference image, and the user of the conference image A control method for an information processing device comprising management means for managing the information processing device in association with each other,
A selection accepting step for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
A first specifying step of specifying an information processing device of a user of a conference image that has received a selection by the selection receiving step, using information managed by the management unit;
A second specifying step of specifying an information processing device that has received the selection of the conference image in the selection receiving step;
Notifying the user's information processing device of the conference image specified in the first specifying step that the user's conference image has been selected in the information processing device specified in the second specifying step. A notification process;
A method for controlling an information processing apparatus, comprising: A function of transmitting a conference image obtained by imaging a user of the information processing device to a plurality of other information processing devices and displaying the received conference image on a display screen; the conference image, the user of the conference image, and the user of the conference image A program for controlling an information processing apparatus comprising management means for managing the information processing apparatus in association with each other,
The information processing apparatus;
Selection accepting means for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
First identifying means for identifying an information processing device of a user of a conference image whose selection is received by the selection receiving means, using information managed by the management means;
Second specifying means for specifying an information processing apparatus that has received selection of the conference image by the selection receiving means;
Notifying the user information processing apparatus of the conference image specified by the first specifying means that the user's meeting image has been selected in the information processing apparatus specified by the second specifying means. A program for an information processing apparatus that functions as a notification unit. The conference image and the conference are communicable with a plurality of information processing devices having a function of transmitting a conference image obtained by imaging a user of the information processing device to another information processing device and displaying the received conference image on a display screen. A server device comprising management means for managing an image user and an information processing device of the conference image user in association with each other,
Selection accepting means for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
First identifying means for identifying an information processing device of a user of a conference image whose selection is received by the selection receiving means, using information managed by the management means;
Second specifying means for specifying an information processing apparatus that has received selection of the conference image by the selection receiving means;
Notifying the user information processing apparatus of the conference image specified by the first specifying means that the user's meeting image has been selected in the information processing apparatus specified by the second specifying means. Notification means;
A server device comprising: The conference image and the conference are communicable with a plurality of information processing devices having a function of transmitting a conference image obtained by imaging a user of the information processing device to another information processing device and displaying the received conference image on a display screen. A control method of a server device comprising management means for managing an image user and an information processing device of the conference image user in association with each other,
A selection accepting step for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
A first specifying step of specifying an information processing device of a user of a conference image that has received a selection by the selection receiving step, using information managed by the management unit;
A second specifying step of specifying an information processing device that has received the selection of the conference image in the selection receiving step;
Notifying the user's information processing device of the conference image specified in the first specifying step that the user's conference image has been selected in the information processing device specified in the second specifying step. A notification process;
A control method for a server device comprising: The conference image and the conference are communicable with a plurality of information processing devices having a function of transmitting a conference image obtained by imaging a user of the information processing device to another information processing device and displaying the received conference image on a display screen. A program for controlling a server device comprising management means for managing an image user and an information processing device of the conference image user in association with each other,
The server device,
Selection accepting means for accepting selection of a conference image of interest from among the plurality of conference images displayed on the display screen of the information processing apparatus;
First identifying means for identifying an information processing device of a user of a conference image whose selection is received by the selection receiving means, using information managed by the management means;
Second specifying means for specifying an information processing apparatus that has received selection of the conference image by the selection receiving means;
Notifying the user information processing apparatus of the conference image specified by the first specifying means that the user's meeting image has been selected in the information processing apparatus specified by the second specifying means. A program of a server device, which functions as a notification unit.

Images