JP7421134B2 - Information processing system, information processing method, program - Google Patents

Description

The present invention relates to an information processing system, an information processing method, and a program.

Recently, as the trend toward remote work has accelerated, meetings are increasingly being held online (web conferences) rather than in real, face-to-face settings. While this type of remote communication has the advantage of eliminating the need to physically secure a conference room, it can also cause inconveniences that would not occur face-to-face, as they share the same space and cannot have face-to-face conversations.

Patent Document 1 discloses a technique for solving "conflict of speech," which is one of the problems in remote communication. Specifically, it describes a technique for reducing collisions between speeches by predicting the next speaker and highlighting the speaker.

JP 2017-111643 Publication

The technique described in Patent Document 1 is based on the premise that comments are shared by all conference participants, but in a conversation, the content to be said is often the content to be communicated only to a specific individual.

Therefore, an object of the present invention is to provide a mechanism that enables dialogue with a specific individual in an online meeting or the like.

The information processing system of the present invention includes a request receiving unit that receives a request for an over-the-ear conversation with a second user from a first user participating in a web conference, and a request destination for the over-the-ear conversation request received by the request accepting unit. a selection receiving means for accepting a selection of a response method to an overheard conversation from a certain second user; A control means for controlling the distribution of audio.

According to the present invention, it is possible to interact with a specific individual in an online meeting or the like.

A diagram showing an example of the configuration of an information processing system to which the present invention can be applied. A block diagram showing an example of the hardware configuration of an information processing device used as a client terminal 101 and a server device 102. Flowchart showing processing contents of the present invention Flowchart showing details of the process of step S302 Flowchart showing details of the process in step S304 Flowchart showing details of the process in step S308 Flowchart showing details of the process of step S608 An example of a screen displayed on the client terminal 101 An example of a screen displayed on the client terminal 101 An example of a screen displayed on the client terminal 101 An example of a screen displayed on the client terminal 101 Diagram explaining the outline of the present invention

Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a diagram showing an example of the configuration of an information processing system to which the present invention can be applied.

As shown in FIG. 1, a client terminal 101 and a server device 102 are communicably connected via a network 110.

The client terminal 101 is a terminal used and operated by users participating in a web conference, etc., and is a terminal on which screens shown in FIGS. 8 to 11 are displayed. Further, as shown in FIG. 1, a plurality of client terminals 101 are connected via a network 110. This configuration is based on the assumption that each user participating in a web conference uses one device. Specifically, the client terminal 101 is a personal computer (such as a notebook PC or a desktop PC), a tablet terminal, or a smartphone, but is not limited to these.

The server device 102 is a device that controls data exchange between client terminals in the web conference building. It has a function of receiving the user's audio data and image data from the client terminal based on the user's operation on the client terminal 101, and distributing it to other client terminals. Although only one server device 102 is shown in FIG. 1, a system in which a plurality of server devices cooperate with each other or a so-called cloud system may be used.

FIG. 2 is a block diagram showing an example of the hardware configuration of an information processing device used as the client terminal 101 and server device 102 of the present invention.

As shown in FIG. 2, the information processing device includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a storage device 204, an input controller 205, An audio controller 206, a video controller 207, a memory controller 208, and a communication I/F controller 209 are connected.

The CPU 201 centrally controls each device and controller connected to the system bus 200.

The ROM 202 or external memory 213 stores a BIOS (Basic Input/Output System) and an OS (Operating System), which are control programs executed by the CPU 201, as well as computer-readable and executable programs and various necessary programs for realizing this information processing method. Holds data (including data tables).

The RAM 203 functions as the main memory, work area, etc. of the CPU 201. The CPU 201 loads necessary programs and the like from the ROM 202 or the external memory 213 into the RAM 203 when executing processing, and executes the loaded programs to realize various operations.

The input controller 205 controls input from input devices such as a keyboard 210 and a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can issue various instructions by pressing (touching with a finger or the like) an icon, cursor, or button displayed on the touch panel.

Further, the touch panel may be a touch panel capable of detecting positions touched by multiple fingers, such as a multi-touch screen.

Video controller 207 controls display on an external output device such as display 212. The display shall also include the display of a notebook computer that is integrated with the main body. Note that the external output device is not limited to a display, but may be a projector, for example. Furthermore, an input device is also provided for the device capable of accepting the above-mentioned touch operation.

Note that the video controller 207 can control a video memory (VRAM) for display control, and can use a part of the RAM 203 as a video memory area, or can provide a separate dedicated video memory. It is possible.

Memory controller 208 controls access to external memory 213. External memory can be an external storage device (hard disk) that stores boot programs, various applications, font data, user files, editing files, various data, etc., a flexible disk (FD), or a PCMCIA card slot connected via an adapter. Compact flash (registered trademark) memory and the like can be used.

The communication I/F controller 209 connects and communicates with external devices via a network, and executes communication control processing on the network. For example, communication using TCP/IP, a telephone line such as ISDN, a 4G line, a 5G line of a mobile phone, etc. is possible.

Note that the CPU 201 enables display on the display 212 by, for example, executing an outline font development (rasterization) process in a display information area in the RAM 203. Further, the CPU 201 allows the user to give instructions using a mouse cursor (not shown) on the display 212.

Next, the outline of the present invention will be explained using FIG. 12.

When a web conference starts, a screen including images of conference participants (images taken with an in-camera, etc.) is displayed on the client terminal 101 of each conference participant, as shown in FIG. 12(1). FIG. 12(1) shows that four users A, B, C, and D are participating in the conference. Note that it is not necessary to display images of all participants participating in the conference, and a configuration may be adopted in which only a predetermined number of participants are displayed.

Here, user A requests an ear interview with user B by selecting user B's area by a predetermined operation (clicking or tapping continuously for a certain period of time, etc.). When a request for listening is made, the screen displays the image of user A in the area of the image of user B, as shown in Figure 12 (2), and the information that identifies user A (name, etc.) is displayed in the area of user A. be done.

User B, who has been requested to listen, selects a response method corresponding to the request via the response selection interface shown in FIG. When a response to allow the listening experience is selected, the screen changes to the screen shown in FIG. 12(3), and the listening conversation begins.

When a listening conversation begins, the voices of user A and user B are controlled so that they are transmitted only to the other party (in this case, user A's voice is transmitted only to user B, and user B's voice is transmitted only to user A. ). Note that the voices of other conference participants can be heard by users A and B, but while certain operations are being performed (for example, selecting an area where the image of the other user is being displayed). controls the audio volume of other conference participants to be lowered. This makes it easier to hear the other person's voice.

When the overheard conversation ends, the screen returns to the screen shown in FIG. 12(1). Audio will also be transmitted to all conference participants.

As described above, by using the listening function of the present invention, it is possible to have a conversation with only desired users during a meeting.

Next, the processing contents of the present invention will be explained using the flowcharts shown in FIGS. 3 to 7.

FIG. 3 shows a process in which the CPU 201 of the client terminal 101 reads and executes a predetermined control program.

In step S301, it is determined whether an earphone response request has been received from another user. An ear tapping response request is a request for confirmation from other users whether they can respond to ear tapping (a function that delivers audio to the other party in a web conference but does not deliver audio to other participants). This is a request issued by another user operating the client terminal 101 and received via the server device 102.

If the earphone response request has been received (step S301: YES), the process moves to step S302, and if it has not been received, the process moves to step S303.

FIG. 9 shows an example of a screen displayed when user B receives an earphone response request from user A. As shown in FIG. 9, user A's image is displayed in a small size in user B's area. Note that this display screen is just an example, and any form of display may be used as long as it is possible to identify which user has sent the earphone response request to which user.

In step S302, response processing to the earphone response request is performed. Details of the response processing will be explained using the flowchart of FIG.

In step S303, it is determined whether an overheard conversation request has been received. The earphone conversation request is a request to have a conversation using the earphone function, and is a request issued from the client terminal 101 used by another user and received via the server device 102.

In step S304, processing for overhearing conversation (listener) is executed. Details will be explained using the flowchart in FIG.

In step S305, a predetermined selection operation (for example, an operation of continuously clicking or tapping an area where another user is displayed on the screen for a predetermined period of time or more) is performed. This may be an operation of displaying the icon and accepting the selection of the icon.) It is determined whether the icon is accepted. If it is accepted (step S305; YES), the process moves to step S306, and if it is not accepted (step S305: NO), the process of this flowchart ends.

In step S306, it is determined whether the user (client terminal used by the user) associated with the area in which the selection operation was determined to be performed in step S305 has turned on (enabled) the ear tapping function. Information on whether each user (client terminal used by the user) has turned on the listening function is collected and managed by the server device 102 from each client terminal, and is distributed to each client terminal. .

If the user who is the other party to the ear tapping function has turned on the ear tapping function (step S306: YES), the process moves to step S307. If it is not turned on (step S306: NO), the process returns to step S301.

In step S307, an ear tapping response request is sent to the user who is the ear listening partner.

In step S308, a process of waiting for an earphone response is performed. The details of the earphone response waiting process will be explained using the flowchart of FIG.

Next, details of the response process shown in step S302 will be explained using the flowchart of FIG. 4.

First, the screen displayed when an earphone response request is received will be described.

Upon receiving the overhearing response request, the client terminal 101 displays a screen, an example of which is shown in FIG. FIG. 9 is an example of a screen that is displayed when user A transmits a listening response request to user B and user B receives the listening response request. When user B's client terminal accepts a predetermined selection operation for the displayed area with this screen displayed, a response selection interface, an example of which is shown in FIG. 11, is displayed. User B's client terminal responds to the request to user A by accepting a response method selection from user B via this response selection interface.

The response selection interface shown in Figure 11 displays four types of reception sections: "Voice", "Text", "〇", and "×", and by accepting a selection from one of the reception sections, a response to the earphone response request is selected. . The circle in the center of the reception area indicates the location where a click or tap operation was performed. The selection of the reception section can be accepted by performing a flick operation, etc. from the place of the circle mark in the direction of the reception section, but the method of accepting the selection is not limited to the flick operation, and any method can be used. Also good.

The response process will be explained below using FIG. 4.

In step S401, it is determined whether the selection of "sentence" has been received from the receiving section shown in FIG. If the selection is accepted (step S401: YES), the process moves to step S402, and if the selection is not accepted (step S402: NO), the process moves to step S403.

In step S402, a text indicating a written response is sent to the sender of the earphone response request. Note that the response in text means a response to communicate with the user who is the sender of the overhearing response request by text (for example, chat) rather than by voice. If a text response is selected, a chat screen (not shown) for exchange between the sender and destination user of the overhearing request will be displayed.

In step S403, it is determined whether the reception section has accepted the selection of "voice". If the selection of "voice" is accepted (step S403: YES), the process moves to step S404, and if the selection of "voice" is not accepted (step S403: NO), the process moves to step S405. Note that "voice" is a response to the effect that a voice communication will be performed.

In step S404, an ear tapping response is transmitted to the transmission source of the ear tapping response request. Then, this flowchart ends.

In step S405, it is determined whether the selection of "〇" has been accepted by the reception unit. If the selection is accepted (step S405: YES), the process moves to step S406, and if the selection is not accepted (step S405: NO), the process moves to step S407.

Note that "〇" indicates a response to the overhearing.

In step S406, a text indicating that the request will be accepted is sent to the sender of the earphone response request.

When "Sentence", "Voice", and "〇" are selected, a screen, an example of which is shown in FIG. 10, is displayed on the client terminal 101, indicating that the ear tapping function is being used in the user AB building.

In step S407, the reception unit determines whether the selection of "x" has been accepted. If the selection of "x" is accepted (step S407: YES), the process proceeds to step S408, and if not accepted (step S407: NO), the process returns to step S401.

Note that "x" is a response to the effect that the user cannot respond to the overhearing.

In step S408, a text is sent to the sender of the earphone response request to the effect that the request cannot be responded to.

In step S409, a simple response is transmitted to the sender of the earphone response request.

Through the above process, the user who has received the overhearing response request is able to tell the other user how he/she will respond/can respond.

Next, the process of overhearing conversation (listener) will be explained using the flowchart of FIG.

Although details will be described later, when the flowchart in FIG. 5 is executed, a screen shown in FIG. 10 is displayed on the display unit of the client terminal 101. In other words, the images of the users who perform ear tapping (user A and user B in FIG. 10) are displayed in the display area of user B (displayed in an evenly divided manner), and the information identifying user A is displayed in the display area of user A. (name, etc.) will be displayed.

In step S501, it is determined whether the user of the listening partner (speaker) has performed a predetermined operation such as a click operation or a tap operation on the displayed area.

If the predetermined operation has been performed (step S501: YES), the process proceeds to step S501, and if the predetermined operation has not been performed (step S501: NO), the process proceeds to step S503.

In step S502, the volume of the voices of users (conference participants) other than the user who is listening to the ear is lowered to a set value (set to a predetermined value). Then, the process returns to step S501. That is, while a predetermined operation is being performed, the volume of the voices of users (conference participants) other than the user who is listening to the other party is lowered to the set value.

In step S503, the volume of the voices of users (conference participants) other than the user who is listening to the other party is set to a default value (a value set at each client terminal).

In step S504, it is determined whether an end quest has been received from the other user. If received (step S504: YES), this flowchart ends, and if not received (step S504: NO), the process moves to step S505.

In step S505, it is determined whether a predetermined operation such as a double-click operation or a double-tap operation (an operation to end the ear-talking) has been performed on the area where the ear-talking partner user is displayed.

If the predetermined operation has been performed (step S505; YES), the process moves to step S506, and if the predetermined operation has not been performed (step S505: NO), the process returns to step S501.

In step S506, a request to end the ear tapping function is sent to the user who is the ear listening partner.

Next, the response waiting process in step S308 will be explained using the flowchart in FIG.

In step S601, the image of the user who sent the earphone response request is reduced in size and displayed superimposed on the image of the user who is the earphone user.

In step S602, the name of the user (information identifying the user such as name) is displayed in the area of the user who sent the earphone response request.

Through the processing in steps S601 and S602, a screen, an example of which is shown in FIG. 9, is displayed.

In step S603, the voice of the user who sent the ear tapping response request is set so that only the other user can hear the voice of the user who transmitted the ear tapping response request. Note that this process is a process to control the audio from being heard by conference participants other than the user who is listening to the audio, and is not limited to controlling to completely cut off the audio, but to a level that is inaudible to human ears. It also includes processing to reduce the volume.

In step S604, it is determined whether an ear tapping response response has been received from the other user. If received (step S604; YES), the process moves to step S607; if not received (step S604: NO), the process moves to step S605.

In step S605, it is determined whether a simple response has been received from the user who is listening to the other party. If received (step S605: YES), the process moves to step S609, and if not received (step S605: NO), the process moves to step S606.

In step S606, it is determined whether a predetermined operation (such as a double-click operation or a double-tap operation) has been received in the area of the user who is listening to the user. If the predetermined operation is accepted (step S606: YES), the process moves to step S609, and if not accepted (step S606: NO), the process returns to step S604.

In step S607, a listening conversation request is sent to the user who is the listening partner.

In step S608, processing of the overheard conversation (speaker) is executed. The details of this process will be explained using the flowchart of FIG.

In step S609, the reduced display executed in step S601 is canceled, and in step S610, the user's own image is displayed in the user's own area. This process causes the screen shown in FIG. 8 to be displayed on the client terminal.

In step S611, control is performed so that the human voice can be heard by all conference participants.

Through the above processing, when the ear tapping function is started, the voice of the speaker user can only be heard by the other party. Furthermore, the conference screen is displayed so that it is possible to identify which user is listening.

Next, the processing of the overheard conversation (speaker) in step S608 will be explained using the flowchart of FIG.

In step S701, as shown in FIG. 10, images of both the listener and the speaker are displayed in the listener user's area. At this time, by using a display format in which the listener user's area is equally divided and displayed as shown in FIG. 10, it is possible to distinguish it from the response waiting state shown in FIG.

In step S702, settings are made so that only the user can hear the voice of the other user (listener user).

In step S703, it is determined whether a selection by a predetermined operation (click operation, tapping, etc.) is accepted for the area of the user who is the listening partner on the screen. If it is accepted (step S703: YES), the process moves to step S704, and if it is not accepted (step S703: NO), the process moves to step S705.

In step S704, the volume of the voices of users (conference participants) other than the user who is listening to the ear is lowered to a set value (set to a predetermined value). Then, the process returns to step S703. That is, while a predetermined operation is being performed, the volume of the voices of users (conference participants) other than the user who is listening to the other party is lowered to the set value.

In step S705, the volume of the voices of users (conference participants) other than the user who is listening to the other party is set to a default value (a value set at each client terminal).

In step S706, it is determined whether a hunting request has been received from the user who is listening. If received (step S706: YES), the process moves to step S709, and if not received (step S706: NO), the process moves to step S707.

In step S707, it is determined whether selection by a predetermined operation (double-click operation, double-tap operation, etc.) on the area of the earphone partner user on the screen is accepted, that is, whether an operation to send an end request is accepted.

If it is accepted (step S707: YES), the process moves to step S708, and if it is not accepted (step S707: NO), the process moves to step S703.

In step S708, a termination request is sent to the user who is listening.

In step S709, the screen display is returned to the screen shown in FIG.

In step S710, settings are made so that all conference participants can hear the voice of the user who is listening to the other party.

By using the overhearing function described above, it becomes possible to have a conversation with only desired users in a web conference.

The present invention can be implemented as, for example, a system, an apparatus, a method, a program, a recording medium, or the like. Specifically, the present invention may be applied to a system consisting of a plurality of devices, or may be applied to a device consisting of a single device.

Further, the program in the present invention is a program that allows a computer to execute the processing methods shown in the flowcharts shown in FIGS. 3 to 7, and the storage medium of the present invention is a program that allows a computer to execute the processing methods shown in is memorized. Note that the program in the present invention may be a program for each processing method of each device shown in FIGS. 3 to 7.

As described above, a recording medium recording a program that implements the functions of the embodiments described above is supplied to a system or device, and the computer (or CPU or MPU) of the system or device reads the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

In this case, the program itself read from the recording medium will realize the novel function of the present invention, and the recording medium on which the program is recorded constitutes the present invention.

Examples of recording media for supplying programs include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, and silicon A disk or the like can be used.

In addition, by executing a program read by a computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) etc. running on the computer are realized based on the instructions of the program. It goes without saying that this also includes a case where part or all of the processing is performed and the functions of the embodiments described above are realized by the processing.

Furthermore, after the program read from the recording medium is written into the memory of the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board It goes without saying that this also includes a case where a CPU or the like provided in a function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

Moreover, the present invention may be applied to a system made up of a plurality of devices, or to a device made up of one device. It goes without saying that the present invention can also be applied to cases where the present invention is achieved by supplying a program to a system or device. In this case, by reading a recording medium storing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

Further, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. Note that all configurations that are combinations of the above-described embodiments and their modifications are also included in the present invention.

101 Client terminal 102 Server device 110 Network

Claims (11)

In a web conference in which a first user, a second user, and a third user participate,
request receiving means for receiving a request for a specific conversation from a first user to a second user;
selection receiving means for receiving a selection of a response method for the specific conversation from among response methods including voice or text responses from a second user who is the requesting party for the request for the specific conversation received by the request receiving section; ,
When a voice response is accepted by the selection acceptance means, the first user and the second user are enabled to talk to each other, and the voice of the third user is distributed to the first user and the second user. , a control means for controlling the distribution of the voices of the first user and the second user to the third user ;
An information processing system comprising: When the control means receives a predetermined operation from the first user or the second user, the control means lowers the volume of the third user's voice and provides the control means to the first user and the second user. The information processing system according to claim 1 , characterized in that the information processing system is distributed to. comprising display control means for controlling display of images of users participating in the web conference on information processing devices used by each user;
3. The information processing system according to claim 1, wherein the display control means controls to display the first user and the second user in a distinguishable manner. When the display control means receives a request for a specific conversation from the first user to the second user, the display control means reduces the size of the first user and displays the first user within the display area of the second user. and while the second user's specific conversation is in progress, the second user's display area is divided to display the first user and the second user together. The information processing system described in . When the display control means receives a request for a specific conversation from the first user to the second user, the display control means does not display the image of the first user in the display area of the first user, and displays the image of the first user. 5. The information processing system according to claim 4, wherein information for identifying the first user is displayed in the display area of the information processing system. When receiving a selection of a response method for a specific conversation, including a voice or text response, from the second user who is the request destination for the specific conversation request received by the request receiving device, the display control device selects a response method. The information processing system according to any one of claims 3 to 5, wherein the information processing system is controlled to display the options on the second user's screen in a discernible manner. 7. The information processing system according to claim 6, wherein the options further include an option indicating that the specific conversation request cannot be met. The information processing system according to claim 6 or 7, wherein the options are displayed around a position where the user performs a touch operation on the screen, and the selection of a response method is accepted by a flick operation by the user. . 3. The display control means controls to display a chat screen with the first user and the second user when the selection reception means accepts the selection of a text response. 8. The information processing system according to any one of 8 . In a web conference in which a first user, a second user, and a third user participate,
a request receiving step in which the request receiving means of the information processing system receives a request for a specific conversation from a first user to a second user;
The selection receiving means of the information processing system selects a response method for the specific conversation from among response methods including a voice or text response from the second user who is the requesting party for the specific conversation request received in the request receiving step. a selection reception step of accepting a selection of a response method;
When the control means of the information processing system receives a voice response in the selection reception step, the control means enables the first user and the second user to communicate with each other, and transmits the third user's voice to the first user. a control step of distributing the audio to the user and the second user, and controlling the distribution of the audio of the first user and the second user to the third user;
An information processing method comprising: A program for causing a computer to function as each means according to any one of claims 1 to 9 .

Images