JP2024054871A - Information processing system, information processing method, and program - Google Patents

Abstract

[Problem] To enable conversation with a specific individual in an online conference or the like [Solution] An information processing system comprising: a request receiving means for receiving a request for a whispered conversation with a second user from a first user participating in a Web conference; a selection receiving means for receiving a selection of a response method to the whispered conversation from the second user who is the destination of the request for the whispered conversation received by the request receiving means; and a control means for controlling distribution of the voices of the first user and the second user according to the response method selected and received by the selection receiving means. [Selected Figure] Figure 1

Description

The present invention relates to an information processing system, an information processing method, and a program.

Recently, as the trend towards promoting remote work has accelerated, more meetings are being held online (web conferences) rather than in real, face-to-face spaces. While this type of remote communication has the advantage of eliminating the need to secure a physical conference room, it can also lead to inconveniences that would not occur in face-to-face situations, as participants share the same space and cannot converse face-to-face.

Patent Document 1 discloses a technology that solves one of the issues with remote communication, "collisions of speech." Specifically, the technology describes a technology that reduces collisions of speech by predicting the next speaker and highlighting that speaker.

JP 2017-111643 A

The technology described in Patent Document 1 assumes that comments are shared with all conference participants, but in conversations, the content of what is said is often intended only for specific individuals.

The present invention aims to provide a mechanism that enables dialogue with specific individuals during online conferences, etc.

The information processing system of the present invention is characterized by comprising a request receiving means for receiving a request for a whispered conversation between a first user participating in a Web conference and a second user, a selection receiving means for receiving a selection of a method of responding to the whispered conversation from the second user who is the recipient of the request for the whispered conversation received by the request receiving means, and a control means for controlling the distribution of the voices of the first user and the second user according to the response method selected and received by the selection receiving means.

The present invention makes it possible to have conversations with specific individuals during online meetings, etc.

FIG. 1 is a diagram showing an example of the configuration of an information processing system to which the present invention can be applied. FIG. 1 is a block diagram showing an example of the hardware configuration of an information processing device used as a client terminal 101 or a server device 102. Flowchart showing the processing contents of the present invention Flowchart showing details of the process in step S302 Flowchart showing details of the process in step S304 Flowchart showing details of the process in step S308 Flowchart showing details of the process in step S608 An example of a screen displayed on the client terminal 101 An example of a screen displayed on the client terminal 101 An example of a screen displayed on the client terminal 101 An example of a screen displayed on the client terminal 101 FIG. 1 is a diagram for explaining the outline of the present invention.

The following describes an embodiment of the present invention in detail with reference to the drawings.

Figure 1 shows an example of the configuration of an information processing system to which the present invention can be applied.

As shown in FIG. 1, a client terminal 101 and a server device 102 are connected to each other so as to be able to communicate with each other via a network 110.

The client terminal 101 is a terminal used and operated by users participating in a Web conference or the like, and is a terminal on which the screens shown in Figures 8 to 11 are displayed. In addition, as shown in Figure 1, multiple client terminals 101 are connected via a network 110. This configuration assumes that each user participating in a Web conference will use one terminal. Specific examples of the client terminal 101 include, but are not limited to, personal computers (such as notebook PCs or desktop PCs), tablet terminals, and smartphones.

The server device 102 is a device that controls data exchange between client terminals in the Web conference building. It has the function of receiving user voice data and image data from the client terminal 101 based on user operations on the client terminal 101, and distributing the data to other client terminals. Although only one server device 102 is shown in FIG. 1, the system may be a system in which multiple server devices work together, or a so-called cloud system.

Figure 2 is a block diagram showing an example of the hardware configuration of an information processing device used as the client terminal 101 or server device 102 of the present invention.

As shown in FIG. 2, the information processing device is connected to a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a storage device 204, an input controller 205, an audio controller 206, a video controller 207, a memory controller 208, and a communication I/F controller 209 via a system bus 200.

The CPU 201 provides overall control over each device and controller connected to the system bus 200.

The ROM 202 or external memory 213 holds the BIOS (Basic Input/Output System) and OS (Operating System), which are control programs executed by the CPU 201, computer-readable and executable programs for implementing this information processing method, and various necessary data (including data tables).

RAM 203 functions as the main memory, work area, etc. of CPU 201. CPU 201 loads programs and the like required for executing processing from ROM 202 or external memory 213 into RAM 203, and executes the loaded programs to realize various operations.

The input controller 205 controls input from input devices such as a keyboard 210 and a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger, etc.) icons, cursors, or buttons displayed on the touch panel.

The touch panel may also be a touch panel capable of detecting positions touched by multiple fingers, such as a multi-touch screen.

The video controller 207 controls the display on an external output device such as a display 212. The display also includes the display of a notebook computer that is integrated with the main body. Note that the external output device is not limited to a display, and may be, for example, a projector. In addition, for devices that can accept the aforementioned touch operations, an input device is also provided.

The video controller 207 can control a video memory (VRAM) for display control, and can use part of the RAM 203 as a video memory area, or can provide a separate dedicated video memory.

The memory controller 208 controls access to the external memory 213. The external memory can be an external storage device (hard disk) that stores the boot program, various applications, font data, user files, edit files, and various data, a flexible disk (FD), or a compact flash (registered trademark) memory connected to a PCMCIA card slot via an adapter.

The communication I/F controller 209 connects and communicates with external devices via a network, and executes communication control processing on the network. For example, communication using TCP/IP, telephone lines such as ISDN, and 4G and 5G lines for mobile phones are possible.

The CPU 201 can display the outline font on the display 212 by, for example, executing a process of expanding (rasterizing) the outline font in a display information area in the RAM 203. The CPU 201 can also allow the user to give instructions using a mouse cursor (not shown) on the display 212.

Next, an overview of the present invention will be explained using Figure 12.

When a Web conference is started, a screen including images of the conference participants (images taken by an in-camera, etc.) is displayed on the client terminal 101 of each conference participant, as shown in FIG. 12 (1). FIG. 12 (1) shows that four users, A, B, C, and D, are participating in the conference. Note that it is not necessary to display images of all participants in the conference, and a configuration may be used in which only a predetermined number of people are displayed.

Here, user A selects user B's area with a specific operation (such as clicking or tapping continuously for a certain period of time) to request a whisper with user B. When a whisper request is made, the image of user A is displayed in the area of user B's image on the screen as shown in Figure 12 (2), and information identifying user A (such as name) is displayed in user A's area.

When a whisper is requested, User B selects a response method that corresponds to the request via the response selection interface shown in FIG. 11. When a response that allows the whisper is selected, the screen transitions to that shown in FIG. 12 (3), and the whisper conversation begins.

When a whispered conversation begins, the voices of user A and user B are controlled so that they can only be heard by the person they are whispering to (in this case, user A's voice is only heard by user B, and user B's voice is only heard by user A). Note that the voices of the other conference participants can also be heard by users A and B, but while a specific operation is being performed (such as an operation to select the area displaying the image of the user they are whispering to), the volume of the voices of the other conference participants is controlled to be lowered. This makes it easier to hear the voice of the person they are whispering to.

When the whispered conversation ends, the screen returns to that shown in Figure 12 (1). The audio will now be transmitted to all conference participants.

As described above, by using the whisper function of the present invention, it is possible to talk only with desired users during a conference.

Next, the processing steps of the present invention will be explained using the flowcharts in Figures 3 to 7.

Figure 3 shows the process in which the CPU 201 of the client terminal 101 reads and executes a specific control program.

In step S301, it is determined whether a whisper response request has been received from another user. A whisper response request is a request to another user to confirm whether they can respond to a whisper (a function that delivers audio to the person being whispered to among Web conference participants, but not to other participants), and is a request issued by another user operating the client terminal 101 and received via the server device 102.

If a whisper response request is received (step S301: YES), the process proceeds to step S302; if not, the process proceeds to step S303.

Figure 9 shows an example of a screen that is displayed when user B receives a whisper response request from user A. As shown in Figure 9, a small image of user A is displayed in user B's area. Note that this display screen is just an example, and any form of display may be used as long as it is possible to identify which user sent the whisper response request to which user.

In step S302, a response process is performed to the whisper response request. Details of the response process are explained using the flowchart in FIG. 4.

In step S303, it is determined whether a whisper conversation request has been received. A whisper conversation request is a request to have a conversation using the whisper function, and is a request issued from a client terminal 101 used by another user and received via the server device 102.

In step S304, processing of the whispered conversation (listener) is executed. Details will be explained using the flowchart in FIG. 5.

In step S305, it is determined whether a predetermined selection operation (for example, an operation of clicking or tapping continuously for a predetermined period of time or more on an area displayed by another user on the screen, or an operation of displaying an icon indicating a whisper function and accepting the selection of that icon) has been accepted. If it has been accepted (step S305: YES), the process proceeds to step S306, and if it has not been accepted (step S305: NO), the process of this flowchart ends.

In step S306, it is determined whether the user (client terminal used by the user) associated with the area determined in step S305 to have had a selection operation performed has the whisper function ON (enabled). Information on whether each user (client terminal used by the user) has the whisper function ON is collected from each client terminal, managed by the server device 102, and distributed to each client terminal.

If the user you are whispering to has the whisper function turned on (step S306: YES), the process proceeds to step S307. If the user does not have the whisper function turned on (step S306: NO), the process returns to step S301.

In step S307, a whisper response request is sent to the user to whom the whisper was spoken.

In step S308, a process of waiting for a whispered response is performed. The details of the process of waiting for a whispered response are explained using the flowchart in FIG. 6.

Next, the response process shown in step S302 will be described in detail using the flowchart in FIG.

First, let me explain what appears on the screen when a whisper response request is received.

When the client terminal 101 receives a whisper response request, it displays a screen, an example of which is shown in FIG. 9. FIG. 9 is an example of a screen that is displayed when user A sends a whisper response request to user B and user B receives the whisper response request. When this screen is displayed and user B's client terminal accepts a specified selection operation in the displayed area for user B (user B), a response selection interface, an example of which is shown in FIG. 11, is displayed. User B's client terminal accepts a selection of a response method from user B via this response selection interface, and responds to the request from user A.

The response selection interface in FIG. 11 displays four types of reception areas: "Voice," "Text," "O," and "X." A response to a whispered response request is selected by accepting the selection of one of the reception areas. Note that the circle in the center of the reception area indicates the location where a click or tap operation was performed. The selection of a reception area can be accepted by performing a flick operation or the like from the location of the circle in the direction of the reception area, but the method of accepting the selection is not limited to a flick operation and any method may be used.

The response process is explained below using Figure 4.

In step S401, it is determined whether a selection of "Sentence" has been received from the reception unit shown in FIG. 11. If a selection has been received (step S401: YES), the process proceeds to step S402. If a selection has not been received (step S402: NO), the process proceeds to step S403.

In step S402, text is sent to the sender of the whisper response request to indicate that a text response will be sent. Note that a text response is a response to communicate with the user who sent the whisper response request by text (e.g., chat) rather than by voice. If a text response is selected, a chat screen (not shown) is displayed for communication between the sender and destination users of the whisper request.

In step S403, it is determined whether the selection of "voice" has been received from the reception section. If the selection of "voice" has been received (step S403: YES), the process proceeds to step S404. If the selection of "voice" has not been received (step S403: NO), the process proceeds to step S405. Note that "voice" is a response indicating that communication will be conducted by voice.

In step S404, a whisper response is sent to the sender of the whisper response request. This flowchart then ends.

In step S405, it is determined whether the selection of "O" has been received from the reception section. If a selection has been received (step S405: YES), the process proceeds to step S406. If a selection has not been received (step S405: NO), the process proceeds to step S407.

Note that "〇" is a response indicating that you agree to the whisper.

In step S406, text is sent to the sender of the whisper response request indicating that the request will be responded to.

When "Sentence," "Voice," and "Circle" are selected, the client terminal 101 displays a screen, an example of which is shown in FIG. 10, indicating that the whisper function is being used in User A and B buildings.

In step S407, it is determined whether the selection of "x" has been received from the reception section. If the selection of "x" has been received (step S407: YES), the process proceeds to step S408. If the selection of "x" has not been received (step S407: NO), the process returns to step S401.

Note that "x" is a response that means you cannot respond to a whisper.

In step S408, a text message is sent to the sender of the whisper response request indicating that the request cannot be fulfilled.

In step S409, a simple response is sent to the sender of the whisper response request.

Through the above process, a user who receives a whisper response request can inform the other user how they will/can respond.

Next, we will explain the processing of whispered conversations (listener) using the flowchart in Figure 5.

Although details will be described later, when the flowchart in FIG. 5 is executed, the screen shown in FIG. 10 is displayed on the display unit of the client terminal 101. That is, the image of the user performing the whisper (user A and user B in FIG. 10) is displayed (evenly divided and displayed) in the display area of user B, and information identifying user A (such as name) is displayed in the display area of user A.

In step S501, it is determined whether the user to whom the whisper is being made (the speaker) has performed a predetermined operation, such as a click or tap, on the displayed area.

If the specified operation has been performed (step S501: YES), the process proceeds to step S501. If the specified operation has not been performed (step S501: NO), the process proceeds to step S503.

In step S502, the volume of the voices of users (conference participants) other than the user who is the person being whispered to is lowered to a set value (set to a predetermined value). Then, the process returns to step S501. In other words, while the predetermined operation is being performed, the volume of the voices of users (conference participants) other than the user who is the person being whispered to is lowered to the set value.

In step S503, the voice volume of users (conference participants) other than the user to whom the whisper was made is set to the default value (the value set on each client terminal).

In step S504, it is determined whether a completion quest has been received from the whispered user. If a completion quest has been received (step S504: YES), this flowchart ends, and if a completion quest has not been received (step S504: NO), the process proceeds to step S505.

In step S505, it is determined whether the user to whom you wish to whisper has performed a predetermined operation (an operation to end the whisper), such as a double-click or double-tap, on the displayed area.

If the specified operation has been performed (step S505: YES), the process proceeds to step S506; if not (step S505: NO), the process returns to step S501.

In step S506, a request to end the whisper function is sent to the user to whom the whisper was directed.

Next, the response waiting process in step S308 will be explained using the flowchart in FIG. 6.

In step S601, the image of the user who sent the whisper response request is reduced in size and displayed superimposed on the image of the user who whispered.

In step S602, the name of the user (information that identifies the user, such as a name) is displayed in the area of the user who sent the whisper response request.

The processing of steps S601 and S602 causes a screen, an example of which is shown in Figure 9, to be displayed.

In step S603, the voice of the user who sent the whisper response request is set so that it can only be heard by the user to whom the whisper was made. Note that this process is a process of controlling the voice so that it cannot be heard by conference participants other than the user to whom the whisper was made, and is not limited to controlling the voice to be completely blocked, but also includes a process of reducing the volume to a level that cannot be heard by the human ear.

In step S604, it is determined whether a whisper response has been received from the user who was the whisper destination. If a whisper response has been received (step S604: YES), the process proceeds to step S607. If a whisper response has not been received (step S604: NO), the process proceeds to step S605.

In step S605, it is determined whether a simple response has been received from the whispered user. If a simple response has been received (step S605: YES), the process proceeds to step S609. If a simple response has not been received (step S605: NO), the process proceeds to step S606.

In step S606, it is determined whether a predetermined operation (such as a double-click operation or a double-tap operation) has been received in the whispered user's area. If the predetermined operation has been received (step S606: YES), the process proceeds to step S609. If the operation has not been received (step S606: NO), the process returns to step S604.

In step S607, a whisper conversation request is sent to the user with whom the whisper is intended.

In step S608, the whisper conversation (speaker) process is executed. Details of this process are explained using the flowchart in FIG. 7.

In step S609, the reduced display performed in step S601 is cancelled, and in step S610, the user's own image is displayed in the user's own area. This process results in the screen shown in FIG. 8 being displayed on the client terminal.

In step S611, control is performed so that the human voice can be heard by all conference participants.

By performing the above process, when the whispering function is started, the voice of the speaking user will be heard only by the person being whispered to. Also, the conference screen will display a screen that makes it easy to identify which users are whispering to each other.

Next, the processing of whispered conversation (speaker) in step S608 will be explained using the flowchart in Figure 7.

In step S701, images of both the listener and the speaker are displayed in the listener user's area as shown in FIG. 10. At this time, the listener user's area may be divided equally as shown in FIG. 10, or the like, to make it possible to distinguish the state from the state of waiting for a response as shown in FIG. 9.

In step S702, the user sets the voice of the user he or she is whispering to (the listener user) so that only he or she can hear it.

In step S703, it is determined whether a selection has been accepted by a predetermined operation (such as a click or tap) on the area of the screen of the user to whom the message is to be whispered. If it has been accepted (step S703: YES), the process proceeds to step S704. If it has not been accepted (step S703: NO), the process proceeds to step S705.

In step S704, the voice volume of users (conference participants) other than the user who is the person to whom the whisper is directed is lowered to a set value (set to a predetermined value). Then, the process returns to step S703. In other words, while the predetermined operation is being performed, the voice volume of users (conference participants) other than the user who is the person to whom the whisper is directed is lowered to the set value.

In step S705, the voice volume of users (conference participants) other than the user to whom the whisper was made is set to the default value (the value set on each client terminal).

In step S706, it is determined whether a hunting request has been received from the whispered user. If a hunting request has been received (step S706: YES), the process proceeds to step S709. If a hunting request has not been received (step S706: NO), the process proceeds to step S707.

In step S707, it is determined whether a selection has been made by a predetermined operation (such as a double-click or double-tap) on the area of the screen of the user to whom the whisper is to be made, i.e., whether an operation to send an end request has been accepted.

If it has been accepted (step S707: YES), the process proceeds to step S708; if it has not been accepted (step S707: NO), the process proceeds to step S703.

In step S708, an end request is sent to the user who whispered.

In step S709, the screen display is returned to the screen shown in FIG.

In step S710, the voice of the user being whispered to is set so that it can be heard by all conference participants.

By using the whisper function described above, you can talk only with desired users during a web conference.

The present invention can be embodied, for example, as a system, device, method, program, or recording medium. Specifically, it may be applied to a system made up of multiple devices, or to an apparatus made up of a single device.

The program of the present invention is a program that enables a computer to execute the processing methods of the flowcharts shown in Figures 3 to 7, and the storage medium of the present invention stores a program that enables a computer to execute the processing methods of Figures 3 to 7. The program of the present invention may be a program for each processing method of each device in Figures 3 to 7.

As described above, it goes without saying that the object of the present invention can be achieved by supplying a recording medium on which a program that realizes the functions of the above-mentioned embodiments is recorded to a system or device, and having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

In this case, the program read from the recording medium itself realizes the novel functions of the present invention, and the recording medium on which the program is recorded constitutes the present invention.

Recording media for supplying the program may include, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a DVD-ROM, a magnetic tape, a non-volatile memory card, a ROM, an EEPROM, a silicon disk, etc.

Furthermore, it goes without saying that not only are the functions of the above-mentioned embodiments realized by the computer executing a program it has read, but also that the functions of the above-mentioned embodiments can be realized by an operating system (OS) or the like running on the computer carrying out some or all of the actual processing based on the instructions of the program.

Furthermore, it goes without saying that this also includes cases where a program read from a recording medium is written into a memory provided on a function expansion board inserted into a computer or a function expansion unit connected to a computer, and then a CPU or the like provided on the function expansion board or function expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.

The present invention may be applied to a system made up of multiple devices, or to a device made up of a single device. Needless to say, the present invention can also be applied to cases where the effects of the present invention are achieved by supplying a program to a system or device. In this case, the effects of the present invention can be enjoyed by reading a recording medium that stores a program for achieving the present invention into the system or device.

Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. Note that the present invention also includes configurations that combine the above-mentioned embodiments and their modified examples.

101 Client terminal 102 Server device 110 Network

Claims (10)

a request receiving means for receiving a request for whisper conversation from a first user participating in a Web conference with a second user;
a selection receiving means for receiving, from the second user who is the destination of the request for the whisper conversation received by the request receiving means, a selection of a method for responding to the whisper conversation;
a control means for controlling distribution of voices of the first user and the second user in accordance with the response method selected by the selection receiving means;
An information processing system comprising: The information processing system according to claim 1, characterized in that the selection receiving means receives a selection from among response methods including a voice response or a text response. The information processing system according to claim 2, characterized in that the control means controls the distribution of the voices of the first user and the second user when a voice response is received by the selection receiving means. The information processing system according to any one of claims 1 to 3, characterized in that the control means controls so that the voices of the first user and the second user are not distributed to conference participants other than the first user and the second user. The information processing system according to any one of claims 1 to 4, characterized in that the control means, when receiving a predetermined operation from the first user and the second user, controls the distribution of the voices of conference participants other than the first user and the second user to the first user and the second user. The information processing system according to any one of claims 1 to 5, characterized in that, when the control means receives a predetermined operation from the first user and the second user, it lowers the volume of the voices of the conference participants other than the first user and the second user and distributes the voices to the first user and the second user. A display control means is provided for controlling the display of images of the Web conference participants on the information processing devices used by each of the participants,
7. The information processing system according to claim 1, wherein the display control means controls so that the user having the whispered conversation is displayed in an identifiable manner. The information processing system according to claim 7, characterized in that the display control means controls to display a chat screen with the first user and the second user when the selection receiving means receives a selection of a text response. A request receiving step in which a request receiving means of the information processing system receives a request for whisper conversation between a first user participating in the Web conference and a second user;
a selection receiving step in which a selection receiving means of the information processing system receives, from the second user who is the request recipient of the request for the whispered conversation received in the request receiving step, a selection of a method of responding to the whispered conversation;
a control step of controlling distribution of voices of the first user and the second user by a control means of the information processing system in accordance with the response method selected in the selection receiving step;
An information processing method comprising: A program for causing a computer to function as each of the means recited in claims 1 to 8.

Images