JP2022041368A - Communication system, user terminal, program, and method - Google Patents

Abstract

To support the smooth start of communication.SOLUTION: In a communication system including a first user terminal used by a first user, and one or more second user terminals used by one or more second users, the first user terminal includes an operation unit that detects an operation for setting a sharing level for each of the one or more second users, and a display control unit that generates a display screen in which a screen element showing the status of the second user is arranged in a manner corresponding to the sharing level set for the second user for each of the second users.SELECTED DRAWING: Figure 2

Description

The present invention relates to communication systems, user terminals, programs and methods.

Recently, online conferencing systems have become widespread as a means for users belonging to organizations such as companies to share the situation online and communicate with each other.

For example, Patent Document 1 discloses an online conference system including a terminal used by a user and a server that manages reservation, progress, and task processing of an online conference. In the management of reservations, the organizer of the online conference accesses the server using the terminal and registers the reservation of the conference. The server sends a conference notification to the email addresses of prospective participants. The server estimates the degree of relationships within the organization based on the frequency of communication exchanged at the meeting.

Further, in Patent Document 2, words and actions and scenes (agenda, meeting type, etc.) are stored in association with each other based on the minutes data of the meeting, and in each scene based on the words and actions and scenes stored in association with each other. A method of estimating the degree of human relations is disclosed.

Japanese Patent Application Laid-Open No. 2003-153224 Japanese Unexamined Patent Publication No. 2013-182560

However, in the online conference system described in Patent Document 1 above, whether or not the user who intends to host the conference is talking to someone about the situation of the other party (whether he / she is busy, whether he / she is concentrated, or not). , Whether or not you are in a bad mood, etc.), so there is concern that you will hesitate to hold a meeting. This concern cannot be resolved by the method described in Patent Document 2.

Therefore, the present invention has been made in view of the above problems, and an object of the present invention is a new and improved communication system and a user terminal capable of supporting a smooth start of communication. , To provide programs and methods.

In order to solve the above problems, according to a certain aspect of the present invention, a first user terminal used by a first user, or one or two or more used by one or more second users. A communication system having a second user terminal, wherein the first user terminal includes an operation unit for detecting an operation of setting a sharing level for each of the one or more second users. For each of the second users, a display screen is generated in which screen elements showing the status of the second user are arranged in an manner corresponding to the sharing level set for the second user. A communication system including a display control unit is provided.

The display screen includes a movable display corresponding to each of the second users, and the operation unit detects an operation of moving the movable display as an operation of setting the sharing level, and the second operation unit. The sharing level set for the user may change according to the movement of the movable display.

In the display screen, the screen element for the second user may be arranged in the movable display corresponding to the second user.

In the display screen, the screen element for the second user may be arranged in a screen area different from the movable display corresponding to the second user.

When the first sharing level is set for the second user, the image of the second user is arranged on the display screen as a screen element showing the situation of the second user, and the display screen is described. When the first user can speak to the second user and the second sharing level is set for the second user, the situation of the second user is displayed. As a screen element to indicate, a text indicating the situation of the second user may be arranged on the display screen.

When the sharing level set from the first user to a second user and the sharing level set from the second user to the first user are different, both It may be provided with a sharing level setting unit for matching the sharing levels of.

Sharing level setting that limits the sharing level that can be set for each of the first users when a predetermined operation is detected by the operation unit of the first user terminal. It may be provided with a part.

When the first user terminal sets a sharing level at which the first user and the second user can communicate with each other, the operation unit detects the conference start operation. And, the conference unit which controls the activation of the conference system in which the first user and the second user participate may be further provided.

Based on the information indicating the operation for each of the first user terminal and the one or more second user terminals, the first user and the one or more second users It may be provided with a human relationship estimation unit that estimates a human relationship between a plurality of users.

The human relations estimation unit is a shared level history set by each of the plurality of users for other users, a user designates a specific other user, and the other user specifies the other user. Estimate the relationships between the plurality of users based on at least one of the restrictions on the sharing level that can be set for the users or the combination of users who participated in the conference system that was held. You may.

A sharing level setting unit that changes the sharing level set from the first user to the second user to a higher sharing level may be provided.

A user terminal that operates as a first user terminal used by a first user, and an operation unit that detects an operation for setting a sharing level for each of one or more second users, and an operation unit. For each of the second users, a display screen is generated in which screen elements showing the status of the second user are arranged in an manner corresponding to the sharing level set for the second user. A user terminal including a display control unit is provided.

Further, in order to solve the above problems, according to another viewpoint of the present invention, the computer is a user terminal that operates as a first user terminal used by the first user, and is 1 or 2. The operation unit for detecting the operation of setting the sharing level for each of the above second users and the sharing level set for the second user for each of the second users correspond to the operation unit. Provided is a program for functioning as a user terminal, comprising a display control unit for generating a display screen in which a screen element indicating the situation of the second user is arranged in an embodiment.

Further, in order to solve the above-mentioned problems, according to another aspect of the present invention, it is a method executed by a user terminal operating as a first user terminal used by a first user. Or, by detecting an operation of setting a sharing level for each of two or more second users, and depending on the sharing level set for the second user for each of the second users. Provided is a method comprising generating a display screen in which a screen element indicating the situation of the second user is arranged in the above-described embodiment.

According to the present invention described above, it is possible to support the smooth start of communication.

It is explanatory drawing which shows the communication system by one Embodiment of this invention. It is explanatory drawing which shows the functional structure of the communication system by one Embodiment of this invention. It is a flowchart which shows the whole operation of the communication system by one Embodiment of this invention. It is explanatory drawing which shows the specific example of the virtual space screen. It is explanatory drawing which shows the specific example of the space operation in the icon operation area 64. It is explanatory drawing which shows the specific example of the space operation in the icon operation area 64. It is explanatory drawing which shows the specific example of the screen element arranged in an icon. It is explanatory drawing which shows the specific example of the screen element arranged in an icon. It is explanatory drawing which shows the specific example of the screen element arranged in an icon. It is explanatory drawing which shows the example of the icon movement for the start of a conference. It is explanatory drawing which shows the activation of a conference system. It is explanatory drawing which shows the example of the icon movement for adding a participant to a meeting. It is explanatory drawing which shows the addition of the participant to the conference. It is explanatory drawing which shows the screen example after the end of a meeting. It is explanatory drawing which shows the estimation result example of the human relationship between four users (user 1, user 2, user 3, user 4). It is explanatory drawing which shows the estimation result example of the human relationship between four users (user 1, user 2, user 3, user 4). It is a block diagram which showed the hardware composition of a user terminal 20.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different alphabets or numbers after the same reference numerals. For example, a plurality of configurations having substantially the same functional configuration or logical significance are distinguished as necessary, such as user terminals 20-1, 20-2, and 20-3. However, when it is not necessary to particularly distinguish each of the plurality of components having substantially the same functional configuration, only the same reference numerals are given to each of the plurality of components. For example, when it is not necessary to distinguish between the user terminals 20-1, 20-2 and 20-3, each user terminal is simply referred to as a user terminal 20.

<1. Overview of communication system>
One embodiment of the present invention relates to a communication system in which a plurality of user terminals communicate, and particularly to a communication system in which a plurality of user terminals communicate for communication between users. Hereinafter, the outline of the communication system according to the embodiment of the present invention will be described with reference to FIG. In the following, an example in which the communication system according to the embodiment of the present invention is applied to communication in a company will be mainly described, but the communication system according to the embodiment of the present invention may be applied to other uses.

FIG. 1 is an explanatory diagram showing a communication system according to an embodiment of the present invention. As shown in FIG. 1, the communication system according to the embodiment of the present invention includes a plurality of user terminals 20, a plurality of bird's-eye views cameras 30, a server 40, and a human relations display terminal 50.

The plurality of user terminals 20, the plurality of bird's-eye cameras 30, and the human relations display terminal 50 are connected to the server 40 via the network 12. The network 12 is a wired or wireless transmission path for information transmitted from a device connected to the network 12. For example, the network 12 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various LANs (Local Area Network) including Ethernet (registered trademark), and a WAN (Wide Area Network). Further, the network 12 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

(User terminal 20)
The user terminal 20 is an information terminal used by the user. FIG. 1 shows a user terminal 20-1 used by a user 1, a user terminal 20-2 used by a user 2, and a user terminal 20-3 used by a user 3. There is. Each user may be located at a remote location. For example, the user 1 may be located at a different base from the user 2 and the user 3. User 1 is an example of a first user, and user terminal 20-1 is an example of a first user terminal. User 2 and user 3 which are other users are examples of the second user, and user terminal 20-2 and user terminal 20-3 are examples of the second user terminal.

According to one embodiment of the present invention, even when a plurality of users are located at remote locations, the functions of the user terminal 20 and the server 40, which will be described later, facilitate smooth operation among the plurality of users. It is possible to realize good communication.

Although a notebook PC (Personal Computer) is shown as an example of the user terminal 20 in FIG. 1, the user terminal 20 may be another information terminal such as a desktop PC, a smartphone, or a tablet terminal.

(Overhead camera 30)
The bird's-eye view camera 30 is an imaging device that acquires bird's-eye view video data by imaging. For example, the bird's-eye view camera 30-1 is installed at a base where the user 1 is located, and acquires the bird's-eye view video data including the video of the user 1. Further, the bird's-eye view camera 30-2 is installed at a base where the user 2 and the user 3 are located, and acquires the bird's-eye view image data including the image of the user 2 and the image of the user 3. Each bird's-eye view camera 30 transmits the acquired bird's-eye view video data to the server 40 via the network 12.

(Server 40)
The server 40 collects media data indicating the situation of each user from the user terminal 20 and the bird's-eye view camera 30 of each user, and transmits the media data to the user terminal 20 of each user. The media data includes, for example, the user's video data captured by the user terminal 20, the audio data picked up by the user terminal 20, the presence data showing the user's situation in text, and the bird's-eye view camera 30. The bird's-eye view video data that has been created can be mentioned. The server 40 according to the embodiment of the present invention uses the user terminal 20 of these media data according to the sharing level set by a certain user for another user. Send to.

Further, the server 40 has a function of estimating a human relationship between a plurality of users based on an operation for each user terminal 20 or information indicating a usage status of each user terminal 20.

(Human relationship display terminal 50)
The human relationship display terminal 50 is a display terminal that receives information indicating a human relationship between a plurality of users from the server 40 and displays a screen showing the human relationship between the plurality of users.

(Arrangement of issues)
As a first technique related to an embodiment of the present invention, an online conference system including a terminal used by a user and a server for managing reservation, progress, and task processing of an online conference can be considered. In the management of reservations, the organizer of the online conference accesses the server using the terminal and registers the reservation of the conference. The server sends a conference notification to the email addresses of prospective participants. The server estimates the degree of relationships within the organization based on the frequency of communication exchanged at the meeting.

Further, as a second technique related to one embodiment of the present invention, words and actions and scenes (agenda, meeting type, etc.) are stored in association with each other based on the minutes data of the meeting, and the words and actions stored in association with each other. And a method of estimating the degree of human relations in each scene based on the scene can be considered.

However, in the above-mentioned first technology, it is premised that the conference is reserved, and when the user suddenly wants to hold the conference instantly, even if the reservation is made on the spot, the other party immediately notices the conference holding notification. There is no problem. In addition, it is difficult for the user to grasp the situation of the other party (whether he / she is busy, whether he / she is concentrated, whether he / she is talking to someone, whether he / she is in a bad mood, etc.). In addition, there is concern that hesitating to hold a meeting.

In the second technique described above, in order to estimate the degree of human relations, words and actions are analyzed, and for each statement content, the statement level (a numerical value indicating whether it is positive or negative) and from whom to whom. It is necessary to estimate both whether it is a statement or not, and there is a problem that the estimation accuracy cannot always be ensured.

The inventor of the present invention has come to create an embodiment of the present invention with the above circumstances as a point of view. One embodiment of the present invention can improve the problem of the first technique and the problem of the second technique. That is, according to one embodiment of the present invention, it is possible to visualize the situation of the other party and smoothly start sudden communication. Further, according to one embodiment of the present invention, it is possible to estimate the human relationship between a plurality of users with high accuracy. Hereinafter, the configuration and operation of the communication system according to the embodiment of the present invention will be sequentially described in detail.

<2. Configuration>
FIG. 2 is an explanatory diagram showing a functional configuration of a communication system according to an embodiment of the present invention.

(Configuration of user terminal 20)
As shown in FIG. 2, the user terminal 20 according to the embodiment of the present invention has an operation unit 220, a media input unit 230, a media reproduction unit 240, and a conference unit 250.

The operation unit 220 detects an operation performed by the user. For example, the operation unit 220 is a setting operation in which a user sets a sharing level described later for another user, and a predetermined operation for limiting a sharing level that another user can set for the user. , And the conference start operation to start the conference using the conference system. The operation unit 220 outputs spatial operation data indicating these detected operations to the server 40.

The media input unit 230 outputs media data corresponding to the media input parameters input from the server 40 to the server 40. The media input parameter is attribute data indicating at least one of video data, audio data, and presence data.

When the media input parameter indicates video data, the media input unit 230 outputs the user's video data captured by the user terminal 20 to the server 40. When the media input parameter indicates voice data, the media input unit 230 outputs the user's voice data picked up by the user terminal 20 to the server 40. When the media input parameter indicates presence data, the media input unit 230 outputs the user's presence data estimated by the user terminal 20 to the server 40. The media input parameter input to the media input unit 230 of all user terminals 20 may be at least attribute data indicating voice data for voice calling.

The user's presence data is useful data for determining whether or not it is possible to speak to the user. For example, the user terminal 20 determines whether or not there is a user in front of the user terminal 20 by using an image obtained by the built-in camera of the user terminal 20. The user terminal 20 determines whether or not the user is busy based on the operation amount of the user terminal 20 and the schedule information (acquired from an external system). The user terminal 20 measures the amount of conversation using the voice of the built-in microphone of the user terminal 20, and determines whether or not the user is in a conversation. The user terminal 20 measures the visual dose to the terminal based on the image of the built-in camera, and whether or not the user is concentrated (concentration / relaxation) based on the measurement result of the visual dose and the measurement result of the conversation amount. ) Is determined. Then, the user terminal 20 determines whether or not it is possible to speak to the user based on the above-mentioned various determination results. The user's presence data may be data showing the result of such a determination.

The media reproduction unit 240 is an example of a display control unit that generates a display screen displayed by the display unit of the user terminal 20. For example, the media reproduction unit 240 reproduces the media data about another user input from the server 40, and generates a display screen in which the reproduced media data is arranged. The display screen corresponds to, for example, a virtual space screen described later.

The conference unit 250 controls the activation of the conference system based on the detection of the conference start operation by the operation unit 220.

(Configuration of server 40)
As shown in FIG. 2, the server 40 according to the embodiment of the present invention has a control unit 420 and a human relationship estimation unit 430.

The control unit 420 is a shared level setting unit that sets a shared level between users based on spatial operation data input from each user terminal 20, outputs media input parameters to each user terminal 20, and each. It has functions such as a media control unit that controls the output of media data to the user terminal 20 and a conference control unit that outputs conference parameters to the user terminal 20 of the user who executes the conference. Further, the control unit 420 outputs the spatial operation data input from each user terminal 20 to the human relationship estimation unit 430.

The human relationship estimation unit 430 estimates the human relationship between a plurality of users based on the spatial operation data input from the control unit 420, and outputs the estimation result to the human relationship display terminal 50.

<3. Operation>
Subsequently, the operation of the communication system according to the embodiment of the present invention will be described. First, the processing of each step constituting the overall operation is briefly organized with reference to FIG. 3, and then the processing of each step will be described in detail with reference to the following drawings.

FIG. 3 is a flowchart showing the overall operation of the communication system according to the embodiment of the present invention. As shown in FIG. 3, the overall operation of the communication system according to the embodiment of the present invention has the following steps. In the following, the description will be given focusing on the user terminal 20-1 used by the user 1.
S1: Virtual space generation S2: Spatial operation detection (movement designation, conference designation)
S3: Determination of media input parameters S4: Input of media data S5: Playback of media data S6: Control of conference S7: Estimate of human relations S8: Display of human relations S9: End judgment

(S1: Virtual space generation)
When the user 1 instructs the display of the virtual space screen (display screen) by operating the operation unit 220 of the user terminal 20-1, the server 40 is based on the follow-up list described later stored for each user. Generates a virtual space screen and sends a virtual space image to the user terminal 20-1. The media playback unit 240 of the user terminal 20-1 displays the virtual space screen on the display unit.

FIG. 4 is an explanatory diagram showing a specific example of the virtual space screen. As shown in FIG. 4, the virtual space screen includes a follower list area 61, a follower list area 62, a user search area 63, and an icon operation area 64.

The followy list area 61 is an area in which a followy list, which is a list of other users of interest to the user 1, is placed. The follower list area 62 is an area in which a follower list, which is a list of other users who are interested in the user 1, is arranged. The user search area 63 is a search area for adding other users currently not included in the followy list to the followy list. The icon operation area 64 is an area in which the icon of one user and the icons of other users included in the follow list are arranged. By operating the operation unit 220, the user 1 can move the icons of other users arranged in the icon operation area 64. That is, the icon is an example of a movable display.

When the user 1 uses this system for the first time, the corresponding person does not exist in the follow list. When the user 1 searches for another user from the user search area 63 and adds it to the follow list, the icon of the other user is displayed in the icon operation area 64.

Further, when the user 1 uses this system for the second time or later, the screen is restarted from the previous state. In the icon operation area 64, the icon of the user 1 (icon 1 in the figure) is fixedly arranged in the center, and the icons of other users are arranged around the icon operation area 64. User 1 can move and operate the icons of other users. In addition, the user 1 can also perform an operation of enlarging or reducing the inside of the icon operation area 64.

In FIG. 4, three concentric areas are shown centered on the icon 1 of the user 1. Different shared levels (level 0, level 1, level 2, level 3 in order from the inside) are defined in the areas surrounded by adjacent concentric circles. The sharing level can be said to be the level of means for user 1 to confirm the situation of other users and to communicate with each other. In one embodiment of the present invention, the smaller the value of the sharing level, the higher the sharing level, that is, the close communication means.

Further, the control unit 420 of the server 40 manages whether or not another user included in the follower list is viewing the video of the user 1. Whether or not another user included in the follower list is viewing the video of the user 1 can be determined by displaying the icon of the other user. For example, as shown in FIG. 4, the icon 2 of the user 2 who has not viewed the video of the user 1 and the icon 3 of the user 3 who has not viewed the video of the user 1 are displayed in different modes. Will be done.

Further, the control unit 420 of the server 40 manages whether or not another user included in the follower list is speaking to the user 1. Whether or not another user included in the follower list is speaking to the user 1 can be determined by displaying the icon of the other user. For example, as shown in FIG. 4, the icon 4 of the user 4 who is speaking to the user 1 is displayed in a different manner from the icons of other users.

It should be noted that the above-mentioned viewing of the video of the user 1, the voice to the user 1, the display content in the icon, etc. are described in detail in "S4: Input of media data" and "S5: Reproduction of media data". explain.

(S2: Spatial operation detection (movement designation, conference designation))
When the user 1 performs a move operation for moving the icon of another user within the icon operation area 64 displayed on the user terminal 20-1, the operation unit 220 moves the shared level to the other user to the destination. The move operation is detected as an operation for setting / changing to the shared level corresponding to the area of.

Further, when the user 1 performs an operation of selecting the icon 1 of the user 1 in the icon operation area 64, the operation unit 220 detects the operation as a barrier setting operation for all other users. The barrier setting is a setting that restricts other users from viewing the video data of the user 1 and other users from speaking to the user 1. User 1 can set the limit level and the time length.

When the limit level is set to 0, other users cannot place the icon of the user 1 in the area of the level 1 in the icon operation area 64 displayed on the user terminal 20 of the user, and the user cannot be placed. Voice to 1 is restricted. When the limit level is set to 1, other users cannot place the user 1 in the level 0 and level 1 areas in the icon operation area 64 displayed on their own user terminal 20, and use the service. Viewing of the video of the person 1 and speaking to the user 1 are restricted. When the restriction level is set to 2, other users can place the user 1 in the level 0 and level 1 and level 2 areas in the icon operation area 64 displayed on their user terminal 20. All information sharing, including viewing presence information about user 1, will be restricted.

The operation unit 220 of the user terminal 20-1 transmits the above-mentioned icon movement information and barrier setting information to the server 40 as spatial operation data. The control unit 420 of the server 40 collects barrier setting information from all user terminals 20, and outputs these as barrier setting data to the operation units 220 of all user terminals 20. The operation unit 220 of each user terminal 20 restricts the movement operation of the icon of another user based on the barrier setting data input from the control unit 420 of the server 40.

5A and 5B are explanatory views showing specific examples of spatial operations in the icon operation area 64. FIG. 5A shows an example in which the icon located in the area of level 2 (position X) is moved to the area of level 0 (position Y). As will be described later, the voice data of the user 1 will be input from the user terminal 20-1 to the user terminal 20 of another user corresponding to the icon moved to the level 0 area. , User 1 can speak to other users corresponding to the icon moved to the level 0 area.

On the other hand, FIG. 5B shows an example in which a barrier area is set for an icon located in the level 2 (position X) area. As shown in FIG. 5B, when the barrier area is set for the icon located in the level 2 (position X) area, the user 1 cannot move the icon to the level 0 area. The icon is forcibly moved to the level 1 (position Z) area.

(S3: Determination of media input parameters)
When the spatial operation data related to the movement of the icon is input from the operation unit 220 of the user terminal 20-1, the control unit 420 of the server 40 uses another use corresponding to the icon at the level of the area including the icon after the movement. Set the sharing level of the person. Then, the control unit 420 determines the attribute data of the media data acquired from the user terminal 20 of the other user according to the sharing level set for the other user, and mediates the determined attribute data. It is output to the user terminal 20 of the other user as an input parameter.

For example, when the sharing level of the other user is level 0 or level 1, the control unit 420 uses the attribute data including the parameters of the video data and the presence data as the media input parameter. Further, when the sharing level of the other user is level 2, the control unit 420 uses the attribute data consisting only of the presence data parameters as the media input parameter. In one embodiment of the present invention, all media input parameters may include voice data parameters for voice calling.

(S4: Media data input)
The media input unit 230 of each user terminal 20 generates only the data indicated by the media input parameters based on the media input parameters input from the server 40, and transmits the data to the server 40.

For example, when the media input parameter includes the parameter of the video data, the media input unit 230 simply uses the coded data as the coded data of the video data captured by the built-in camera of the user terminal 20 (in the present specification, the coded data is simply video. Sometimes referred to as data.) Is transmitted.

When the media input parameter includes the parameter of voice data, the media input unit 230 simply uses the coded data of the voice data picked up by the built-in microphone of the user terminal 20 (in the present specification, the coded data is simply used. It may also be called voice data).

When the media input parameter includes the presence data parameter, the media input unit 230 transmits the user's presence data. As described above, the user presence data is useful data for determining whether or not it is possible to speak to the user.

Further, each bird's-eye view camera 30 transmits the coded data of the bird's-eye view video data acquired by imaging (in the present specification, the coded data may be simply referred to as the bird's-eye view video data) to the server 40.

(S5: Playback of media data)
The control unit 420 of the server 40 collects media data corresponding to the sharing level of each icon indicated by the spatial operation data input from the user terminal 20-1 from the media data collected from each user terminal 20 and the bird's-eye view camera 30. Select from the collected bird's-eye view video data. Then, the control unit 420 outputs the selected media data to the user terminal 20.

The media reproduction unit 240 of the user terminal 20 reproduces / decodes the media data for each other user input from the server 40, and the obtained screen element corresponds to the icon operation area 64 of the virtual space screen. Place it inside the icon. It can be said that the screen element for another user is a screen element showing the situation of another user in a manner corresponding to the sharing level set for the other user. Hereinafter, a specific example of the screen element will be described.

6A to 6C are explanatory views showing specific examples of screen elements arranged in the icon. FIG. 6A shows the icons of other users whose sharing level is set to level 0. As shown in FIG. 6A, the screen element placed on the icon of another user whose sharing level is set to level 0 is the video data of the other user in addition to the name of the other user. 71, and the presence data 72 of the other user. The video data 71 is video data captured by the built-in camera of the user terminal 20 of another user, and the facial expressions of the other users can be confirmed in detail.

FIG. 6B shows the icons of other users whose sharing level is set to level 1. As shown in FIG. 6B, the screen element placed on the icon of another user whose sharing level is set to level 1 shows the image of the other user in addition to the name of the other user. Includes bird's-eye view video data 73 and presence data 72 of the other user. In the bird's-eye view video data 73, it is possible to confirm the approximate operation of other users.

FIG. 6C shows the icons of other users whose sharing level is set to level 2. As shown in FIG. 6C, the screen element arranged on the icon of another user whose sharing level is set to level 2 includes the presence data 74 and does not include the video data.

Since there is no information to be shared with other users whose sharing level is set to level 3, the icons of other users whose sharing level is set to level 3 are not displayed in the icon operation area 64.

When there is another user whose sharing level is set to level 0 from the user 1, the control unit 420 of the server 40 transfers the voice data of the user 1 to the user terminal 20 of the other user. Output the included media data. As a result, the user 1 can speak to other users whose sharing level is set to level 0 by voice.

(S2: Spatial operation detection (conference designation))
The processing when the user performs an operation to move another user's icon in the virtual space screen has already been described. Here, the operation for the virtual space screen when the user suddenly holds a meeting will be described with reference to the drawings.

(A) Confirmation of intention to participate in the conference FIG. 7A is an explanatory diagram showing an example of moving an icon for starting a conference. When the user 1 holds an instant meeting with another user (three people), the user 1 has an icon (icon 2, icon 2, which corresponds to the other user) in the virtual space screen as shown in FIG. 7A. The icon 3 and the icon 4) are placed close to the icon (icon 1) of the user 1 and placed in the area of level 0. At this point, the user 1 can speak to the other three users. When another user notices the voice and responds to the user 1 by voice or the like, the intention to participate in the conference can be confirmed.

(B) Start-up of the conference FIG. 7B is an explanatory diagram showing the start-up of the conference system. When the user 1 designates the icons of the other three users on the virtual space screen and performs the operation of starting the conference, the conference unit 250 of the user terminal 20 activates the conference system. As a result, the conference screen is displayed as shown in FIG. 7B, and the conference is started. If there are icons other than conference participants in the level 0 area of the virtual space screen, the conference unit 250 forces such icons to move to level 1.

(C) Confirmation of intention to participate in the meeting (addition)
FIG. 8A is an explanatory diagram showing an example of moving an icon for adding a participant to the conference. The user 1 can also call another user who has not participated in the conference to the conference in the middle of the held conference. For example, the user 1 places an icon (icon 5) corresponding to another adder who has not participated in the conference in the virtual space screen close to the icon (icon 1) of the user 1 in the area of level 0. .. At this point, the user 1 can speak to four other users including the adder. When the adder notices the voice and responds to the user 1 by voice or the like, the intention to participate in the conference can be confirmed.

(D) Addition of Participants to the Conference FIG. 8B is an explanatory diagram showing the addition of participants to the conference. When the user 1 specifies the icon of the adder in the virtual space screen and performs the operation of adding the conference, the conference unit 250 adds the adder to the conference. As a result, as shown in FIG. 8B, the conference screen with the adder is displayed.

(E) Exiting from the meeting and ending the meeting FIG. 9 is an explanatory diagram showing an example of a screen after the meeting is finished. When the user 1 specifies the icon (icon 1) of the user 1 in the virtual space screen and operates the end of the conference, the conference screen disappears as shown in FIG. 9, and the conference unit 250 ends the conference. .. At this point, the icons of the five people who participated in the meeting are in the area of level 0, so that the five people can continue the voice conversation. When the user 1 specifies an icon corresponding to one of the users in the virtual space screen and performs an operation to leave the conference, the conference unit 250 causes the user corresponding to the icon to leave the conference. .. In this case, the conference unit 250 forcibly moves the exit icon to the level 1 area.

(S6: Meeting control)
The control executed based on the screen operation related to the above-mentioned conference will be specifically described.

Regarding the activation of the conference, the operation unit 220 of the user terminal 20 includes the participant information indicating the user participating in the conference in the spatial operation data and transmits the participant information to the server 40. The control unit 420 of the server 40 transmits parameters (meeting URL, ID, etc.) necessary for holding the conference to the user terminal 20 of each participant as conference parameters based on the participant information. The conference unit 250 of each user terminal 20 starts operation based on the conference parameters, and exchanges conference data (video, audio) with other user terminals 20 via the control unit 420 of the server 40.

Regarding the addition of participants to the conference, the operation unit 220 of the user terminal 20 includes the adder information indicating the adder in the spatial operation data and transmits it to the server 40. The control unit 420 of the server 40 transmits parameters (meeting URL, ID, etc.) necessary for adding participants to the user terminal 20 of the adder as meeting parameters based on the adder information. The conference unit 250 of the user terminal 20 of the adder starts the operation based on the conference parameters, and exchanges conference data (video, audio) with the other user terminals 20 via the control unit 420 of the server 40. conduct.

Regarding the exit from the conference, the operation unit 220 of the user terminal 20 includes the exit information indicating the exit person leaving the conference in the spatial operation data and transmits the exit information to the server 40. Based on the exit information, the control unit 420 of the server 40 transmits data for notifying the user terminal 20 of the exiter of the conference exit as a conference parameter. The conference unit 250 of the user terminal 20 of the exiter executes the exit from the conference based on the conference parameters received from the server 40.

Regarding the end of the conference, the operation unit 220 of the user terminal 20 includes the conference end information in the spatial operation data and outputs it to the control unit. The control unit 420 of the server 40 transmits data for notifying the end of the conference to the user terminal 20 of each participant as a conference parameter based on the conference end information. The conference unit 250 of the user terminal 20 of each participant ends the operation of the conference based on the conference parameters received from the server 40.

(S7: Human relationship estimation)
The control unit 420 of the server 40 stores the spatial operation data input from each user terminal 20. Then, the control unit 420 periodically (for example, once a week) outputs the accumulated spatial operation data to the human relationship estimation unit 430.

The human relationship estimation unit 430 calculates two types of human relationship indexes based on the spatial operation data input from the control unit 420, and uses these as human relationship estimation data indicating the human relationships between a plurality of users. Output to the relationship display terminal 50. Hereinafter, specific examples of the method for estimating human relationships will be described with reference to FIGS. 10 and 11.

(A) Human relations based on virtual spatial distance The spatial operation data includes information indicating an operation in which each user sets a sharing level for other users, as described above. The human relationship estimation unit 430 can calculate the average of the shared level between the two parties in the past predetermined period (for example, one week) and use this as an index of the human relationship. For example, the human relations estimation unit 430 sets the shared level 0 as "0", the shared level 1 as "1/3", the shared level 2 as "2/3", and the shared level 3 as "3/3". Calculate the average of.

FIG. 10 is an explanatory diagram showing an example of estimation results of human relationships between four users (user 1, user 2, user 3, user 4). For example, if the numerical value from user 1 to user 2 is "0.2", the average of the indexes related to the sharing level set by user 1 for user 2 is "0.2" (sharing level 0). And shared level 1). In the example of this figure, the user 1 and the user 2 have a close relationship, the user 1 and the user 4 have a sparse relationship, and the user 1 and the user 3 have an unbalanced relationship. It is inferred that the user 4 does not have much to do with other users.

(B) Human relations based on the conference time The spatial operation data includes operation information regarding the user's participation in the conference, as described above. The human relations estimation unit 430 can calculate the average of the meeting time between two parties in the past predetermined period (for example, one week) and use this as an index of human relations. For example, the human relations estimation unit 430 calculates the human relations index by setting the total meeting time of the past week to be "0" and the total meeting time of 0 hours to be "1".

FIG. 11 is an explanatory diagram showing an example of estimation results of human relationships between four users (user 1, user 2, user 3, user 4). For example, if the numerical value of the combination of the user 1 and the user 2 is "0.0", it means that the user 1 and the user 2 are having a close meeting, and the combination including the user 4 is included. When the numerical value is generally close to "1.0", it means that the user 4 does not have many meetings with other users.

(S8: Human relations display)
When the human relations estimation data is input from the human relations estimation unit 430, the human relations display terminal 50 displays the human relations estimation data, the analysis result of the human relations estimation data, and the like.

(S9: End judgment)
When the user 1 instructs the operation unit 220 of the user terminal 20-1 to end the display of the virtual space screen, the user terminal 20-1 ends the operation related to this system. If the user 1 does not instruct the end of the display, the process from S2 is repeated.

<4. Action effect>
According to one embodiment of the present invention described above, various effects can be obtained. For example, in one embodiment of the present invention, the user terminal 20 of the user displays a virtual space screen in which an icon corresponding to the user himself / herself and an icon corresponding to another user are arranged, and corresponds to the user himself / herself. The sharing level for other users is set according to the distance between the icon to be displayed and the icon corresponding to another user. If the distance between the icon corresponding to the user himself / herself and the icon corresponding to another user is short and the sharing level is set to level 0, the user can see the state of the other user as the other user. It is possible to see the video data, check it, and start a conversation by calling out. If the distance between the icon corresponding to the user himself / herself and the icon corresponding to another user is long and the sharing level is set to level 2, the user can see the state of other users based on the presence data. confirm. In order for the user to see the state of the other user or call out to the other user, the user draws the icon corresponding to the other user to the user's own icon. Just do it.

Further, according to one embodiment of the present invention, the user obtains consent to hold a conference with video data and audio data from another user corresponding to the icon located near the user's own icon. It is possible to smoothly start an unscheduled sudden meeting. In this way, the situation of other users can be visualized at a shared level according to the relationship between the user and other users, and sudden and sudden communication can be started smoothly. It is possible to suppress the occurrence of a situation in which the other party does not notice the conference holding notification or hesitates to hold the conference because the situation of other users is unknown.

In addition, when the user wants to avoid communication with other users, such as when he / she wants to concentrate on his / her work, the user icon is set to level 0 on the user terminal 20 of another user by the barrier setting operation. It can be prevented from being placed in the area. This makes it possible to limit the viewing of the user's video data by other users and the ability of other users to speak to the user.

According to one embodiment of the present invention, it is possible to estimate the psychological distance and human relationships between users based on the distance between the icons of each user on the virtual space screen, the usage time of the conference system, and the like. .. Such an estimation method has the effect of being able to estimate human relationships with higher accuracy than other methods that require estimation of the content of speech and the person to speak.

<5. Modification example>
The embodiment of the present invention has been described above. Hereinafter, some modifications of the above-described embodiment will be described. In addition, each modification described below may be applied alone to the above-described embodiment, or may be applied in combination to the above-mentioned embodiment. Further, each modification may be applied in place of the configuration of the above-described embodiment, or may be additionally applied to the configuration of the above-described embodiment.

For example, in the above, the example in which the sharing level in the virtual space screen is level 0, level 1, level 2, and level 3 has been described. In addition, the shared content at level 0 is the viewing of the other party's video data, the viewing of the other party's presence data, and the voice call, and the shared content at level 1 is the viewing of the other party's video data and the other party's presence data. Yes, the shared content at level 2 is the browsing of the presence data of the other party, and the example where there is no shared content at level 3 was explained. However, the number of sharing levels and the content shared at each level are not limited to the above examples. For example, even if the number of shared levels is increased or decreased, the same effect as that of the above-described embodiment can be obtained. Further, the difference in the shared content at each level may be the resolution of the video of the other party. In addition to video data and audio data, the shared content for communication with the other party may include text messages, illustrations, symbols, etc. expressing intentions and feelings. In this case as well, it is possible to obtain the same effect as that of the above-described embodiment.

In the above, the case where each user sets the sharing level for other individual users has been described. However, when any two users set different sharing levels for each other, the level of information and communication of the other party shared by each is different. Therefore, in order to make the levels of information sharing between users equal, if the sharing levels set for each other by any two users are different, the control unit 420 of the server 40 will use these two sharing levels. You may set a new sharing level based on. For example, when the first user sets the level 0 for the second user and the second user tries to set the level 1 for the first user, the control unit 420 sets the level 0. Both shared levels may be unified to level 0 or level 1. With such a configuration, the first user and the second user can communicate with each other at an equal sharing level in both directions.

In the above, an example in which a screen element indicating the situation of another user is displayed in the icon arranged on the virtual space screen has been described. As another configuration, the screen element indicating the situation of another user may be displayed in any area in the display screen instead of in the icon of the other user. For example, the display screen includes a virtual space area and a communication area for each sharing level, and as the icon moves in the virtual space area, another communication area of the corresponding sharing level among the communication areas for each sharing level is added. User information (screen elements) may be displayed. Even in such a configuration, it is possible to obtain the same effect as that of the above-described embodiment.

In the above, an example in which an icon corresponding to another user is arranged on the virtual space screen has been described, but it is possible to obtain the same effect as the above-described embodiment without using the virtual space screen. For example, icons corresponding to other users may be arranged on the display screen of the user terminal 20 and the sharing level may be set for each icon. Further, the display screen of the user terminal 20 may be divided into areas for each sharing level, and the sharing level may be set by arranging icons corresponding to other users in each area.

In the above, an example of estimating the psychological distance and human relations between users based on the distance between the icons of each user on the virtual space screen and the usage time of the conference system has been described. Further, when estimating the human relationship, the history of the barrier setting operation in the virtual space screen may be used. That is, when a user performs a barrier setting operation for a specific other user, it can be considered that the psychological distance from the user to the specific other user is far. In such a case, the control unit 420 of the server 40 may increase the numerical value shown in FIG. 10 from the user to the specific other user. As a result, it is expected that the estimation accuracy of human relationships will be further improved.

Further, the control unit 420 of the server 40 is from another user whose sharing level is set to level 3 (not displayed on the virtual space screen) and level 2 (only presence information is displayed). At least some users may be selected and the sharing level for the selected other users may be temporarily raised by one level. In this case, the icon corresponding to the other user for which level 3 has been set is placed in the area of level 2, and the icon corresponding to the other user for which level 2 has been set moves to the area of level 1. .. This is an opportunity for users to review the sharing level with other users. After that, the user can improve the estimation accuracy of the human relationship by reviewing the arrangement of the icons in the virtual space screen.

<6. Hardware configuration>
Each embodiment of the present invention has been described above. Information processing such as the display of the virtual space screen and the control of the conference described above is realized by the cooperation between the software and the hardware of the user terminal 20 described below.

FIG. 12 is a block diagram showing the hardware configuration of the user terminal 20. The user terminal 20 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and a host bus 204. Further, the user terminal 20 includes a bridge 205, an external bus 206, an interface 207, an input device 208, a display device 209, an audio output device 210, a storage device (HDD) 211, a drive 212, and a network. It is equipped with an interface 215.

The CPU 201 functions as an arithmetic processing unit and a control device, and controls the overall operation in the user terminal 20 according to various programs. Further, the CPU 201 may be a microprocessor. The ROM 202 stores programs, calculation parameters, and the like used by the CPU 201. The RAM 203 temporarily stores a program used in the execution of the CPU 201, parameters that appropriately change in the execution, and the like. These are connected to each other by a host bus 204 composed of a CPU bus or the like. By the cooperation between the CPU 201, ROM 202 and RAM 203 and the software, the above-mentioned functions such as the operation unit 220, the media input unit 230, the media reproduction unit 240 and the conference unit 250 can be realized.

The host bus 204 is connected to an external bus 206 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 205. It is not always necessary to separately configure the host bus 204, the bridge 205, and the external bus 206, and these functions may be implemented in one bus.

The input device 208 is an input means for the user to input information such as a mouse, a keyboard, a touch panel, a button, a microphone, a sensor, a switch, and a lever, and an input that generates an input signal based on the input by the user and outputs the input signal to the CPU 201. It consists of a control circuit and so on. By operating the input device 208, the user of the user terminal 20 can input various data to the user terminal 20 and instruct the processing operation.

The display device 209 includes, for example, a liquid crystal display (LCD) device, a projector device, an OLED (Organic Light Emitting Diode) device, and a display device such as a lamp. Further, the audio output device 210 includes an audio output device such as a speaker and headphones.

The storage device 211 is a data storage device configured as an example of the storage unit of the user terminal 20 according to the present embodiment. The storage device 211 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting data recorded on the storage medium, and the like. The storage device 211 is composed of, for example, an HDD (Hard Disk Drive) or SSD (Solid Storage Drive), or a memory having an equivalent function. The storage device 211 drives the storage and stores programs and various data executed by the CPU 201.

The drive 212 is a reader / writer for a storage medium, and is built in or externally attached to the user terminal 20. The drive 212 reads the information recorded in the removable storage medium 24 such as the mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 203 or the storage device 211. The drive 212 can also write information to the removable storage medium 24.

The network interface 215 is, for example, a communication interface composed of a communication device or the like for connecting to the network 12. Further, the network interface 215 may be a wireless LAN (Local Area Network) compatible communication device or a wire communication device that performs wired communication.

The hardware configuration of the user terminal 20 described above can also be applied to the server 40.

<7. Supplement>
Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person having ordinary knowledge in the field of technology to which the present invention belongs can come up with various modifications or modifications within the scope of the technical ideas described in the claims. , These are also naturally understood to belong to the technical scope of the present invention.

For example, each step in the processing of the user terminal 20 and the server 40 of the present specification does not necessarily have to be processed in chronological order in the order described as a flowchart. For example, each step in the processing of the user terminal 20 and the server 40 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

Further, it is possible to create a computer program for making the hardware such as the CPU, ROM, and RAM built in the user terminal 20 and the server 40 exhibit the same functions as the above-mentioned configurations of the user terminal 20 and the server 40. Is. A storage medium for storing the computer program is also provided.

20 User terminal 220 Operation unit 230 Media input unit 240 Media playback unit 250 Conference 30 Bird's-eye view camera 40 Server 420 Control unit 430 Human relationship estimation unit 50 Human relationship display terminal

Claims (14)

A communication system having a first user terminal used by a first user and one or more second user terminals used by one or more second users.
The first user terminal is
An operation unit that detects an operation for setting a sharing level for each of the one or more second users, and an operation unit.
For each of the second users, a display screen is generated in which screen elements showing the status of the second user are arranged in an manner corresponding to the sharing level set for the second user. Display control unit and
A communication system. The display screen includes a movable display corresponding to each of the second users.
The operation unit detects the operation of moving the movable display as the operation of setting the sharing level.
The communication system according to claim 1, wherein the sharing level set for the second user changes according to the movement of the movable display. The communication system according to claim 2, wherein in the display screen, the screen element for the second user is arranged in a movable display corresponding to the second user. The communication system according to claim 2, wherein in the display screen, the screen element for the second user is arranged in a screen area different from the movable display corresponding to the second user. When the first sharing level is set for the second user, the image of the second user is arranged on the display screen as a screen element showing the situation of the second user, and the display screen is described. It is possible to make a voice from the first user to the second user.
When the second sharing level is set for the second user, a text indicating the situation of the second user is arranged on the display screen as a screen element indicating the situation of the second user. The communication system according to any one of claims 1 to 4. When the sharing level set by the first user for a second user and the sharing level set by the second user for the first user are different, both The communication system according to any one of claims 1 to 5, further comprising a shared level setting unit for matching the shared levels of the above. Sharing level setting that limits the sharing level that can be set for each of the first users when a predetermined operation is detected by the operation unit of the first user terminal. The communication system according to any one of claims 1 to 5, further comprising a unit. When the first user terminal sets a sharing level at which the first user and the second user can communicate with each other, the operation unit detects the conference start operation. The communication system according to any one of claims 1 to 7, further comprising a conference unit that controls activation of a conference system in which the first user and the second user participate. Based on the information indicating the operation for each of the first user terminal and the one or more second user terminals, the first user and the one or more second users The communication system according to any one of claims 1 to 8, further comprising a human relationship estimation unit that estimates a human relationship between a plurality of users. The human relations estimation unit is a shared level history set by each of the plurality of users for other users, a user designates a specific other user, and the other user specifies the other user. Estimate the relationships between the plurality of users based on at least one of the restrictions on the sharing level that can be set for the users or the combination of users who participated in the conference system that was held. , The communication system according to claim 9. The invention according to any one of claims 1 to 5, further comprising a shared level setting unit for changing the shared level set from the first user to the second user to a higher shared level. Communication system. A user terminal that operates as a first user terminal used by a first user, and is a user terminal.
An operation unit that detects an operation for setting a sharing level for each of one or more second users, and an operation unit.
For each of the second users, a display screen is generated in which screen elements showing the status of the second user are arranged in an manner corresponding to the sharing level set for the second user. Display control unit and
A user terminal equipped with. Computer,
A user terminal that operates as a first user terminal used by a first user, and is a user terminal.
An operation unit that detects an operation for setting a sharing level for each of one or more second users, and an operation unit.
For each of the second users, a display screen is generated in which screen elements showing the status of the second user are arranged in an manner corresponding to the sharing level set for the second user. Display control unit and
A program for functioning as a user terminal. It is a method executed by a user terminal that operates as a first user terminal used by the first user.
To detect an operation to set a sharing level for each of one or more second users,
For each of the second users, a display screen is generated in which screen elements showing the status of the second user are arranged in an manner corresponding to the sharing level set for the second user. That and
Including, how.

Images