JP7279646B2 - Information processing device, information processing method and program - Google Patents

Description

The present disclosure relates to an information processing device, an information processing method, and a program.

2. Description of the Related Art In a virtual world with a predetermined spatial extent built on a computer network such as the Internet, technology is being developed that allows multiple users to participate simultaneously and perform real-time communication.

For example, Patent Literature 1 below discloses a conversation support system that enables communication with other users by performing a calling operation using a virtual mobile phone that can be used by the user's avatar who has logged into the virtual space. It is

JP-A-2001-154966

However, in the conversation support system disclosed in Patent Document 1, when the user's avatar who is in communication logs out of the virtual space, it is not possible to continuously communicate with the user existing in the real space. There is still room for improvement in switching communication spaces between users.

Therefore, in view of the above circumstances, the present disclosure proposes an information processing device, an information processing method, and a program that can smoothly switch the place of communication between users between the virtual space and the real space.

According to the present disclosure, in a virtual space where a virtual image corresponding to a real image of a user existing in the real space is generated, a communication control unit that controls communication between users via the virtual image of the user. a user information acquisition unit that acquires at a predetermined timing attribute information including at least position information about the user's position in real space and information about the degree of intimacy between users; and at least one of the position information and the attribute information. An information processing apparatus is provided that includes a switching control unit that controls switching between communication performed in a real space and communication performed in a virtual space.

In addition, according to the present disclosure, in a virtual space where a virtual image corresponding to a real image of a user existing in the real space is generated, communication between users can be controlled via the virtual image of the user. and acquiring, at a predetermined timing, attribute information including at least position-related information about the user's position in the real space and information about the degree of intimacy between the users; An information processing method is provided that includes controlling switching between communication performed in a real space and communication performed in a virtual space.

Further, according to the present disclosure, in a virtual space in which a computer generates a virtual image corresponding to a real image of a user existing in the real space, communication is performed between users via the virtual image of the user. a communication control unit that controls; an information acquisition unit that acquires at a predetermined timing attribute information including at least position-related information about the user's position in real space and information about the degree of intimacy between users; and at least one of the position information and the attribute information. According to one, a program is provided for functioning as a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space.

According to the present disclosure, the content to be output to the virtual space is controlled according to the attribute information including at least the position information regarding the user's position in the real space and the information regarding the degree of intimacy between the users.

As described above, according to the present disclosure, it is possible to smoothly switch the communication space between the virtual space and the real space.

In addition, the above effects are not necessarily limited, and together with the above effects or instead of the above effects, any of the effects shown in this specification, or other effects that can be grasped from this specification effect may be exhibited.

1 is a block diagram showing an example of the configuration of an information processing device according to an embodiment of the present disclosure; FIG. It is a block diagram which shows an example of a structure of the information processing system which concerns on the same embodiment. It is an explanatory view for explaining operation of an information processor concerning the embodiment. It is an explanatory view for explaining operation of an information processor concerning the embodiment. It is an explanatory view for explaining operation of an information processor concerning the embodiment. It is an explanatory view for explaining operation of an information processor concerning the embodiment. It is an explanatory view for explaining operation of an information processor concerning the embodiment. It is an explanatory view for explaining operation of an information processor concerning the embodiment. It is an explanatory view for explaining operation of an information processor concerning the embodiment. It is a flow chart for explaining an example of the flow of operation of the information processing apparatus according to the embodiment. It is a flow chart for explaining an example of the flow of operation of the information processing apparatus according to the embodiment. It is a block diagram which shows the hardware structural example of the information processing apparatus which concerns on the same embodiment.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, thereby omitting redundant description.

In addition, in this specification and drawings, a plurality of components having substantially the same functional configuration may be distinguished by attaching different alphabets after the same reference numerals. However, when there is no particular need to distinguish between a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are used.

Note that the description will be given in the following order.
<<Configuration>>
<<Operation>>
<Operation example 1>
<Operation example 2>
<<Hardware configuration>>
<< Conclusion >>

<<Configuration>>
First, the configuration of an information processing apparatus 10 according to an embodiment of the present disclosure will be described with reference to FIGS. 1 and 2. FIG. FIG. 1 is a block diagram showing a configuration example of an information processing device 10 according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing a configuration example of an information processing system 5 according to an embodiment of the present disclosure.

The information processing apparatus 10 according to the present embodiment is an apparatus in which a plurality of users communicate with each other and communicate with each other while displaying images and avatars of each other on a display screen. In the information processing system 5 according to the present embodiment, as shown in FIG. 2, a plurality of information processing apparatuses 10 are interconnected via a network 20 to perform mutual communication in virtual space. The information processing apparatus 10 according to the present embodiment is an apparatus that generates a virtual image in a virtual space corresponding to a real image of a user existing in the real space, and realizes communication between users through the virtual image. The information processing device 10 according to the present embodiment is a device that can be used while the user is moving, and is worn by the user, such as a glasses type display or a head mounted display (HMD). It may be a device to be used, or a portable communication device having a display device such as a smartphone.

The network 20 is implemented using a known information communication technology such as the Internet, and is capable of interconnecting a plurality of information processing apparatuses 10 and allowing a plurality of users to communicate at the same time. and

As shown in FIG. 1, the information processing apparatus 10 according to the present embodiment includes an information acquisition unit 110 that acquires information such as user information about a user, motion information about a user's action, voice information about a voice uttered by a user, A switching control unit 120 that switches the space in which communication is carried out according to the information acquired by the information acquisition unit 110, and a communication control that controls various functions in communication such as the connection state of communication mutually carried out between users. and at least a portion 130 . Furthermore, the information processing apparatus 10 according to this embodiment preferably includes a recognition determination section 140 , a communication control section 150 and a storage section 160 .

[Information acquisition unit 110]
The information processing apparatus 10 according to this embodiment includes an information acquisition unit 110 . The information acquisition unit 110 acquires information for smoothly switching the communication space that the user is implementing. The information acquisition section 110 includes at least a user information acquisition section 112 , and may further include a motion information acquisition section 114 and a voice information acquisition section 116 . At least one of the user information acquisition unit 112, the motion information acquisition unit 114, and the voice information acquisition unit 116 functions according to the content of the information to be acquired, thereby smoothly switching the space for communication performed by the user. Various kinds of information for

(User information acquisition unit 112)
The user information acquisition unit 112 acquires, at a predetermined timing, attribute information including at least position information about the user's position in the real space and information about the degree of intimacy between users. Specifically, the user information acquisition unit 112 may acquire location information of the information processing device 10 in which the user information acquisition unit 112 is provided. The user information acquisition unit 112 may also have a function of acquiring position information and attribute information of other users via the communication control unit 150, which will be described later.

The user information acquisition unit 112 may acquire user position information using a known position information acquisition system. Known position information acquisition systems include, for example, GNSS (Global Navigation Satellite System), indoor positioning systems, mobile networks, Wi-Fi (registered trademark) base station positioning systems, and other position acquisition systems. may be Further, the information processing apparatus 10 may be provided with a tag such as a beacon that is used for position detection, and the user information acquisition unit 112 may receive information regarding this tag as position information. Furthermore, the information processing apparatus 10 may have a communication function capable of identifying the user's direction, and the user information acquisition unit 112 may acquire information regarding this user's direction as position information.

In addition, as described above, other users' location information may be acquired from other users via the communication control unit 150, but acquisition of other users' location information may involve other known location information. Information acquisition techniques may be utilized. For example, the information processing apparatus 10 according to the present embodiment may include an imaging device that captures an environment image of a real space, and uses the captured image generated by this imaging device to obtain location information of other users. may be obtained. Various position estimation techniques disclosed in Japanese Unexamined Patent Application Publication No. 2010-16743, Japanese Patent No. 5869883, Japanese Unexamined Patent Application Publication No. 2017-182739, etc., for example, are appropriately used to acquire location information of other users. be able to. The imaging device provided in the information processing apparatus 10 may be, for example, a known imaging device having an imaging function, such as a digital still camera or a digital video camera. In addition, the user information acquisition unit 112 acquires location information of other users based on images of the other users captured by an external camera or the like provided in real space, via a known network or the like. may be obtained.

The user information acquisition unit 112 acquires rough position information between the users using GNSS or an indoor positioning system when the separation distance between the users who are communicating is large. is captured, the position information of the other user may be acquired by image processing.

The location information also includes secondary information obtained based on the locations of multiple users, such as distances between users.

In addition, the information on the degree of intimacy between users, which is one of the attribute information acquired by the user information acquisition unit 112, includes, for example, the frequency of communication performed using the information processing apparatus 10 according to the present embodiment, Information such as time and content of conversation, presence/absence of user information in the contact list stored in storage unit 160, and the like are included. Information such as the communication frequency, time, and conversation content may be stored as history information in the storage unit 160 provided in the information processing apparatus 10 .

In addition, the attribute information includes non-verbal communication information such as user's facial expressions, gestures, and voice quality (hereinafter simply referred to as "non-verbal communication abbreviated as "information".) may be included. A part of the functions of the user information acquisition unit 112 may be implemented by, for example, an imaging device as already described. The imaging device used preferably has a function of picking up the voice uttered by the user of the information processing device 10 according to the present embodiment, and nonverbal communication information is acquired by such an imaging device. good too. Furthermore, by presetting the user's mental state according to the non-verbal communication information, the user information acquisition unit 112 can acquire mental state information regarding the user's mental state. This mental state information is also included in the attribute information. The mental state information may be automatically generated from the non-verbal communication information through machine learning by the information processing device 10 . For example, the information processing apparatus 10 according to the present embodiment may be given a tag related to a user's attribute, and the user information acquisition unit 112 may acquire information related to this tag as attribute information.

As described above, the user information acquisition section 112 may acquire attribute information of other users via the communication control section 150 . Furthermore, as described above, the user information acquisition unit 112 may be acquired by an imaging device separately provided in the information processing device 10, or by an image or sound acquired by an imaging device provided in the real space. may be

The user information acquisition unit 112 acquires position information and attribute information of a user and position information and attribute information of other users at a predetermined timing. For example, the user information acquisition unit 112 may acquire the position information and the attribute information at any time, or at a timing preset by the user. Also, the user information acquisition unit 112 may transmit the position information and the attribute information to at least one of the switching control unit 120 and the communication control unit 130, which will be described later, at any time, or may transmit the information at a preset timing.

(Motion information acquisition unit 114)
The motion information acquisition unit 114 acquires motion information related to motions such as gestures and hand movements of the user. The motion information acquired by the motion information acquisition unit 114 may be transmitted to the switching control unit 120, which will be described later, and used as a kind of trigger for switching spaces for communication between users. Further, this operation information may be used as a trigger when selecting a user to connect communication from among a plurality of users and starting communication.

The function of the motion information acquisition unit 114 may be realized by, for example, a known imaging device having the imaging function as described above. More specifically, to acquire the motion information, for example, a known gesture recognition technology can be used, and for example, the technology disclosed in Japanese Unexamined Patent Application Publication No. 2013-205983 can be used as appropriate.

(Voice information acquisition unit 116)
The voice information acquisition unit 116 acquires voice information related to the voice uttered by the user. The voice information acquired by the voice information acquisition unit 116 may be provided to other users via the communication control unit 130, which will be described later. Further, the voice information acquisition unit 116 may acquire voice recognition information by analyzing voice information using a known voice recognition technology and recognizing the contents of the voice information. The voice information acquired by the voice information acquisition unit 116 may be transmitted to the switching control unit 120, which will be described later, and used as a kind of trigger for switching spaces for communication between users. The function of the audio information acquisition unit 116 may be realized by a sound collecting device such as a sound collecting microphone, or may be realized by a sound collecting device provided in an imaging device separately provided in the information processing device 10. . Further, the voice information acquisition unit 116 may acquire voices generated around the information processing device 10 .

[Switching control unit 120]
The information processing apparatus 10 according to this embodiment includes a switching control unit 120 . The switching control unit 120 controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information transmitted from the information acquisition unit 110. . For example, when the separation distance between a plurality of users communicating in the real space becomes equal to or greater than a predetermined threshold, the switching control unit 120 connects the communication between the users and enables communication in the virtual space. Communication may be enabled. The switching control unit 120 can switch the communication space according to the separation distance between users and time information. For example, even if the separation distance between users is equal to or greater than a predetermined threshold, if the time period during which the separation distance is equal to or greater than the predetermined threshold is shorter than the predetermined time, communication in real space may not be possible. may be maintained. Further, for example, when the separation distance between a plurality of users communicating in the virtual space becomes equal to or less than a predetermined threshold, the switching control unit 120 disconnects the communication between the users, It may be possible to perform communication on the

In addition, the switching control unit 120 can switch the communication space according to the degree of intimacy between users, which is one piece of attribute information. For example, when the frequency or duration of communication between specific users using the information processing device 10 is equal to or greater than a predetermined frequency or duration, the communication space may be switched from the real space to the virtual space.

The switching control unit 120 further establishes a network connection between users according to the motion information acquired by the motion information acquisition unit 124, and switches the communication space between the users from the real space to the virtual space. good. More specifically, the switching control unit 120 is set in advance with control details related to switching according to the motion information, and the switching control unit 120 can switch the communication space according to the acquired motion information. For example, when any of the users who are communicating in the real space performs an action intended to break up on the spot (for example, an action such as waving), the switching control unit 120 performs the action information acquisition unit Based on the motion information acquired from 114, the communication space may be switched from the real space to the virtual space. By switching the communication space from the real space to the virtual space in this way, the user can naturally continue the conversation without interruption. Note that the information processing apparatus 10 may perform machine learning to associate the user's action intention with the action information, and the switching control unit 120 changes the communication space from the real space to the virtual space based on the action information. You can switch to

The switching control unit 120 may further establish a network connection between users according to the voice information acquired by the voice information acquisition unit 116, and switch the communication space from the real space to the virtual space. More specifically, control details regarding switching are set in advance according to the voice recognition information acquired from the voice information acquisition unit 116, and the switching control unit 120 can switch the communication space according to the voice recognition information. Specifically, the voice information acquisition unit 116 of one of a plurality of users communicating in real space acquires predetermined voice information such as a farewell greeting on the spot, and the content of the voice information is obtained. It analyzes and acquires the speech recognition information corresponding to the speech information. The switching control section 120 may switch the communication space from the real space to the virtual space based on the speech recognition information acquired from the speech information acquisition section 116 . Similarly, when voice information acquisition section 116 acquires predetermined voice information, switching control section 120 may switch the communication space from the virtual space to the real space.

The switching control unit 120 further selects at least one other user from among the plurality of users according to the motion information acquired by the motion information acquisition unit 114, and changes the communication space from the real space to the virtual space. You can switch. Specifically, virtual images of a plurality of users communicating in real space are displayed in the virtual space, and switching control unit 120 receives action information corresponding to the action of selecting a virtual image. In this case, the switching control unit 120 may switch the communication performed with the other user who is the target of the motion information from the real space to the virtual space.

[Communication control unit 130]
The information processing device 10 according to this embodiment includes a communication control unit 130 . The communication control unit 130 controls the connection state of communication mutually implemented between users via the virtual image of the user in the virtual space where the virtual image corresponding to the real image of the user existing in the real space is generated, Overall control of various functions in communication. The communication control unit 130 has a function of allowing users to have a conversation while displaying an image or an avatar of a communication partner on a display provided in the information processing apparatus 10, for example. Further, communication control section 130 may end communication without switching the communication space to the virtual space depending on the user information, motion information, and voice information acquired by information acquisition section 110 .

The communication control unit 130 outputs according to various information acquired by the information acquisition unit 110 and user recognition information including recognition confirmation information of the real image of the communication partner in real space acquired by the recognition determination unit 140. It has an output control unit 132 that controls the content.

The function of the output control unit 132 will be described with reference to FIG. The output control unit 132 displays in the virtual space a virtual image corresponding to the real image of another user in the real space. For example, as shown in FIG. 3, the output control unit 132 provided in the information processing apparatus 10 used by the user U1 can provide the user U1 with a virtual space U1V and a virtual image of the user U2 who is the communication partner. . Similarly, the output control unit 132 provided in the information processing apparatus 10 used by the user U2 can provide the virtual space U2V and the virtual image of the user U1 who is the communication partner to the user U1.

The output control unit 132 acquires the user information of other users acquired by the information acquisition unit 110 at any time and reflects it on the virtual image, thereby displaying the virtual image corresponding to the latest real image of the other user. is possible. Since the virtual images of other users are updated in real time, as shown in FIG. 4, the virtual image VI2 of the user U2 also changes to an angry expression at the timing when the facial expression of the user U2 changes from a smile to an angry expression. , and the user U1 can recognize this state.

The virtual image to be displayed in the virtual space may be, for example, an image of another user captured by an imaging device separately provided in the information processing device 10 used by the user and having a function of capturing an image of the external environment in the real space. Alternatively, the image may be an image obtained by subjecting a captured image of another user to predetermined conversion processing. In addition, an image acquired by an imaging device that is separately provided in the information processing device 10 and captures at least part of the face of the user of the information processing device 10 may be used as the image of the other user. Specifically, the virtual image may be generated by applying image information of a part of the user's face captured by an imaging device as a parameter for rendering a three-dimensional model prepared in advance.

The output control unit 132 may further control the content to be output to the virtual space according to at least one of the user's position information and attribute information acquired by the user information acquisition unit 112 . The output control unit 132 may display a virtual image of another user in the virtual space, for example, when the separation distance between users communicating in the real space is greater than or equal to a predetermined threshold. . In addition, the output control unit 132 controls the output control unit 132 according to attribute information such as facial expressions and voice quality of users acquired immediately before a plurality of users communicating in the same place in the real space start to move to different places. may be output so that a virtual image is displayed.

In addition, the output control unit 132 may control the virtual image of the other user by using the image information of the other user captured by the imaging device while the communication is being performed in the real space. . Specifically, the output control unit 132 may change the virtual image of the other user so as to correspond to the captured appearance of the other user. When communication is performed in the virtual space and the displayed virtual image of the other user is different from the real image of the other user, the output control unit 132 outputs the user information of the other user to the user information acquisition unit 112. When the image is acquired by , processing such as morphing may be performed to gradually reduce the difference between the real image and the virtual image.

In addition, the output control unit 132 may control the virtual image of another user by using image information such as photos and videos of another user uploaded on an existing network such as the Internet. Specifically, the output control unit 132 applies image information such as photographs of other users uploaded on the network as parameters when rendering a three-dimensional model prepared in advance to generate a virtual image. may be generated.

The output control unit 132 may further output virtual images of other users connected to each other in a predetermined display method. For example, as shown in FIG. 5, when the real image RI2 of the user U2 is wearing a hat, the output control unit 132 performs an operation such that the virtual image VI2 wears the hat. may be changed to the virtual image VI2 wearing the . Further, for example, the output control unit 132 may output the virtual image so that the virtual image moves to the position of the real image. A method of displaying the movement of the virtual image may be, for example, a method of displaying a compromise. However, the display method in which the virtual image moves to the position of the real image is not particularly limited as long as it overlaps the real image. For example, as shown in the lower diagram of FIG. A trajectory may be displayed.

Here, a deviation may occur between the actual movement of the real image and the movement of the virtual image output in the virtual space. may remember However, the output control unit 132 controls the virtual image as described above, so that the user is concerned about the shift in movement that may occur between the movement of the real image and the movement of the virtual image. less things. Therefore, the output control unit 132 further receives user recognition information acquired by a recognition determination unit 140 having a function of determining whether or not the user has recognized a real image of another user in the real space, which will be described later. Alternatively, the virtual image may be displayed by a display method based on the received user recognition information. For example, as shown in FIG. 6, after the recognition determination unit 140 determines that the user U1 has recognized the real image RI2 of the user U2, the output control unit 132 determines whether the virtual image VI2 has moved toward the real image RI2. You may control so that such a locus|trajectory may be displayed. The output control unit 132 is not limited to the display method described above, and may output the virtual image in any display method as long as the display method expresses the behavior of the virtual image.

Further, the output control unit 132 may control to erase the virtual images after displaying the virtual images of the other users who are connected to each other so as to move to the position of the real images. The switching control unit 120 may release the connection established between the users in the virtual space after the virtual image of the other user is deleted. The display method in which the virtual image moves to the position of the real image is, as described above, a trajectory as if the virtual image VI2 were moving toward the real image RI2, as shown in the lower diagram of FIG. 6, for example. may The output control unit 132 displays the virtual image of the other user so as to move to the position of the real image, and then controls the virtual image to disappear, so that the user can easily locate the position of the real image of the other user. can grasp.

The output control unit 132 may further output a virtual image of another user connected to each other so as to indicate the real image direction of the other user. For example, in the case where communication is being carried out in the virtual space, the output control unit 132 controls, for example, the user U2 as shown in FIG. may be displayed so as to face the position of the real image RI2 based on the user information acquired by the user information acquisition unit 112 . By displaying the virtual image of the other user so as to face the direction of the real image, the user can easily grasp the position of the other user in the real space (that is, the real image).

The output control unit 132 may further output a virtual image preset by another mutually connected user. Specifically, the user information acquisition unit 112 transmits only predetermined user information set by other users to the output control unit 132, and does not transmit user information acquired by other methods to the output control unit 132. may The user information at this time may be information set by another user, and the output control unit 132 controls such that, for example, as a result of reflecting the user information, a virtual image different in appearance from the real image is displayed. You may

The output control unit 132 further superimposes the virtual images of the other users connected to each other on the real images of the other users, which are output in a manner different from the real space. may be controlled. Specifically, as shown in FIG. 8, for example, the output control unit 132 controls the virtual image VI2 of the user U2 to be displayed in a manner set by the user U1 so that the appearance is different from the real image RI2 of the user U2. You can output. The output control unit 132 controls such that the virtual image of the other user, which is different from the real image, is superimposed on the real image of the other user and displayed, so that the virtual image of the other user is displayed in a manner that makes the user feel uncomfortable if it exists in the real space. It is possible to display other users only in the virtual space. As a result, users can conduct limited communications that do not involve people around them, without the other users receiving unwanted attention from people around other users in real space. . As a result, users who communicate using the information processing apparatus 10 according to the present embodiment can feel as if they are holding a party or the like wearing costumes only among themselves, for example. .

Further, the output control unit 132 may display a virtual image of the other user in a form different from the real image while communicating in real space with another user who is mutually connected for communication. . For example, as shown in FIG. 9, the output control unit 132 is configured to indicate that the function is stopped while outputting the virtual image VI2 even when the users communicate in the real space. , the virtual image VI2 may be displayed. After that, when the communication between the users is switched from the real space to the virtual space, the output control unit 132 controls so that the virtual image displayed in a different form from the real image is displayed in a form corresponding to the real image. You may Further communication between users may then take place.

The information processing apparatus 10 according to the present embodiment may include an audio output unit having an audio output function, and the audio information output to the user may be output by the audio output unit. The output control unit 132 may further perform control such that the voice information acquired by the voice information acquisition unit 116 is artificially output from a predetermined position. That is, the position of the sound image is gradually brought closer to the position of the real image, and when the position of the real image and the position of the sound image match, the output of the voice information is stopped, and the user can hear the voice of another user in the real space, that is, the voice of the real image. It may be made recognizable. Also, when the communication space is switched from the real space to the virtual space, the output control unit 132 may match the position of the real image and the position of the sound image to output the audio information. Furthermore, the output control unit 132 may change the volume according to the separation distance between users. Specifically, the output control unit 132 may control the position of the sound image using, for example, a known audio output technology. This allows the user to feel as if he or she is actually meeting and conversing with another user in the virtual space.

In practice, there may be a gap between the output timing of audio information to be output and the generation timing of audio emitted from a real image. For this reason, the output control unit 132 may use, for example, one of known techniques, audio separation technology, to cancel audio emitted by the real image and output the audio information from the virtual image. Moreover, not only the sound emitted by the real image but also all sounds existing in the real space may be blocked, and the sound information may be reconstructed and output from the virtual space as necessary. Also, the output control unit 132 may control video output conditions and audio output conditions of an output device that outputs the virtual space.

The output control unit 132 not only performs control so that a virtual image of another user who communicates is displayed, but also displays some object specified by the user, such as a pet owned by the user, in the virtual space. may be controlled to be displayed as a virtual image in A virtual image generated by a user's designation may be provided to a plurality of users who are communicating with each other, or may be provided only to the user who designated the object.

Also, the output control unit 132 may control video output conditions and audio output conditions of an output device that outputs the virtual space.

[Recognition judgment unit 140]
The information processing apparatus 10 according to the present embodiment may include a recognition determination unit 140, and the recognition determination unit 140 recognizes a real image of another user with whom the user is mutually connected for communication on the real space. It may have a function of judging whether or not The recognition determination of the real image of the other user by the recognition determination unit 140 may be performed using a known viewpoint recognition technology. The recognition determination unit 140 detects the viewpoint position of the user using, for example, viewpoint recognition technology, and detects the viewpoint position based on the viewpoint position information about the viewpoint position and the position information of other users acquired by the user information acquisition unit 112. , it may be determined whether the user has recognized the real image of another user.

Further, the recognition determination unit 140 may determine whether or not the user has recognized the real image of another user using a known voice recognition technology. For example, the recognition determination unit 140 uses voice recognition technology to detect voice information for user determination, such as words used by a user when calling another user, and recognizes the real image of the other user. You can assume that you recognize it. At this time, the recognition determination unit 140 uses the position information acquired by the user information acquisition unit 112 in addition to the voice information, so that the recognition determination unit 140 can recognize other users with higher accuracy than when the position information is not used. real image recognition judgment can be performed.

[Communication control unit 150]
The communication control unit 150 controls communication between users via the network 20, and executes processing for connecting or disconnecting communication with a predetermined other user according to control instructions from the communication control unit 130. do. In addition, the communication control unit 150 executes processing for providing the user's location information, attribute information, motion information, voice information, etc. acquired by the information acquisition unit 110 to other users connected to the communication.

[Storage unit 160]
The storage unit 160 is an example of a storage device included in the information processing apparatus 10 according to this embodiment. In the storage unit 160, various programs, databases, and the like that are used by the information processing apparatus 10 when executing various types of processing as described above are appropriately recorded. Various types of information acquired by the information acquisition unit as described above may be recorded in the storage unit 160 as history information. Furthermore, in the storage unit 160, for example, when the information acquisition unit 110, the recognition determination unit 140, and the like perform their respective processes, intermediate progress of various processes that need to be saved may be appropriately recorded. Not limited to the processing executed by the information acquisition unit 110 and the recognition determination unit 140, various parameters that need to be saved when the information processing apparatus 10 according to the present embodiment performs some processing, progress of the processing, etc. may be recorded. The storage unit 160 allows the information acquisition unit 110, the switching control unit 120, the communication control unit 130, the recognition determination unit 140, and the like to freely perform read/write processing.

So far, an example of the configuration of the information processing apparatus 10 has been described in detail. Each component described above may be configured using general-purpose members and circuits, or may be configured by hardware specialized for the function of each component. Also, the functions of each component may be performed entirely by a CPU or the like. Therefore, it is possible to appropriately change the configuration to be used according to the technical level at which the present embodiment is implemented. Next, an example of the operation of the information processing device 10 will be described in detail.

<<Operation>>
<Operation example 1>
The operation of the information processing apparatus 10 according to this embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing an example of the operation flow of the information processing apparatus 10 according to this embodiment. First, the operation of switching communication performed in the virtual space to communication performed in the real space will be described. In this operation example, the user U1 and the user U2 exist in separate locations, and while using the information processing apparatus 10 according to the present embodiment, the user U1 and the user U2 meet at a meeting place in real space. will be described.

First, the operation according to the present embodiment is started while the user U1 and the user U2 are connected for communication, and the user information of the user U1 and the user U2 is acquired and exchanged (S101). Specifically, the user information acquiring unit 112 of the information processing apparatus 10 used by the user U1 and the user U2 acquires user information such as position information and attribute information of each of the users U1 and U2. Each acquired user information is transmitted and received through the communication control unit 150 . Next, virtual images of the user U1 and the user U2 are displayed based on the acquired user information (S103). Specifically, as shown in FIG. 2, a virtual space U1V is generated in the information processing device 10 of the user U1, and a virtual image VI2 of the user U2 is generated in the virtual space U1V. Similarly, a virtual space U2V is generated in the information processing device 10 of the user U2, and a virtual image VI1 of the user U1 is generated in the virtual space U2V. The generated virtual space is displayed by another user in the real space. may Then, the user U1 can converse with the virtual image VI2 in the virtual space U1V, and the user U2 can converse with the virtual image VI1 in the virtual space virtual image VI2.

S101 and S103 are repeated as needed, and the user information acquisition unit 112 acquires the latest user information of the user U1 and the user U2, and the virtual image U1V and the virtual image U2V are generated according to the latest user information. and the virtual image U2V are updated at any time. Next, the recognition determination unit 140 determines whether or not the user has recognized another user (S105). Specifically, when the separation distance between the user U1 and the user U2 becomes equal to or less than a predetermined threshold and the user U1 and the user U2 exist in the same environment, for example, as shown in the upper diagram of FIG. Image VI2 may be oriented toward real image RI2 of user U2. Then, when user U1 discovers user U2 (that is, real image RI2 of user U2) in the real space, recognition determination unit 140 determines that real image RI2 of user U2 has been recognized based on recognition information associated with the discovery of user U1. may At this time, although the user information has been updated to the latest in S101 and S103, if the appearance of the real image RI2 and the virtual image VI2 of the user U2 are different, the output control unit 132 changes the appearance of the virtual image VI2 to correspond to the real image RI2. may be changed to For example, when the real image RI2 is wearing a hat as shown in the lower diagram of FIG. You may

In S101 and S103, although the user information is updated to the latest by the user information acquisition unit 112, if only the user information set in advance by the user U2 is reflected in the virtual image VI2, the appearance of the virtual image VI2 is different from that of the real image RI2. VI2 is displayed. As a result, the user U1 can confirm the actual appearance of the user U2 for the first time when confirming the real image RI2 of the user U2.

Subsequently, the output control unit 132 may move the virtual image VI2 of the user U2 to the position of the real image RI2 of the user U2 (S107). The output control unit 132 may move the virtual image VI2 so as to approach the position of the real image RI2. If the appearance of the real image RI2 and the appearance of the virtual image VI2 are significantly different, the user U1 may feel uncomfortable. It is also possible to output the content indicating that the For example, the output control unit 132 may output a display of a trajectory or the like as if the real image RI2 were moved toward the position of the real image RI2, as shown in the lower part of FIG. After the virtual image VI2 of the user U2 moves to the position of the real image RI2 of the user U2, the output control unit 132 erases the virtual image VI2 (S109), and the communication control unit 150 cancels the connected communication. (S111). By operating the information processing apparatus 10 according to the present embodiment as described above, the space in which communication between the user U1 and the user U2 is carried out changes continuously and naturally from the virtual space to the real space. can be switched.

In S109, the output control unit 132 may control display so that a virtual image VI2 different from the real image RI2 of the user U2 is superimposed on the real image RI2, as shown in FIG. 8, for example. Further, the output control unit 132 may not erase the virtual image VI2, and may display the virtual image VI2 in a manner indicating that the function is stopped, as shown in FIG. 9, for example. .

<Operation example 2>
Next, another operation of the information processing apparatus 10 according to this embodiment will be described with reference to FIG. 11 . FIG. 11 is a flowchart showing an example of the operation flow of the information processing apparatus 10 according to this embodiment. A description will be given of an operation in which communication carried out in real space is switched to communication carried out in virtual space. In this operation example, a case in which user U1 and user U2, who are present at the same place in real space, move to different places will be described as an example.

First, the operation according to the present embodiment is started while the user U1 and the user U2 are connected for communication. The information acquisition unit 110 provided in the information processing apparatus 10 used by the user U1 acquires user information on the user U1 (S201). At this time, the information acquisition unit 110 may acquire not only the user information of the user U1, but also the motion information and voice information of the user U1. Next, communication control unit 130 provided in information processing apparatus 10 used by user U1 generates virtual space U1V provided to user U1 according to an instruction from user U1 (S203). The communication control unit 130 may automatically generate the virtual space U1V according to the user information of the user U1 acquired by the information acquisition unit 110 as well as the instruction of the user U1. For example, the virtual space U1V may be automatically generated when the separation distance between the user U1 and the user U2 is greater than or equal to a predetermined value. Further, the virtual space U1V may be automatically generated based on attribute information such as the facial expression and voice quality of the user U2 immediately before the user U1 and the user U2 start moving to different places. If the user U1 wishes to communicate with a plurality of users, the user U1 may perform a predetermined operation to select that user and generate virtual images of a plurality of other users in the virtual space U1V. User information of user U1 and user U2 is then exchanged (S205), and a virtual image VI2 of user U2 is generated (S207). Since S205 and S207 are the same as S101 and S103 already described in Operation Example 1, detailed description thereof will be omitted.

Finally, the communication control unit 130 executes communication end processing (S209). For example, when predetermined voice information such as a farewell greeting is acquired by the voice information acquisition unit 116, the voice information acquisition unit 116 performs voice recognition to generate voice recognition information, and communication control is performed. Unit 130 may terminate the communication based on the speech recognition information. Further, when the information processing device 10 is realized by a wearable device such as AR (Augmented Reality) glasses, communication may be terminated by removing the wearable device worn by the user from the body.

So far, the operation example of the information processing apparatus 10 according to the present embodiment has been described in detail. Next, another operation example will be described in detail.

<<Hardware configuration>>
The embodiment according to the present disclosure has been described above. The information processing described above is realized by cooperation between software and hardware of the information processing apparatus described below.

FIG. 12 is a block diagram showing the hardware configuration of the information processing apparatus according to this embodiment. The information processing apparatus includes a CPU (Central Processing Unit) 180, a ROM (Read Only Memory) 181, a RAM (Random Access Memory) 182, a host bus 183, a bridge 184, an external bus 185, an interface 186, It comprises a display device 188 , an audio output device 189 , a storage device (HDD) 190 and a network interface 913 . The information processing apparatus according to this embodiment may have an input device 187 and a drive 191 .

The CPU 180 functions as an arithmetic processing device and a control device, and controls general operations within the information processing device according to various programs. CPU 180 may also be a microprocessor. The ROM 181 stores programs used by the CPU 180, calculation parameters, and the like. The RAM 182 temporarily stores programs used in the execution of the CPU 180, parameters that change as appropriate during the execution, and the like. These are interconnected by a host bus 183 comprising a CPU bus or the like. The functions of the information acquisition unit 110, the switching control unit 120, the communication control unit 130, the recognition determination unit 140, and the like can be realized by the cooperation of the CPU 180, the ROM 181, the RAM 182, and the software.

The host bus 183 is connected via a bridge 184 to an external bus 185 such as a PCI (Peripheral Component Interconnect/Interface) bus. Note that the host bus 183, bridge 184 and external bus 185 do not necessarily have to be configured separately, and these functions may be implemented in one bus.

The input device 187 includes input means for members to input information such as touch panels, buttons, microphones, sensors, switches and levers, and an input control circuit that generates an input signal based on the input by the member and outputs it to the CPU 180. can be configured.

The display device 188 includes, for example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, or the like.

The storage device 190 is a data storage device configured as an example of a storage unit of the information processing device according to this embodiment. The storage device 190 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 190 is configured by, for example, a HDD (Hard Disk Drive), an SSD (Solid Storage Drive), or a memory having equivalent functions. The storage device 190 drives storage and stores programs executed by the CPU 180 and various data.

The drive 191 is a reader/writer for storage media, and is built in or externally attached to the information processing apparatus. The drive 191 reads information recorded in a removable storage medium 193 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 182 or the storage device 190 . Drive 191 can also write information to removable storage medium 193 .

The network interface 192 is, for example, a communication interface configured with a communication device or the like for connecting to the network 20 . The network interface 192 may be a terminal compatible with a wireless LAN (Local Area Network) or a wire terminal that performs wired communication.

<< Conclusion >>
Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is obvious that those who have ordinary knowledge in the technical field of the present disclosure can conceive of various modifications or modifications within the scope of the technical idea described in the claims. is naturally within the technical scope of the present disclosure.

Also, the effects described herein are merely illustrative or exemplary, and are not limiting. In other words, the technology according to the present disclosure can produce other effects that are obvious to those skilled in the art from the description of this specification in addition to or instead of the above effects.

Note that the following configuration also belongs to the technical scope of the present disclosure.
(1)
a communication control unit for controlling mutual communication between users via the virtual image of the user in the virtual space where the virtual image corresponding to the real image of the user existing in the real space is generated;
a user information acquisition unit configured to acquire, at a predetermined timing, attribute information including at least position information relating to the position of the user in real space and information relating to the degree of intimacy between the users;
an information processing apparatus comprising a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information. .
(2)
further comprising a motion information acquisition unit that acquires motion information related to the user's motion;
The switching control unit establishes a network connection between the users according to at least one of the operation information of the user and the operation information of other mutually connected users, and switches the communication space to the The information processing apparatus according to (1), which switches from a real space to the virtual space.
(3)
further comprising a voice information acquisition unit that acquires voice information related to the voice uttered by the user;
The switching control unit establishes a network connection between the users according to at least one of the voice information of the user and the voice information of other mutually connected users to switch the communication space to the The information processing device according to (1) or (2), which switches from a real space to the virtual space.
(4)
further comprising a motion information acquisition unit that acquires motion information related to the user's motion;
The switching control unit selects at least one other user from among the plurality of users according to the motion information acquired by the motion information acquisition unit, and selects the user from the communication performed in the real space. The information processing device according to any one of (1) to (3), which switches to communication performed in a virtual space.
(5)
Any one of (1) to (4), wherein the communication control unit further comprises an output control unit that controls content to be output to the virtual space according to at least one of the position information and the attribute information. The information processing device according to .
(6)
The information processing apparatus according to any one of (1) to (5), wherein the output control unit outputs the virtual images of other users with whom the users are mutually connected in a predetermined display method.
(7)
further comprising a recognition determination unit that recognizes other users connected to each other and acquires user recognition information related to user recognition;
The output control unit, after receiving the user recognition information transmitted from the recognition determination unit, causes the virtual image of the other user to be output in a predetermined expression method, and outputs the real image of the other user, The information processing device according to any one of (1) to (6), which synchronizes the virtual image of the other user.
(8)
The information according to any one of (1) to (7), wherein the output control unit causes the virtual images of the mutually connected other users to be output so as to indicate the real image directions of the other users. processing equipment.
(9)
The output control unit erases the virtual images of other users connected to each other at the positions of the real images;
The information processing apparatus according to any one of (1) to (8), wherein the switching control unit cancels the connection established between the users in the virtual space.
(10)
The information processing apparatus according to any one of (1) to (9), wherein the output control unit outputs a virtual image preset by another mutually connected user.
(11)
The output control unit superimposes and displays the virtual image, which is the virtual image of the mutually connected other users and is output in a mode different from the mode of the real space, on the real image of the other user. , (1) to (10).
(12)
The output control unit causes the virtual image of the other user to be displayed in a manner different from that of the real space while the user is communicating with another user connected to each other in the real space. The information processing device according to any one of (1) to (11).
(13)
further comprising a voice information acquisition unit that acquires voice information related to the voice uttered by the user;
The communication control unit further includes an output control unit that controls content to be output to the virtual space according to the position information and the attribute information,
The information processing apparatus according to any one of (1) to (12), wherein the output control section controls an output position of the audio information acquired by the audio information acquisition section.
(14)
Controlling mutual communication between users via the virtual image of the user on the virtual space where the virtual image corresponding to the real image of the user existing in the real space is generated;
Acquiring at a predetermined timing attribute information including at least position-related information about the position of the user in real space and information about the degree of intimacy between the users;
An information processing method, comprising controlling switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position-related information and the attribute information.
(15)
to the computer,
A communication control unit that controls communication between users via the virtual image of the user in the virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
an information acquisition unit that acquires, at a predetermined timing, attribute information including at least position-related information about the position of the user in real space and information about the degree of intimacy between the users;
functioning as a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position-related information and the attribute information; program.

5 information processing system 10 information processing device 20 network 110 information acquisition unit 112 user information acquisition unit 114 motion information acquisition unit 116 voice information acquisition unit 120 switching control unit 130 communication control unit 140 recognition determination unit 150 communication control unit 160 storage unit 550 display Control device

Claims (12)

a communication control unit for controlling communication between users via the virtual image of the user in a virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
a user information acquisition unit configured to acquire, at a predetermined timing, attribute information including at least position information relating to the position of the user in real space and information relating to the degree of intimacy between the users;
a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information;
a motion information acquisition unit that acquires motion information related to the user's motion;
with
The switching control unit selects at least one other user from among the plurality of users according to the motion information acquired by the motion information acquisition unit, and selects at least one other user from the communication performed in the real space. switching to communication conducted in said virtual space;
Information processing equipment. further comprising a voice information acquisition unit that acquires voice information related to the voice uttered by the user;
The switching control unit establishes a network connection between the users according to at least one of the voice information of the user and the voice information of other mutually connected users, and communication is performed. 2. The information processing apparatus according to claim 1, wherein the space is switched from the real space to the virtual space. a communication control unit for controlling communication between users via the virtual image of the user in a virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
a user information acquisition unit configured to acquire, at a predetermined timing, attribute information including at least position information relating to the position of the user in real space and information relating to the degree of intimacy between the users;
a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information;
a recognition determination unit that recognizes other mutually connected users and acquires user recognition information related to user recognition ;
with
The communication control unit further comprises an output control unit that controls content to be output to the virtual space according to at least one of the position information and the attribute information,
The output control unit, after receiving the user recognition information transmitted from the recognition determination unit, causes the virtual image of the other user to be output in a predetermined expression method, and outputs the real image of the other user, synchronizing the virtual image of the other user;
Information processing equipment. 4. The information processing apparatus according to claim 3 , wherein said output control unit outputs said virtual images of other users with whom said users are mutually connected in a predetermined display method. a communication control unit for controlling communication between users via the virtual image of the user in a virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
a user information acquisition unit configured to acquire, at a predetermined timing, attribute information including at least position information relating to the position of the user in real space and information relating to the degree of intimacy between the users;
a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information;
with
The communication control unit further comprises an output control unit that controls content to be output to the virtual space according to at least one of the position information and the attribute information,
The output control unit causes the virtual images of the mutually connected other users to be output so as to indicate the real image direction of the other users.
Information processing equipment. a communication control unit for controlling communication between users via the virtual image of the user in a virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
a user information acquisition unit configured to acquire, at a predetermined timing, attribute information including at least position information relating to the position of the user in real space and information relating to the degree of intimacy between the users;
a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information;
with
The communication control unit further comprises an output control unit that controls content to be output to the virtual space according to at least one of the position information and the attribute information,
The output control unit erases the virtual images of other users connected to each other at the positions of the real images;
the switching control unit cancels the connection established between the users in the virtual space;
Information processing equipment. 6. The information processing apparatus according to claim 4 , wherein said output control unit outputs a virtual image preset by said other mutually connected users. a communication control unit for controlling communication between users via the virtual image of the user in a virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
a user information acquisition unit configured to acquire, at a predetermined timing, attribute information including at least position information relating to the position of the user in real space and information relating to the degree of intimacy between the users;
a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information;
with
The communication control unit further comprises an output control unit that controls content to be output to the virtual space according to at least one of the position information and the attribute information,
The output control unit superimposes and displays the virtual image, which is the virtual image of the mutually connected other users and is output in a mode different from the mode of the real space, on the real image of the other user. ,
Information processing equipment. a communication control unit for controlling communication between users via the virtual image of the user in a virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
a user information acquisition unit configured to acquire, at a predetermined timing, attribute information including at least position information relating to the position of the user in real space and information relating to the degree of intimacy between the users;
a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position information and the attribute information;
with
The communication control unit further comprises an output control unit that controls content to be output to the virtual space according to at least one of the position information and the attribute information,
The output control unit displays the virtual image of the other user in a mode different from the mode of the real space while the user is communicating with other interconnected users in the real space. let
Information processing equipment. further comprising a voice information acquisition unit that acquires voice information related to the voice uttered by the user;
The output control unit is
controlling content to be output to the virtual space according to the position information and the attribute information;
controlling an output position of the audio information acquired by the audio information acquiring unit;
The information processing device according to claim 5 . The information processing device
In a virtual space where a virtual image corresponding to a real image of a user existing in the real space is generated, controlling communication mutually implemented between users via the virtual image of the user;
Acquiring at a predetermined timing attribute information including at least position-related information about the position of the user in real space and information about the degree of intimacy between the users;
controlling switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position-related information and the attribute information ;
obtaining motion information about the user's motion;
including
Selecting at least one other user from among the plurality of users according to the acquired operation information, and switching from the communication performed in the real space to the communication performed in the virtual space. include,
Information processing methods. to the computer,
A communication control unit that controls communication between users via the virtual image of the user in a virtual space where a virtual image corresponding to the real image of the user existing in the real space is generated;
an information acquisition unit that acquires, at a predetermined timing, attribute information including at least position-related information about the position of the user in real space and information about the degree of intimacy between the users;
a switching control unit that controls switching between communication performed in the real space and communication performed in the virtual space according to at least one of the position-related information and the attribute information ;
a motion information acquisition unit that acquires motion information related to the user's motion;
A program for functioning as
The switching control unit selects at least one other user from among the plurality of users according to the motion information acquired by the motion information acquisition unit, and selects at least one other user from the communication performed in the real space. switching to communication conducted in said virtual space;
program.

Images