JP2021002318A - Virtual space providing system, virtual space providing method, and program - Google Patents

Abstract

To provide a virtual space providing system, method, and a program that can strengthen the relationship between users communicating through a virtual space and can improve the quality of the users' recognition experience.SOLUTION: In a virtual space providing system, a VR computer comprises: an object association unit that associates at least one object with each of a plurality of users; an object specification unit that specifies a second object associated with a second user of the plurality of users, the second object capable of communicating with a first object associated with a first user of the plurality of users; a virtual space definition unit that defines a virtual space including the first object and a virtual viewpoint of the specified second object; an acquisition unit that acquires expression information on respective expressions given by the first user and the second user; and an object control unit that controls, based on the acquired expression information, the first object and the second object according to the expressions.SELECTED DRAWING: Figure 5

Description

The present invention relates to a virtual space providing system, a virtual space providing method, and a program for providing a virtual space.

Non-Patent Document 1 discloses a technique in which a talent object and a fan object associated with each of an unspecified number of fans who are enthusiastic enthusiasts of talent can interact with each other in a virtual space. There is.

Internet <URL: https://twitter.com/wahekui_1/status/1119961062613237766>

In the conventional virtual space providing technology as described in Non-Patent Document 1, since the talent object and an unspecified number of fan objects interact with each other in the virtual space, the talent object and a certain virtual fan interact with each other. Even if you try, you may be disturbed by other fan objects. In such a case, there is a possibility that sufficient communication cannot be obtained between one talent object and one specific fan object, and the relationship between the two cannot be sufficiently constructed. Fans also want the talents they support to remember through communication in virtual space. However, if other fan objects interfere with the interaction with the talent object, such a request cannot be met.

Therefore, the present disclosure provides a virtual space providing technology capable of strengthening the relationship between users who communicate via the virtual space and improving the quality of the user's cognitive experience. With the goal.

The virtual space providing system according to one aspect of the present invention is a virtual space providing system that provides a virtual space, and includes an object association unit that associates at least one object with each of a plurality of users, and a plurality of users. First associated with the first user of
A virtual space that includes an object identification unit that identifies a second object associated with a second user among a plurality of users that can communicate with the object, a first object, and a virtual viewpoint of the identified second object. The virtual space definition part that defines the first
It includes an acquisition unit that acquires representation information related to each representation of the user and the second user, and an object control unit that controls the first object and the second object according to the representation based on the acquired representation information.

The virtual space providing method according to one aspect of the present invention is a virtual space providing performed by a computer that provides a virtual space, and includes a step of associating at least one object with each of a plurality of users and a method of providing a plurality of users. A step to identify a second object associated with a second user among a plurality of users that can communicate with the first object associated with our first user, a first object, and the identified second object. A step of defining a virtual space including the virtual viewpoint of the above, a step of acquiring expression information regarding each expression of the first user and the second user, and a first object and a second object based on the acquired expression information. Includes steps to control the object according to its representation.

In the program according to one aspect of the present invention, a computer that provides a virtual space is associated with an object association unit that associates at least one object with each of a plurality of users, and a first user among the plurality of users. Includes an object identification unit that identifies a second object associated with a second user among a plurality of users who can communicate with the first object, a first object, and a virtual viewpoint of the identified second object. The virtual space definition unit that defines the virtual space, the acquisition unit that acquires the expression information related to the expressions of the first user and the second user, and the first object and the second object are expressed based on the acquired expression information. It functions as an object control unit that controls according to.

In the present invention, the "part" and "device" do not simply mean a physical means, but also include a case where the functions of the "part" and "device" are realized by software.
Further, even if the functions of one "part" or "device" are realized by two or more physical means or devices, the functions of two or more "parts" or "devices" are realized by one physical means or device. It may be realized by the device.

According to the above aspect of the present invention, it is possible to strengthen the relationship between users who communicate via the virtual space, and to improve the quality of the user's cognitive experience.

It is a schematic block diagram of the virtual space provision system which concerns on embodiment of this invention. It is a figure which shows an example of the image displayed on the operator terminal which concerns on embodiment of this invention. (A) is an image displayed on a talent terminal according to an embodiment of the present invention, and is a diagram showing an example of an image when a talent object and one fan object communicate with each other in a virtual space. (B) is an image displayed on the talent terminal according to the embodiment of the present invention, and is a diagram showing an example of an image when the talent object and a plurality of fan objects can communicate with each other in a virtual space. It is a schematic diagram of VR (V irtual R eality) computers according to embodiments of the present invention. It is a schematic block diagram which shows an example of the functional structure of the VR computer which concerns on embodiment of this invention. It is a figure which showed conceptually which shows an example of the virtual space which concerns on embodiment of this invention. It is a flowchart which shows an example of the virtual space provision processing which concerns on embodiment of this invention. It is a figure for demonstrating an example of the handshake meeting in the virtual space which concerns on embodiment of this invention. (A) is a figure which shows an example of the handshake operation of a talent and a fan in a real space. (B) is a diagram showing an example of an image displayed by the HMD according to the embodiment (H ead M ounted D isplay) of the present invention. It is a figure for demonstrating an example of the high touch meeting in the virtual space which concerns on embodiment of this invention. (A) is a diagram showing an example of high-touch operation of a talent and a fan in a real space. (B) is a figure which shows an example of the image displayed by the HMD which concerns on embodiment of this invention. It is a figure for demonstrating another example of the handshake meeting in the virtual space which concerns on embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and there is no intention of excluding the application of various modifications and techniques not specified below. That is, the present invention can be implemented with various modifications (combining each embodiment, etc.) within a range that does not deviate from the gist thereof. Further, in the description of the following drawings, the same or similar parts are designated by the same or similar reference numerals. The drawings are schematic and do not necessarily match the actual dimensions and ratios. Even between drawings, parts with different dimensional relationships and ratios may be included.

FIG. 1 is a block diagram showing an overall configuration of a virtual space providing system 100 according to an embodiment of the present invention. First, an outline of the virtual space providing system 100 in the present embodiment will be described.
HMD (H ead M ounted D isplay ) 5a shown in FIG. 1, is an avid enthusiast for talent TL to stay in the studio, to the fan Fa to stay in the local,
Virtual viewpoint of the talent object TLO associated with the talent TL and the fan object FO1 associated with the fan Fa (for example, corresponding to the camera provided in the HMD5a).
A virtual space image VG based on a virtual space including and is provided. In the example of FIG. 1, a virtual space image VG showing a situation in which the talent object TLO and the fan object FO1 are shaking hands from the viewpoint of the talent object TLO on the display unit 30 included in the HMD 5.
Is displayed. Virtual space image VG is VR (V irtual R eality) image generated by computer 1.

In the example of FIG. 1, the number of fans Fa and Fb is two, but the number is not limited to this, and the number of fans may be one or three or more. More specific processing will be described. For example, the VR computer 1 communicates with the talent object TLO (first object) by dialogue or operation in the virtual space among the fan objects associated with the fans Fa and Fb.
Fan object F corresponding to fan Fa as a partner who can communicate)
Identify O1 (second object). Then, the VR computer 1 defines a virtual space including the talent object TLO and the virtual viewpoint of the specified fan object FO1. "Communication" may include both fan F and talent TL interacting by voice or by action through objects associated with each. Also, "communication" may include one of the fan F or the talent T simply observing the other's actions or simply listening to the other's remarks.

Here, a concrete example of "communication" between a fan and a talent is given. "Communication" includes, for example, a handshake event in which a talent object and a specific fan object can enjoy a conversation while shaking hands in a virtual space, as shown in a virtual space image VG. In addition, "communication" is a high-touch meeting in which a talent object and a specific fan object make a high touch in the virtual space, for example, a send-off party in which the fan object sees off the talent object after a live performance in the virtual space, and a talent object in the virtual space. Includes photo sessions where specific fan objects take pictures. There are various types of handshake events such as a conveyor belt handshake event in which multiple fan objects automatically move in a virtual space as if they were on a conveyor belt and shake hands with the talent objects in order. Including.

The fan object may hold an item at a live performance by a talent or a handshake event held in a virtual space. The item includes, for example, a psyllium of a particular color, or a gift to a talent such as a bouquet. The item may also be something worn by the fan object, such as a T-shirt for a talent.

"Object" includes an avatar based on talent TL or fan F. For example, the avatar includes an image showing a humanoid object that imitates the talent TL or fan F, or corresponds to the talent TL or fan F. The object may be a simple character icon representing a talent TL or a fan F with two heads, or an elaborate character icon. The object is at least a part of the body of the talent TL or fan F (eg whole body, head, hands,
It may include an object corresponding to (feet, etc.). Further, the avatar may include an image imitating an animal or the like associated with the talent TL or the fan F.

The VR computer 1 acquires the expression information of the talent TL and the expression information of the fan Fa as the communication partner of the talent TL. "Expression information" includes information indicating the emotions and thoughts of talent TL and fan F in attitudes, languages, and the like. The expression information includes, for example, at least one of the operation of the talent TL and the fan F or the voice. The expressive information may include information on facial expressions and mouth movements indicating smiles, smiles, or sadness.

The VR computer 1 generates a virtual space image in which the expressions of the talent TL and the fan Fa are reflected, based on the acquired expression information. That is, the VR computer 1 is a talent T.
A virtual space image is generated each time the expressions of L and fan Fa change.

Therefore, the fan Fa can communicate with the talent TL while checking the virtual space image VG displayed on the display unit 30 of the HMD5a for the operation of the object in the virtual space. In addition to utilizing the movement of the object in the virtual space, the fan Fa can communicate with the talent TL using voice. For example, the voice of the talent TL acquired by the microphone 3 is transmitted to the operator terminal 4 as voice information. The voice information of the talent TL is transmitted to the HMD5a via the VR computer 1. The speaker (not shown) included in the HMD5a outputs the sound of the talent TL. On the other hand, the sound of the fan Fa acquired by the microphone (not shown) provided in the HMD 5a is transmitted to the VR computer 1 as voice information. The voice information of the fan Fa is the operator terminal 4
Is transmitted to the earphone monitor 10 used by the talent TL. Earphone monitor 10
Outputs the sound of the fan Fa.

In this way, the virtual space providing system 100 reflects the operations of the fan Fa and the talent TL in the real space, and the fan object FO1 and the talent object TLO are reflected.
Provides a virtual space image that can grasp the operation of. Further, the virtual space providing system 100 provides the sounds of both the fan object FO1 and the talent object TLO, which are output so as to be synchronized with the operation of the fan object FO1 and the talent object TLO.

As described above, in the virtual space providing system 100, the fan object that can communicate with the talent object is specified, and the virtual space including the talent object and the virtual viewpoint of the specified fan object is defined. Then, in the defined virtual space, the talent object and at least a part of the fan objects (whole body object, hand object, head object, foot object, etc.) are controlled based on the expression information. Therefore, when both parties communicate with each other, it is possible to prevent them from being disturbed by other fans (fan objects). Therefore, it is possible to strengthen the relationship between users who communicate via the virtual space, and to improve the quality of the user's cognitive experience.
Hereinafter, a specific configuration of the virtual space providing system 100 will be described.

As shown in FIG. 1, the virtual space providing system 100 typically acquires 3D image information (image information) generated by imaging a predetermined range including the talent TL, and audio information of the camera 2c and the talent TL. The microphone 3 for acquiring the image and the earphone monitor 10 for outputting the sound of the fan F are provided. The virtual space providing system 100 is exemplified by the talent T from the microphone 3.
An operator terminal 4 that receives the audio information of the L and receives the image information of the talent TL from the camera 2c, and is an operator that transfers the audio information and the image information of the talent TL to the VR computer 1 via the communication network N. The terminal 4 is provided. Communication network N
Is, for example, the Internet, such as a cellular telephone network network, LAN (L ocal
It is realized by A rea N etwork), or a network of a combination of these.

FIG. 2 is a diagram showing an example of an image displayed on the operator terminal 4 according to the embodiment of the present invention. The operator O shown in FIG. 1 is a person who manages and operates the virtual space providing system 100. For example, the operator O can select a fan capable of communicating with the talent TL and control the communication content of both by operating the operator terminal 4.

As shown in FIG. 2, in the display unit 400 of the operator terminal 4 operated by the operator O, a plurality of facial expression selection buttons B1 capable of selecting the facial expression content of the talent object TLO corresponding to the facial expression information of the talent TL, and the talent A scroll bar SB for controlling how much volume the sound of the fan F is output from the earphone monitor 10 used by the TL is displayed.

In the display unit 400, the start button B3 for starting the communication between the specific fan object and the talent object TLO and the forced end button B5 for forcibly ending the communication between the specific fan object and the talent object TLO. And the period (for example, remaining time) TM in which the specific fan object and the talent object TLO can communicate with each other are displayed. The instruction of the operator O (for example, pressing each button B1, B3, B5, operating the scroll bar SB, etc.) input in the operator terminal 4 is sent to the VR computer 1 as instruction information via the communication network N shown in FIG. Will be sent.

The communication period of the talent object TLO and the fan object in the virtual space can be changed by the operator O operating the operator terminal 4. For example, the preset communication period may be changed. The time remaining after the communication is started may be increased or decreased. In this case, the operator terminal 4 transmits instruction information based on the operation of the operator O to the VR computer 1, and the VR computer 1 executes the change processing of the communication period.

When there are a plurality of fan objects that may communicate with the talent object TLO, the order in which the operator O communicates can be changed by operating the operator terminal 4. In this case, the operator terminal 4 transmits instruction information based on the operation of the operator O to the VR computer 1, and the process of changing the order of communication in the VR computer 1 is executed.

Returning to FIG. 1, the virtual space providing system 100 typically provides a virtual space including a talent object TLO associated with a talent TL and a virtual viewpoint of a fan object associated with a fan F. It also includes an HMD 5 that displays a virtual space image based on the virtual space provided by the VR computer 1. When the fans Fa and Fb are described without distinction, some reference numerals are omitted and they are simply referred to as "fan F". HM
When explaining D5a and 5b without distinguishing them, some reference numerals are omitted and simply "HMD" is used.
It is called "5". In the example shown in FIG. 1, the VR computer 1 is arranged outside the HMD 5, but the present invention is not limited to this configuration, and the VR computer 1 may be built in the HMD 5.
That is, the HMD 5 may have each function of the VR computer 1 described later.

The HMD 5 is attached to the head of the fan F, for example, and displays a virtual space image (for example, a three-dimensional image) based on the virtual space corresponding to the movement of the head of the fan F. The display unit 30 of the HMD 5 can display virtual space image information, text information, or the like transmitted from the VR computer 1. The display unit 30 may be arranged so as to be located in front of the eyes of the fan F. Display unit 30 is, for example, a liquid crystal display or an organic EL (E lectro L uminescence
) Including display. Further, the display unit 30 may include a transmissive type or semi-transparent type display device. The display unit 30 may simultaneously display at least a part of the virtual space image and a predetermined range of the real space captured by a camera (not shown) provided in the HMD 5. Although the function of the display unit 30 of the HMD 5 is not shown, a display unit provided in a mobile terminal such as a smartphone or a laptop computer separate from the HMD 5 may be substituted. In addition, the display unit 3 of the HMD 5
Although the function of 0 is not shown, a stationary display device such as a display monitor separate from the HMD 5 may be substituted.

As an example, the virtual space providing system 100 captures local video information obtained by capturing a monitoring terminal 6 used by a staff member Sa staying at the site and a state of the site using a camera 2b, via a communication network N. Local video distribution terminal 7 to be distributed to the video display terminal 8,
Local video display terminal 8, which receives and displays local video information distributed from the local video distribution terminal 7.
Further, the talent terminal 9 for displaying a virtual space image for the talent TL itself to confirm how the talent object associated with the talent TL communicates with the fan object in the virtual space is further provided.

For example, when a plurality of fans F equipped with the HMD 5 are present in the monitoring terminal 6, a virtual space image based on the virtual viewpoint of the fan object of the fan Fa and a virtual viewpoint of the fan object of the fan Fb are used as a reference. The virtual space image and the above can be displayed. Staff S
Since a can confirm each of the virtual space images visually recognized by each fan, it is possible to confirm whether or not there is a defect in the mounting state of the HMD 5 of each fan F, for example, the orientation or inclination of the HMD 5.

FIG. 3A is an image displayed on the display unit 900 of the talent terminal 9 shown in FIG.
It is a figure which shows an example of the virtual space image VG1 when the talent object TLO and one fan object FO1 communicate with each other in a virtual space. FIG. 3 (b) shows
It is a figure which shows an example of the virtual space image VG2 which is the image displayed on the display part 900 of a talent terminal 9, and is the case where the talent object TLO and a plurality of fan objects FO1 and FO2 can communicate in order in a virtual space. .. 3 (a) and 3 (b)
), The virtual space image displayed on the display unit 900 of the talent terminal 9 is a plurality of virtual space images corresponding to the field of view from various positions (virtual viewpoints) in the virtual space corresponding to a certain handshake event venue. May include. The virtual space image displayed on the display unit 900 of the talent terminal 9 may be an image generated by the VR computer 1 and transmitted from the VR computer 1. Further, the virtual space image displayed on the display unit 900 of the talent terminal 9 may be an image generated by the talent terminal 9.

The display unit 900 of the talent terminal 9 displays fan information related to the fan F currently communicating with the talent TL via the virtual space, for example, a fan ID, a fan name, a fan face icon, and a fan emotion. , The color of the psyllium owned by the fan, the number of live participations of the fan, the presence or absence of the gift sent by the fan to the talent TL, the degree of charge of the fan, etc. are displayed. The fan information may be displayed together with the virtual space image, or may be displayed instead of the virtual space image.

FIG. 4 is a block diagram showing an example of the hardware configuration of the VR computer according to the embodiment of the present invention. As shown in FIG. 4, the VR computer 1 is exemplified by a CPU ( Ce).
and ntral P rocessing U nit) 11, ROM (R ead O nly
A memory 12 consisting of M emory) and RAM (R andom A ccess M emory ) , etc., an input unit 13, an output unit 14, a recording unit 15, a communication unit 16, a bus 17,
Is equipped with.

The CPU 11 executes various processes according to the program recorded in the memory 12 or the program loaded into the memory 12 from the recording unit 15.

Data and the like necessary for the CPU 11 to execute various processes are also appropriately stored in the memory 12. The CPU 11 and the memory 12 are connected to each other via the bus 17. Further, the input unit 13, the output unit 14, the recording unit 15, and the communication unit 16 are connected to the bus 17.

The input unit 13 inputs various information according to an instruction operation of a person who operates the VR computer 1 (for example, staff Sa or a staff in the field). The input unit 13 is composed of, for example, various buttons, a touch panel, a microphone, or the like. The input unit 13 may be realized by an input device such as a keyboard or a mouse, which is independent of the main body that houses the other parts of the computer 1.

The output unit 14 outputs image data and audio data. The output unit 14 is composed of, for example, a display, a speaker, or the like. The image data and audio data may be recognizable by the user as images and music from a display, a speaker, or the like.

The recording unit 15 stores various data. Recording unit 15, for example, DRAM (D ynam
ic R andom A ccess M emory) consists of a semiconductor memory or the like.

The communication unit 16 realizes communication with other devices. For example, the communication unit 16 communicates with other terminals via the network N.

Although not shown, a drive is appropriately provided in the VR computer 1 as needed. For example, a removable medium composed of a magnetic disk, an optical disk, a magneto-optical disk, or the like may be appropriately mounted on the drive. Various data such as programs and image data read from the removable media by the drive are installed in the recording unit 15 as needed.

The operator terminal 4, the monitoring terminal 6, the local video distribution terminal 7, the local video display terminal 8, or the talent terminal 9 may include at least a part of each of the above configurations included in the VR computer 1.

FIG. 5 is a block diagram showing an example of the functional configuration of the VR computer according to the embodiment of the present invention. By way of example, the computer 1 includes an information processing unit 111 that executes a process for providing a virtual space, and a database (DB) that records various information necessary for executing the process and information on the result of the process. 112 and. Note that the information processing unit 111 uses, for example, C the program stored in the memory 12 and the recording unit 15 shown in FIG.
This can be achieved by executing the PU 11. The DB 112 corresponds to, for example, the recording unit 15 shown in FIG.

The information processing unit 111 functionally includes an object association unit 113 that associates at least one object with each of a plurality of users including a talent and one or a plurality of fans, and a talent among the plurality of users (first). An object identification unit 114 that identifies a fan object (second object) associated with a fan (second user) among a plurality of users, which can communicate with the talent object (first object) associated with the user), and To be equipped with. The information processing unit 111 functionally provides the virtual space definition unit 115 that defines the virtual space including the talent object and the virtual viewpoint of the specified fan object, and at least the expression information regarding each representation of the talent and the fan. It includes an acquisition unit 116 to acquire, and an object control unit 117 that controls a talent object and a fan object according to the expression based on the acquired expression information.
Details of each of the above functions will be described later.

The information processing unit 111 further includes an image generation unit 118. The image generation unit 118 generates a virtual space image based on the virtual space based on the virtual space data.

FIG. 6 is a conceptual diagram showing an example of a virtual space according to an embodiment of the present invention. Figure 6
As shown in, the virtual space 50 defined by the virtual space definition unit 115 includes, for example, a hemisphere with respect to the center point CP. In the virtual space 50, an XYZ coordinate system with the center point CP as the origin is defined. For example, when the HMD 5 shown in FIG. 1 is activated, a virtual camera 60 (virtual viewpoint) (for example, corresponding to a camera included in the HMD 5) is arranged at the center point CP of the virtual space 50. An image is displayed on the display unit 30 of the HMD 5 based on the image information acquired by the virtual camera 60 taking an image. As shown in FIG. 1, the virtual camera 60 can move according to the movement (change in position and inclination) of the HMD 5 mounted on the head of the fan F in the real space. That is, the virtual camera 60 in the virtual space 50 moves so as to correspond to the movement of the HMD 5 in the real space.

Here, the position of the virtual camera 60 corresponds to the viewpoint of the fan F (user) in the virtual space 50. The image generation unit 16 defines the field of view area 80 in the virtual space 50 based on the reference line-of-sight SG based on the virtual camera 60. For example, the field of view 80 corresponds to the area of the virtual space 50 that is visually recognized by the fan F equipped with the HMD 5.

In this way, the image generation unit 16 arranges, for example, the virtual camera 60 at the center point CP in the virtual space 50, and sets the line of sight (reference line of sight SG) of the virtual camera 60 in the direction in which the fan F is facing. Therefore, the image generation unit 16 generates the field of view image data for generating the field of view image (virtual space image VG3). After that, when the movement of the HMD 5 changes, the image generation unit 16 generates a new virtual space image according to the change.

The virtual camera 60 is not limited to the HMD5 camera mounted on the head of the fan F in FIG. For example, the virtual camera 60 may correspond to the camera of the HMD 5 when the HMD 5 is attached to the head of the talent. As shown in FIGS. 3A and 3B, virtual space images corresponding to the fields of view from various viewpoints (virtual cameras) in a specific virtual space may be generated.

Returning to FIG. 5, the DB 112 exemplifies the spatial information SI including one or more virtual space data templates for defining the virtual space 50 shown in FIG. 6, the object information OI about the object, and the expression information EI. , Record. In the DB 112, the above-mentioned various information may be associated and recorded for each user based on the information about the user.

<Virtual space provision processing>
An example of the virtual space providing process according to the embodiment of the present invention will be described with reference to FIGS. 7 to 10.
FIG. 7 is a flowchart showing an example of the virtual space providing process according to the embodiment of the present invention.

(Step S1)
The object association unit 113 of the VR computer 1 shown in FIG. 5 associates an object with each of a plurality of users. With reference to the examples of FIGS. 1 and 3, the object association unit 113 sets the talent object TLO for the talent TL and the fan Fa for example based on the object information OI shown in FIG. The fan object FO1 is associated with the fan Fb, and the fan object FO2 is associated with the fan Fb.

(Step S3)
The object identification unit 114 of the VR computer 1 shown in FIG. 5 has a fan object FO associated with the fan F among the plurality of users capable of communicating with the talent object TLO associated with the talent TL among the plurality of users. Identify.
With reference to the examples of FIGS. 1 and 3, the object identification unit 114 is the talent object T.
The fan object FO1 associated with the fan Fa is specified as the LO communication partner.

(Step S5)
The virtual space definition unit 115 of the VR computer 1 shown in FIG. 5 has a talent object TLO and a virtual viewpoint of the specified fan object FO1 (for example, the HMD5a used by the fan Fa) as in the example of the virtual space shown in FIG. Defines a virtual space that includes (corresponding to a equipped camera).

(Step S7)
The acquisition unit 116 of the VR computer 1 shown in FIG. 5 acquires at least the representation information regarding the representation of the talent TL and the fan Fa. First, the camera 2c shown in FIG. 1 is a talent TL.
By imaging a predetermined range including the above, 3D image information including the operation of the talent TL and the like is generated and output to the operator terminal 4 as expression information. The operator terminal 4 displays the expression information.
It is transmitted to the VR computer 1 via the communication network N. Further, the camera 2a generates 3D image information including the operation of the fan Fa by photographing a predetermined range including the fan Fa, and outputs the 3D image information including the operation of the fan Fa to the VR computer 1 as expression information. The microphone 3 acquires the voice of the talent TL and outputs the voice information as expression information to the operator terminal 4. The operator terminal 4 transmits the expression information to the VR computer 1 via the communication network N. Also,
For example, the microphone (not shown) included in the camera 2a acquires the sound of the fan Fa and outputs the sound information to the VR computer 1 as expression information. The VR computer 1 may include a microphone for acquiring the sound of the fan as a configuration separate from the camera 2a.

In this way, the acquisition unit 116 acquires the 3D image information and the audio information of the talent TL as the expression information of the talent TL, and acquires the 3D image information and the audio information of the fan Fa as the expression information of the fan Fa. .. The acquisition unit 116 obtains the expression information of the talent TL and the fan Fa.
It may be acquired regularly or at any time.

(Step S9)
The object control unit 117 of the VR computer 1 shown in FIG. 5 controls the talent object and the fan object according to the expression based on the expression information based on the acquired expression information.

FIG. 8 is a diagram for explaining an example of a handshake event in the virtual space according to the embodiment of the present invention. FIG. 8A is a diagram showing an example of a handshake operation of a talent and a fan in a real space. FIG. 8B is a diagram showing an example of an image displayed by the HMD according to the embodiment of the present invention. As shown in FIG. 8A, when the talent TL turns its hand H1 downward and the fan Fa turns its hand H2 downward and shakes hands with each other in the real space, FIG. As shown in 8 (b), the handshake operation of the talent TL and the fan Fa is reflected in the talent object TLO and the fan object FO1 in the virtual space.

Specifically, the talent object TLO includes an object (third object) corresponding to at least a part of the body of the talent TL. The object corresponding to at least a part of the body of the talent TL comprises a hand object HO1 (first hand object) corresponding to the hand of the talent TL. The fan object FO1 includes an object (fourth object) corresponding to at least a part of the body of the fan Fa. The object corresponding to at least a part of the body of the fan Fa is the hand object HO corresponding to the hand of the fan Fa.
2 (second hand object) is included.

The object control unit 117 controls the contact state of the hand object HO1 and the hand object HO2 according to the orientation or inclination of the hand object HO1 and the hand object HO2 when the hand object HO1 and the hand object HO2 come into contact with each other in the virtual space. To do. More specifically, the object control unit 117 determines the contact state of the hand object HO1 and the hand object HO2 when the hand object HO1 and the hand object HO2 satisfy a predetermined condition when the hand object HO1 and the hand object HO2 come into contact with each other. Control to maintain for a predetermined period.

For example, the object control unit 117 has a hand object HO1 and a hand object HO.
When the hand object HO1 and the hand object HO2 when the two come into contact with each other are located below a predetermined reference position, the talent object and the specific fan object are controlled to shake hands. The predetermined reference position for the talent object TLO and the fan object FO to shake hands may be a position where the handshake operation performed by both objects looks natural. The predetermined reference position may be, for example, a position corresponding to the shoulder (shoulder object) of the talent object TLO with respect to the hand object HO1, and the shoulder of the fan object FO with respect to the hand object HO2. It may be a position corresponding to (shoulder object). The predetermined reference position is not limited to the above, and may be a position corresponding to the chest of the talent object TLO and the fan object FO.

For example, the predetermined reference position is the talent object TLO and the fan object FO.
Different positions may be set in. For example, the reference position of the talent object TLO may be a position corresponding to the shoulder, and the reference position of the fan object FO may be a position corresponding to the chest.

The object control unit 117 may consider other conditions as conditions for executing the handshake operation. For example, in the object control unit 117, among the plurality of finger objects included in each hand object HO1 and HO2, the height of the first joint (the first joint counting from the fingertip) of the middle finger object is the second joint (the second joint of the middle finger). If it is lower than the height of the second joint counting from the fingertip), the talent object and a specific fan object may be controlled to shake hands. It should be noted that, among the plurality of finger objects, a finger other than the middle finger may be adopted to determine whether or not the condition for executing the handshake operation is satisfied.

According to this configuration, it is possible to realize a handshake event in which a talent object and a specific fan object can enjoy a conversation while shaking hands in a virtual space. Therefore, the relationship between users who communicate via the virtual space can be further strengthened, and the quality of the user's cognitive experience can be further improved. Further, by further considering the position of the finger object in addition to the position of the hand object, it is possible to more accurately determine whether or not to execute the handshake operation.

“When the hand object HO1 and the hand object HO2 come into contact with each other” includes before the two objects touch (for example, immediately before), when both objects touch each other, or after both objects touch (for example, immediately after). The "predetermined period" is not particularly limited, and may be, for example, several seconds or several tens of seconds. The predetermined period can be changed, for example.
The predetermined period may be changed by the operator O shown in FIG. 1 operating the operator terminal 4.

FIG. 9 is a diagram for explaining an example of a high touch meeting in the virtual space according to the embodiment of the present invention. FIG. 9A is a diagram showing an example of high-touch operation of a talent and a fan in a real space. FIG. 9B is a diagram showing an example of an image displayed by the HMD according to the embodiment of the present invention. As shown in FIG. 9A, when the talent TL turns its hand H1 upward and the fan Fa turns its hand H2 upward in the real space and performs a high five operation with both of them, FIG. As shown in 9 (b), the high-touch operation of the talent TL and the fan Fa is the talent object TLO and the fan object F in the virtual space.
It is reflected in O1.

When the hand object HO1 and the hand object HO2 satisfy a predetermined condition when the hand object HO1 and the hand object HO2 come into contact with each other, the object control unit 117 immediately after the hand object HO1 and the hand object HO2 come into contact with each other, the hand object HO1
And the hand object HO2 is controlled to be separated.

For example, the object control unit 117 has a hand object HO1 and a hand object HO.
When the hand object HO1 and the hand object HO2 when the two come into contact with each other are located above a predetermined reference position, the talent object and the specific fan object are controlled to give a high five. The predetermined reference position for the talent object TLO and the fan object FO to high-five may be a position where the high-five operation performed by both objects looks natural. The predetermined reference position is, for example, the hand object HO.
1 may be a position corresponding to the shoulder (shoulder object) of the talent object TLO, and hand object HO2 may be a position corresponding to the shoulder (shoulder object) of the fan object FO. May be good. The predetermined reference position is not limited to the above, and may be a position corresponding to the chest of the talent object TLO and the fan object FO. For example, a predetermined reference position may be set differently between the talent object TLO and the fan object FO. For example, the reference position of the talent object TLO may be a position corresponding to the shoulder, and the reference position of the fan object FO may be a position corresponding to the chest.

The object control unit 117 may consider other conditions as conditions for executing the handshake operation. For example, when the height of the first joint of the middle finger object is higher than the height of the second joint of the middle finger object among the plurality of finger objects included in each hand object HO1 and HO2, the object control unit 117 sets the object as a talent object. It may be controlled to make a high touch with a specific fan object. It should be noted that, among the plurality of finger objects, a finger other than the middle finger may be adopted to determine whether or not the condition for executing the handshake operation is satisfied.

According to this configuration, it is possible to realize a high-touch meeting in which a talent object and a specific fan object can enjoy a conversation or the like while performing a high-five in a virtual space. Therefore, the relationship between users who communicate via the virtual space can be further strengthened, and the quality of the user's cognitive experience can be further improved. Further, by further considering the position of the finger object in addition to the position of the hand object, it is possible to determine whether or not to execute the high touch operation with higher accuracy.

FIG. 10 is a diagram for explaining an example of a belt conveyor type handshake event in the virtual space according to the embodiment of the present invention. As shown in FIG. 10, in the belt conveyor type handshake event, as if a plurality of fan objects FO0 to FO7 are on the belt conveyor, for example, the talent objects are automatically moved in the direction indicated by the broken line arrow A. You can shake hands and have a conversation with the TLO. Each of the fan objects FO0 to FO7 can communicate when the distance from the talent object TLO becomes a constant distance L. For example, in the example of FIG. 10, the object identification unit 114 shown in FIG. 5 calculates the distance between the talent object TLO and each of the plurality of fan objects FO0 to FO7, and is located at a constant distance L from the talent object TLO. Identify the fan object FO1. The virtual space definition unit 115 defines a virtual space in which the talent object TLO and the fan object FO1 can communicate with each other. The distance L can be changed as appropriate.

According to this configuration, the communication between the talent object and the fan object can be automated, the load on the operator can be reduced, and the number of fans who can communicate with the talent can be increased.

According to the embodiment described above, a fan object that can communicate with the talent object is specified, and a virtual space including the talent object and the virtual viewpoint of the specified fan object is defined. Then, in the defined virtual space, the talent object and at least a part of the fan objects are controlled based on the expression information. Therefore, when both parties communicate with each other, it is possible to prevent them from being disturbed by other fans (fan objects). Therefore, it is possible to strengthen the relationship between users who communicate via the virtual space, and to improve the quality of the user's cognitive experience.

(Other embodiments)
Each of the above embodiments is for facilitating the understanding of the present invention, and does not limit the interpretation of the present invention. The present invention can be modified / improved (for example, combining each embodiment, omitting a part of the configuration of each embodiment) without departing from the spirit thereof, and the present invention also includes an equivalent thereof. Is done.

As shown in FIGS. 3 (b) and 8 (b), fan identification information (for example, fan name text information T1, T) is located in the vicinity of the fan object associated with the fan in the virtual space.
2, T3) may be displayed. Specifically, the acquisition unit 116 shown in FIG. 5 acquires the fan identification information, and the object control unit 117 bases the acquired identification information on the fan corresponding to the fan object in the vicinity of the fan object in the virtual space. Place the identification information of. The fan identification information may be stored in the database 112 in advance, or may be separately input by the operator O or the like.

In the example of FIG. 1, in the virtual space providing system 100, the VR computer 1 is commonly used by a plurality of HMDs 5, and usually, a separate virtual space can be defined for each different fan. A different VR computer may be used for each of the plurality of HMD5s. Further, the virtual space providing system 100 has a talent object TLO associated with the talent TL and a specified fan object FO1 associated with the fan Fa.
Provides a virtual space image based on a virtual space including a virtual viewpoint of. However, the virtual space providing system 100 provides a virtual space image further including a fan object of a fan different from the fan Fa in response to a request from the operator O, the staff Sa, the talent TL, or the fan F (for example, It is possible to switch the displayed virtual space image).

The virtual space providing system 100 includes a VR computer 1, an operator terminal 4, a monitoring terminal 6, a local video distribution terminal 7, a local video display terminal 8, and a talent terminal 9. However, the virtual space providing system 100 may include a device that integrates two or more of each of these devices. For example, the local video display terminal 8 and the talent terminal 9 used by the talent TL.
May be a single terminal device.

In each of the above embodiments, communication between one talent TL and one or more fans F has been described, but the present invention is not limited to these embodiments. For example, the present invention is also applicable to communication between a plurality of talent TLs and a plurality of fans F. That is, when a plurality of talent objects TLO and a plurality of fan objects FO are arranged in a certain virtual venue, for example, a virtual space X for the talent X object and the fan X object to communicate with each other is provided in the virtual venue. Moreover, it is possible to provide a virtual space Y different from the virtual space X for the talent Y object different from the talent X object and the fan Y object different from the fan X object to communicate with each other.

As described above, the operation (expression information) of the talent TL and the fan F is acquired by using the cameras provided around each of the talent TL and the fan F, but the present invention is not limited to this form. The operation of the talent TL and the fan F may be acquired by using a controller (not shown) that can be gripped by each of the talent TL and the fan F. The controller is talent TL
Alternatively, the fan F may be configured to be attached to the body or a part of clothing. The controller may be a glove-type controller corresponding to the hand of the talent TL or the fan F, or a stick-type controller. For example, the controller connected to the VR computer 1 shown in FIG. 1 by wire or wirelessly includes a motion sensor. For example, the controller may accept operations from the talent TL or fan F to control the position or movement of at least a part of the objects placed in the virtual space. More specifically, the controller is attached to, for example, the hand of the talent TL or the fan F to detect the movement of the hand of the talent TL or the fan F. For example, the motion sensor provided in the controller is the number of rotations of the hand,
Or, the rotation speed and the like are detected. The detection result is transmitted to the VR computer 1 as a detection signal.

1 ... VR computer, 2a, 2b, 2c ... camera, 3 ... microphone, 4 ... operator terminal,
5a, 5b ... HMD, 6 ... Surveillance terminal, 7 ... Local video distribution terminal, 8 ... Local video display terminal, 9
... talent terminal, 10 ... earphone monitor, 11 ... CPU, 12 ... memory, 13 ... input unit,
14 ... Output unit, 15 ... Recording unit, 16 ... Communication unit, 17 ... Bus, 30, 400, 900 ... Display unit, 100 ... Virtual space providing system, 111 ... Information processing unit, 112 ... Database (DB)
), 113 ... Object association unit, 114 ... Object identification unit, 115 ... Virtual space definition unit, 116 ... Acquisition unit, 117 ... Object control unit, 118 ... Image generation unit,

Claims (11)

A virtual space providing system that provides virtual space
An object association that associates at least one object with each of multiple users,
An object identification unit that identifies a second object associated with a second user among the plurality of users, which can communicate with the first object associated with the first user among the plurality of users.
A virtual space definition unit that defines a virtual space including the first object and the specified virtual viewpoint of the second object.
An acquisition unit that acquires representation information related to each representation of the first user and the second user, and
It includes an object control unit that controls the first object and the second object according to the expression based on the acquired expression information.
Virtual space providing system. The first object comprises a third object corresponding to at least a part of the body of the first user.
The second object comprises a fourth object corresponding to at least a part of the body of the second user.
The object control unit determines the contact state of the third object and the fourth object according to the orientation or inclination of the third object and the fourth object when the third object and the fourth object come into contact with each other. Control,
The virtual space providing system according to claim 1. The third object is a first hand object corresponding to the first user's hand.
The fourth object is a second hand object corresponding to the second user's hand.
The object control unit is the first when the first hand object and the second hand object are located below a predetermined reference position when the first hand object and the second hand object come into contact with each other. Controlled to maintain the contact state of the hand object and the second hand object for a predetermined period of time.
The virtual space providing system according to claim 2. The third object is a first hand object corresponding to the first user's hand.
The fourth object is a second hand object corresponding to the second user's hand.
The object control unit is the first when the first hand object and the second hand object are located above a predetermined reference position when the first hand object and the second hand object come into contact with each other. Control so that the first hand object and the second hand object are separated immediately after the hand object and the second hand object come into contact with each other.
The virtual space providing system according to claim 2. The acquisition unit further acquires the identification information of the second user,
The object control unit arranges the identification information of the second user in the vicinity of the second object in the virtual space based on the acquired identification information.
The virtual space providing system according to any one of claims 1 to 4. The object identification unit changes the communication period of the first object and the second object in the virtual space based on an instruction from the operator.
The virtual space providing system according to any one of claims 1 to 5. The object identification unit identifies the second object based on the distance between the first object and each of the plurality of objects.
The virtual space providing system according to any one of claims 1 to 5. The object identification unit changes the order in which a plurality of objects communicate with the first object based on an instruction from the operator.
The virtual space providing system according to any one of claims 1 to 7. The plurality of objects include avatars associated with each of the plurality of users.
The virtual space providing system according to any one of claims 1 to 8. It is a virtual space provision executed by the computer that provides the virtual space.
Steps to associate at least one object for each of multiple users,
A step of identifying a second object associated with a second user of the plurality of users that can communicate with the first object associated with the first user of the plurality of users.
A step of defining a virtual space including the first object and the specified virtual viewpoint of the second object.
A step of acquiring representation information regarding each representation of the first user and the second user, and
A step of controlling the first object and the second object according to the representation based on the acquired representation information is included.
How to provide virtual space. A computer that provides virtual space
An object association that associates at least one object with each of multiple users,
An object identification unit that identifies a second object associated with a second user among the plurality of users, which can communicate with the first object associated with the first user among the plurality of users.
A virtual space definition unit that defines a virtual space including the first object and the specified virtual viewpoint of the second object.
An acquisition unit that acquires representation information related to each representation of the first user and the second user, and
An object control unit that controls the first object and the second object according to the expression based on the acquired expression information.
A program that works.

Images