JP6675136B1 - Virtual space providing system, virtual space providing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a virtual space providing technique capable of strengthening a relationship between users who communicate through a virtual space and improving the quality of user's cognitive experience. A virtual space providing system 100 for providing a virtual space, in which an object associating unit 113 associates at least one object with each of a plurality of users and a first user of the plurality of users. An object specifying unit 114 that specifies a second object that is capable of communicating with the first object and that is associated with a second user of a plurality of users; a first object; and a virtual viewpoint of the specified second object, A virtual space defining unit 115 that defines a virtual space that includes an expression, an acquisition unit 116 that acquires expression information about expressions of the first user and the second user, and a first object and a first object based on the acquired expression information. An object control unit 117 that controls two objects according to the expression, Obtain. [Selection diagram] Fig. 1

Description

  The present invention relates to a virtual space providing system for providing a virtual space, a virtual space providing method, and a program.

  Non-Patent Document 1 discloses a technology in which a talent object and a fan object associated with each of an unspecified number of fans who are enthusiasts for the talent can interact in a virtual space. I have.

Internet <URL: https://twitter.com/wahekui_1/status/1119961062613237766>

  In the conventional virtual space providing technology described in Non-Patent Document 1, since a talent object and an unspecified number of fan objects interact in a virtual space, the talent object and a certain virtual fan interact. Attempting to do so may be disturbed by other fan objects. In such a case, sufficient communication cannot be performed between one talent object and a specific one fan object, and there is a possibility that a relationship between the two cannot be sufficiently established. Fans also want their talents to be able to remember through communication in the virtual space. However, if the interaction with the talent object is hindered by another fan object, such a request cannot be satisfied.

  Therefore, the present disclosure provides a virtual space providing technology capable of strengthening the relationship between users who communicate through a virtual space and improving the quality of a user's cognitive experience. With the goal.

  A virtual space providing system according to an aspect of the present invention is a virtual space providing system that provides a virtual space, and includes: an object association unit that associates at least one object with each of a plurality of users; An object specifying unit that specifies a second object associated with a second user of the plurality of users, which is capable of communicating with a first object associated with the first user, a first object, and a specified second object A virtual space definition unit that defines a virtual space including a virtual viewpoint of the object, an acquisition unit that acquires expression information on each expression of the first user and the second user, and a virtual space definition unit based on the acquired expression information. An object control unit that controls the first object and the second object according to the expression. .

  A virtual space providing method according to one aspect of the present invention is a virtual space providing method executed by a computer that provides a virtual space, wherein at least one object is associated with each of a plurality of users; Identifying a second object associated with a second user of the plurality of users capable of communicating with the first object associated with the first user, the first object and the identified second object Defining a virtual space including virtual viewpoints; acquiring expression information on expressions of the first user and the second user; and acquiring the first object and the second object based on the acquired expression information. Controlling the object according to the expression.

  A program according to an embodiment of the present invention relates to a computer that provides a virtual space by associating an object associating unit that associates at least one object with each of a plurality of users and a first user among the plurality of users. An object specifying unit that specifies a second object associated with a second user among a plurality of users, which can communicate with the first object, includes a first object, and a virtual viewpoint of the specified second object. A virtual space definition unit that defines a virtual space, an acquisition unit that acquires expression information about expressions of the first user and the second user, and expresses the first object and the second object based on the acquired expression information. And an object control unit that controls according to.

  In the present invention, “unit” and “apparatus” do not simply mean physical means, but also include a case where functions of the “unit” and “apparatus” are realized by software. Further, even if the function of one “unit” or “device” is realized by two or more physical units or devices, the function of two or more “units” or “devices” is one physical unit or It may be realized by a device.

  According to the above aspect of the present invention, it is possible to strengthen the relationship between users who communicate with each other via the virtual space, and to improve the quality of the user's cognitive experience.

FIG. 1 is a schematic configuration diagram of a virtual space providing system according to an embodiment of the present invention. It is a figure showing an example of an image displayed on an operator terminal concerning an embodiment of the present invention. (A) is an image displayed on the talent terminal according to the embodiment of the present invention, showing an example of an image in a case where a talent object and one fan object communicate in a virtual space. (B) is an image displayed on the talent terminal according to the embodiment of the present invention, showing an example of an image when a talent object and a plurality of fan objects can communicate with each other in a virtual space. It is a schematic diagram of VR (V irtual R eality) computers according to embodiments of the present invention. 1 is a schematic configuration diagram illustrating an example of a functional configuration of a VR computer according to an embodiment of the present invention. FIG. 3 is a diagram conceptually illustrating an example of a virtual space according to the embodiment of the present invention. It is a flow chart which shows an example of virtual space provision processing concerning an embodiment of the present invention. It is a figure for explaining an example of a handshake meeting in a virtual space concerning an embodiment of the present invention. (A) is a figure showing an example of a handshake operation of a talent and a fan in real space. (B) is a diagram showing an example of an image displayed by the HMD according to the embodiment (H ead M ounted D isplay) of the present invention. FIG. 4 is a diagram illustrating an example of a high-touch event in a virtual space according to the embodiment of the present invention. (A) is a figure showing an example of the high touch operation of the talent and the fan in the real space. FIG. 2B is a diagram illustrating an example of an image displayed on the HMD according to the embodiment of the present invention. It is a figure for explaining other examples of a handshake meeting in virtual space concerning an embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude various modifications and application of technology not explicitly described below. That is, the present invention can be implemented with various modifications (eg, by combining the embodiments) without departing from the spirit of the invention. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. The drawings are schematic and do not always match actual dimensions and ratios. The drawings may include portions having different dimensional relationships and ratios between drawings.

FIG. 1 is a block diagram showing an overall configuration of a virtual space providing system 100 according to an embodiment of the present invention. First, an outline of the virtual space providing system 100 according to the present embodiment will be described. HMD (H ead M ounted D isplay ) 5a shown in FIG. 1 is an avid enthusiast for talent TL to stay in the studio, to the fan Fa to stay in the field, the talent object TLO associated with Talent TL And a virtual space image VG based on a virtual space including a virtual viewpoint of the fan object FO1 associated with the fan Fa (for example, corresponding to a camera provided in the HMD 5a). In the example of FIG. 1, the display unit 30 included in the HMD 5 displays a virtual space image VG that shows a situation where the talent object TLO and the fan object FO1 are shaking hands as seen from the viewpoint of the talent object TLO. Virtual space image VG is VR (V irtual R eality) image generated by computer 1.

  In the example of FIG. 1, the number of fans Fa and Fb is two. However, the number of fans is not limited to two, and may be one or three or more. More specifically, for example, the VR computer 1 communicates with the talent object TLO (first object) in the virtual space among the fan objects associated with the fans Fa and Fb by communication or communication. The fan object FO1 (second object) corresponding to the fan Fa is specified as a possible partner. Then, the VR computer 1 defines a virtual space including the talent object TLO and the virtual viewpoint of the specified fan object FO1. "Communication" may include both the fan F and the talent TL interacting with each other using an audio or an action via an object associated with each. “Communication” may also include one of the fan F and the talent T simply observing the operation of the other, or simply listening to the other's remark.

  Here, a specific example of “communication” between a fan and a talent will be described. “Communication” includes, for example, a handshake event where a talent object and a specific fan object can enjoy a conversation while shaking hands in a virtual space, as shown in a virtual space image VG. In addition, the `` communication '' is a high-touch party in which a talent object and a specific fan object perform a high touch in a virtual space, for example, a show-off party in which a fan object sees a talent object after a live in a virtual space, and a talent object in a virtual space. This includes a photo event where a specific fan object takes a picture. In the handshake event, various types of handshake events such as a belt conveyer handshake event where multiple fan objects automatically move as if they are riding on a belt conveyor and shake hands in sequence with the talent object are performed. Including.

  The fan object may hold the item at a live or handshake event with a talent performed in the virtual space. Items include gifts to talent, such as sylliums or bouquets of a particular color. Further, the item may be something worn by the fan object, for example, a T-shirt related to a talent.

  The “object” includes an avatar based on the talent TL or the fan F. For example, the avatar includes an image imitating the talent TL or the fan F, or an image showing a humanoid object corresponding to the talent TL or the fan F. The object may be a simple character icon representing the talent TL or the fan F with two heads, or an elaborate character icon. The object may include an object corresponding to at least a part of the body of the talent TL or the fan F (for example, a whole body, a head, a hand, a foot, and the like). Further, the avatar may include an image imitating an animal or the like associated with the talent TL or the fan F.

  The VR computer 1 acquires expression information of the talent TL and expression information of the fan Fa as a communication partner of the talent TL. The “expression information” includes information indicating the emotions and thoughts of the talent TL and the fan F in an attitude, a language, and the like. The expression information includes, for example, at least one of the operation of the talent TL and the fan F or the sound. The expression information may include a facial expression indicating a smile, a smile, sadness, and the like, and information on the movement of the mouth.

  The VR computer 1 generates a virtual space image in which the expressions of the talent TL and the fan Fa are reflected based on the acquired expression information. That is, the VR computer 1 generates a virtual space image each time the expressions of the talent TL and the fan Fa change.

  Therefore, the fan Fa can communicate with the talent TL about the operation of the object in the virtual space while checking the virtual space image VG displayed on the display unit 30 of the HMD 5a. In addition to the use of the motion of the object in the virtual space, the fan Fa can communicate with the talent TL using voice. For example, the voice of the talent TL acquired by the microphone 3 is transmitted to the operator terminal 4 as voice information. The audio information of the talent TL is transmitted to the HMD 5a via the VR computer 1. A speaker (not shown) provided in the HMD 5a outputs the sound of the talent TL. On the other hand, the sound of the fan Fa acquired by a microphone (not shown) provided in the HMD 5a is transmitted to the VR computer 1 as sound information. The voice information of the fan Fa is transmitted via the operator terminal 4 to the earphone monitor 10 used by the talent TL. The earphone monitor 10 outputs the sound of the fan Fa.

  As described above, the virtual space providing system 100 provides a virtual space image in which the operations of the fan object FO1 and the talent object TLO, which reflect the operations of the fan Fa and the talent TL in the real space, can be grasped. In addition, the virtual space providing system 100 provides both sounds output in synchronization with the operations of the fan object FO1 and the talent object TLO.

As described above, in the virtual space providing system 100, the fan object that can communicate with the talent object is specified, and the virtual space including the talent object and the virtual viewpoint of the specified fan object is defined. Then, in the defined virtual space, the talent object and at least a part of the fan object (such as the whole body object, the hand object, the head object, or the foot object) are controlled based on the expression information. Therefore, it is possible to prevent other fans (fan objects) from being disturbed when communicating with each other. Therefore, it is possible to further strengthen the relationship between users who communicate with each other via the virtual space, and to improve the quality of the user's cognitive experience.
Hereinafter, a specific configuration of the virtual space providing system 100 will be described.

As illustrated in FIG. 1, the virtual space providing system 100 exemplarily includes a camera 2 c that acquires 3D image information (image information) generated by imaging a predetermined range including a talent TL, and audio information of the talent TL. , And an earphone monitor 10 that outputs the sound of the fan F. The virtual space providing system 100 is, for example, an operator terminal 4 that receives the audio information of the talent TL from the microphone 3 and the image information of the talent TL from the camera 2c. And an operator terminal 4 for transferring audio information and image information of the talent TL to the operator. The communication network N, for example, the Internet, a network such as a cellular phone network, LAN (L ocal A rea N etwork), or is realized by a network of a combination of these.

  FIG. 2 is a diagram illustrating an example of an image displayed on the operator terminal 4 according to the embodiment of the present invention. The operator O shown in FIG. 1 is a person who manages and operates the virtual space providing system 100. For example, by operating the operator terminal 4, the operator O can select a fan that can communicate with the talent TL and control the communication content between the two.

  As shown in FIG. 2, on the display unit 400 of the operator terminal 4 operated by the operator O, a plurality of expression selection buttons B1 capable of selecting the contents of the expression of the talent object TLO corresponding to the expression information of the talent TL; A scroll bar SB for controlling at what volume the sound of the fan F is output from the earphone monitor 10 used by the TL is displayed.

  In the display unit 400, a start button B3 for starting communication between a specific fan object and the talent object TLO, and a forced end button B5 for forcibly terminating communication between the specific fan object and the talent object TLO. And a period (eg, remaining time) TM during which communication between the specific fan object and the talent object TLO can be performed is displayed. The instruction of the operator O (for example, pressing of the buttons B1, B3, B5 or operation of the scroll bar SB) input at the operator terminal 4 is transmitted to the VR computer 1 as instruction information via the communication network N shown in FIG. Sent.

  The communication period of the talent object TLO and the fan object in the virtual space can be changed by operating the operator terminal 4 by the operator O. For example, a preset communication period may be changed. The time remaining after the communication is started may be increased or decreased. In this case, the instruction information based on the operation of the operator O is transmitted from the operator terminal 4 to the VR computer 1, and the VR computer 1 executes the communication period changing process.

  When there are a plurality of fan objects that may communicate with the talent object TLO, the operator O can operate the operator terminal 4 to change the communication order. In this case, the instruction information based on the operation of the operator O is transmitted from the operator terminal 4 to the VR computer 1, and the VR computer 1 executes a process of changing the order of communication.

  Returning to FIG. 1, the virtual space providing system 100 is, for example, a VR computer 1 that provides a virtual space including a talent object TLO associated with the talent TL and a virtual viewpoint of the fan object associated with the fan F; And an HMD 5 that displays a virtual space image based on the virtual space provided by the VR computer 1. In the case where the fans Fa and Fb are described without distinction, some of the reference numerals are omitted, and the fans are simply referred to as “fans F”. In the case where the HMDs 5a and 5b are described without distinction, some of the reference numerals are omitted, and they are simply referred to as “HMD5”. In the example illustrated in FIG. 1, the VR computer 1 is disposed outside the HMD 5. However, the configuration is not limited to this, and the VR computer 1 may be built in the HMD 5. That is, the HMD 5 may include the functions of the VR computer 1 described below.

The HMD 5 is mounted on the head of the fan F, for example, and displays a virtual space image (for example, a three-dimensional image) based on a virtual space corresponding to the movement of the head of the fan F. The display unit 30 of the HMD 5 can display virtual space image information or text information transmitted from the VR computer 1. The display unit 30 can be arranged so as to be located in front of the eyes of the fan F. Display unit 30 includes, for example, a liquid crystal display or an organic EL (E lectro L uminescence) display. The display unit 30 may include a transmissive or transflective display device. The display unit 30 may simultaneously display at least a part of the virtual space image and a predetermined range of the real space captured by a camera (not shown) provided in the HMD 5. Although the function of the display unit 30 of the HMD 5 is not shown, a display unit included in a mobile terminal such as a smartphone or a laptop computer separate from the HMD 5 may be substituted. Although the function of the display unit 30 of the HMD 5 is not shown, a stationary display device such as a display monitor separate from the HMD 5 may be used instead.

  The virtual space providing system 100 exemplifies, via the communication network N, the monitoring terminal 6 used by the staff Sa who stays at the site and the local image information obtained by imaging the state of the site using the camera 2b. A local video distribution terminal 7 for distributing to the video display terminal 8, a local video display terminal 8 for receiving and displaying local video information distributed from the local video distribution terminal 7, and a talent object associated with the talent TL in a virtual space. There is further provided a talent terminal 9 for displaying a virtual space image for the talent TL itself to confirm how to communicate with the fan object.

  For example, when a plurality of fans F to which the HMD 5 is attached are present at the monitoring terminal 6, a virtual space image based on the virtual viewpoint of the fan object of the fan Fa and a virtual space image based on the virtual viewpoint of the fan object of the fan Fb are used. And the virtual space image set as above. Since the staff Sa can check each of the virtual space images visually recognized by each fan, it is possible to check whether the mounting status of the HMD 5 of each fan F, for example, the orientation or the inclination of the HMD 5 is not defective. .

  FIG. 3A is an image displayed on the display unit 900 of the talent terminal 9 shown in FIG. 1, and is a virtual space image VG1 when the talent object TLO and one fan object FO1 communicate in the virtual space. It is a figure showing an example of. FIG. 3B is an image displayed on the display unit 900 of the talent terminal 9, and is a virtual space image VG2 in which the talent object TLO and the plurality of fan objects FO1 and FO2 can communicate in order in the virtual space. It is a figure showing an example of. As shown in FIGS. 3A and 3B, virtual space images displayed on the display unit 900 of the talent terminal 9 are obtained from various positions (virtual viewpoints) in the virtual space corresponding to a certain handshake meeting place. May include a plurality of virtual space images corresponding to the field of view. The virtual space image displayed on the display unit 900 of the talent terminal 9 may be an image generated by the VR computer 1 and transmitted from the VR computer 1. Further, the virtual space image displayed on the display unit 900 of the talent terminal 9 may be an image generated by the talent terminal 9.

  The display unit 900 of the talent terminal 9 displays fan information related to the fan F currently communicating with the talent TL via the virtual space, such as a fan ID, a fan name, a fan face icon, and a fan emotion. , The color of the psyllium held by the fan, the number of times the fan participates in the live, the presence or absence of a gift sent by the fan to the talent TL, the degree of the fan's billing, and the like. Note that the fan information may be displayed together with the virtual space image, or may be displayed instead of the virtual space image.

FIG. 4 is a block diagram illustrating an example of a hardware configuration of the VR computer according to the embodiment of the present invention. As shown in FIG. 4, VR computer 1 includes, for example, a CPU (C entral P rocessing U nit ) 11, composed of ROM (R ead O nly M emory ) and RAM (R andom A ccess M emory ) , etc. It includes a memory 12, an input unit 13, an output unit 14, a recording unit 15, a communication unit 16, and a bus 17.

  The CPU 11 executes various processes according to a program recorded in the memory 12 or a program loaded into the memory 12 from the recording unit 15.

  The memory 12 also stores data and the like necessary for the CPU 11 to execute various processes. The CPU 11 and the memory 12 are connected to each other via a bus 17. Further, the input unit 13, the output unit 14, the recording unit 15, and the communication unit 16 are connected to the bus 17.

  The input unit 13 inputs various information in accordance with an instruction operation of a person who operates the VR computer 1 (for example, a staff Sa or a staff at the site). The input unit 13 includes, for example, various buttons, a touch panel, a microphone, and the like. The input unit 13 may be realized by an input device such as a keyboard and a mouse, which is independent of a main body that accommodates other units of the computer 1.

  The output unit 14 outputs image data and audio data. The output unit 14 includes, for example, a display, a speaker, and the like. The image data and audio data may be output from a display, a speaker, or the like so that the user can recognize the image or music as an image or music.

The recording unit 15 stores various data. Recording unit 15, for example, a semiconductor memory such as a DRAM (D ynamic R andom A ccess M emory).

  The communication unit 16 realizes communication performed with another device. For example, the communication unit 16 performs mutual communication with another terminal via the network N.

  Although not shown, the VR computer 1 is provided with a drive as needed. For example, a removable medium including a magnetic disk, an optical disk, a magneto-optical disk, and the like may be appropriately mounted on the drive. The programs read from the removable media by the drive and various data such as image data are installed in the recording unit 15 as needed.

  The operator terminal 4, the monitoring terminal 6, the local video distribution terminal 7, the local video display terminal 8, or the talent terminal 9 may include at least a part of each of the above components included in the VR computer 1.

  FIG. 5 is a block diagram illustrating an example of a functional configuration of the VR computer according to the embodiment of the present invention. The computer 1 exemplarily includes an information processing unit 111 that executes a process for providing a virtual space, and a database (DB) that records various types of information necessary to execute the process and information regarding a result of the process. 112. Note that the information processing section 111 can be realized, for example, by the CPU 11 executing a program stored in the memory 12 or the recording section 15 shown in FIG. The DB 112 corresponds to, for example, the recording unit 15 illustrated in FIG.

  The information processing unit 111 functionally includes an object associating unit 113 that associates at least one object with each of a plurality of users including a talent and one or a plurality of fans, and a talent (first An object specifying unit 114 that specifies a fan object (second object) associated with a fan (second user) among a plurality of users, which can communicate with a talent object (first object) associated with the user); Is provided. The information processing unit 111 functionally defines a virtual space definition unit 115 that defines a virtual space including a talent object and a virtual viewpoint of the specified fan object, and at least expression information on expressions of the talent and the fan. An acquisition unit 116 is provided for acquiring, and an object control unit 117 for controlling a talent object and a fan object in accordance with the expression based on the acquired expression information. Details of the above functions will be described later.

  The information processing unit 111 further includes an image generation unit 118. The image generation unit 118 generates a virtual space image based on the virtual space based on the virtual space data.

  FIG. 6 is a diagram conceptually illustrating an example of the virtual space according to the embodiment of the present invention. As shown in FIG. 6, the virtual space 50 defined by the virtual space definition unit 115 includes, for example, a hemisphere based on the center point CP. In the virtual space 50, an XYZ coordinate system having the center point CP as the origin is defined. For example, when the HMD 5 shown in FIG. 1 is activated, a virtual camera 60 (virtual viewpoint) (e.g., equivalent to a camera included in the HMD 5) is arranged at the center point CP of the virtual space 50. An image is displayed on the display unit 30 of the HMD 5 based on image information obtained by the virtual camera 60 taking an image. As shown in FIG. 1, the virtual camera 60 can move according to the movement (change in position and inclination) of the HMD 5 mounted on the head of the fan F in the real space. That is, the virtual camera 60 in the virtual space 50 moves so as to correspond to the movement of the HMD 5 in the real space.

  Here, the position of the virtual camera 60 corresponds to the viewpoint of the fan F (user) in the virtual space 50. The image generation unit 16 defines a view area 80 in the virtual space 50 based on the reference line of sight SG based on the virtual camera 60. For example, the view area 80 corresponds to an area in the virtual space 50 where the fan F wearing the HMD 5 visually recognizes.

  Thus, the image generation unit 16 arranges the virtual camera 60 at the center point CP in the virtual space 50, for example, and sets the line of sight (reference line of sight SG) of the virtual camera 60 to the direction in which the fan F is facing. Therefore, the image generation unit 16 generates view image data for generating a view image (virtual space image VG3). Thereafter, when the movement of the HMD 5 changes, the image generation unit 16 generates a new virtual space image according to the change.

  The virtual camera 60 is not limited to the HMD 5 camera mounted on the head of the fan F in FIG. For example, when the HMD 5 is mounted on the head of a talent, the virtual camera 60 may correspond to the camera of the HMD 5. As shown in FIGS. 3A and 3B, a virtual space image corresponding to a field of view from various viewpoints (virtual cameras) in a specific virtual space may be generated.

  Returning to FIG. 5, the DB 112 illustratively includes space information SI including one or more virtual space data templates for defining the virtual space 50 illustrated in FIG. 6, object information OI regarding an object, and expression information EI. , Record. In the DB 112, the above various information may be recorded in association with each user on the basis of information on the user.

<Virtual space provision processing>
An example of the virtual space providing process according to the embodiment of the present invention will be described with reference to FIGS. FIG. 7 is a flowchart illustrating an example of the virtual space providing process according to the embodiment of the present invention.

(Step S1)
The object association unit 113 of the VR computer 1 illustrated in FIG. 5 associates an object with each of a plurality of users. Referring to the examples of FIGS. 1 and 3, the object associating unit 113, based on the object information OI shown in FIG. To associate the fan object FO1 with the fan object FO2.

(Step S3)
The object specifying unit 114 of the VR computer 1 illustrated in FIG. 5 generates the fan object FO associated with the fan F of the plurality of users, which can communicate with the talent object TLO associated with the talent TL of the plurality of users. Identify. Referring to the examples of FIGS. 1 and 3, the object specifying unit 114 specifies the fan object FO1 associated with the fan Fa as the communication partner of the talent object TLO.

(Step S5)
The virtual space definition unit 115 of the VR computer 1 illustrated in FIG. 5 includes, as in the example of the virtual space illustrated in FIG. 6, a talent object TLO and a virtual viewpoint (for example, the HMD 5a used by the fan Fa Virtual space).

(Step S7)
The acquisition unit 116 of the VR computer 1 illustrated in FIG. 5 acquires at least expression information on the expressions of the talent TL and the fan Fa. First, the camera 2c illustrated in FIG. 1 captures a predetermined range including the talent TL, generates 3D image information including the operation of the talent TL, and outputs the 3D image information to the operator terminal 4 as expression information. The operator terminal 4 transmits the expression information to the VR computer 1 via the communication network N. The camera 2a captures a predetermined range including the fan Fa, generates 3D image information including an operation of the fan Fa, and outputs the 3D image information to the VR computer 1 as expression information. The microphone 3 acquires the sound of the talent TL and outputs the sound information to the operator terminal 4 as expression information. The operator terminal 4 transmits the expression information to the VR computer 1 via the communication network N. Further, for example, a microphone (not shown) provided in the camera 2a acquires the sound of the fan Fa and outputs the sound information to the VR computer 1 as expression information. The VR computer 1 may include a microphone that acquires the sound of the fan as a configuration separate from the camera 2a.

  As described above, the acquisition unit 116 acquires the 3D image information and the audio information of the talent TL as the expression information of the talent TL, and acquires the 3D image information and the audio information of the fan Fa as the expression information of the fan Fa. . The acquisition unit 116 may acquire the expression information of the talent TL and the fan Fa periodically or at an arbitrary timing.

(Step S9)
The object control unit 117 of the VR computer 1 shown in FIG. 5 controls the talent object and the fan object according to the expression based on the acquired expression information.

  FIG. 8 is a diagram illustrating an example of a handshake event in the virtual space according to the embodiment of the present invention. FIG. 8A is a diagram illustrating an example of a handshake operation of a talent and a fan in the real space. FIG. 8B is a diagram illustrating an example of an image displayed on the HMD according to the embodiment of the present invention. As shown in FIG. 8 (a), when the talent TL turns his or her hand H1 downward in the real space and the fan Fa makes his or her hand H2 downward and performs an operation of shaking hands with both. As shown in FIG. 8B, the handshake operation of the talent TL and the fan Fa is reflected on the talent object TLO and the fan object FO1 in the virtual space.

  Specifically, the talent object TLO includes an object (third object) corresponding to at least a part of the body of the talent TL. The object corresponding to at least a part of the body of the talent TL includes a hand object HO1 (first hand object) corresponding to the hand of the talent TL. The fan object FO1 includes an object (fourth object) corresponding to at least a part of the body of the fan Fa. The object corresponding to at least a part of the body of the fan Fa includes a hand object HO2 (second hand object) corresponding to the hand of the fan Fa.

  The object control unit 117 controls the contact state between the hand object HO1 and the hand object HO2 in accordance with the respective directions or inclinations of the hand object HO1 and the hand object HO2 when the hand object HO1 and the hand object HO2 contact each other in the virtual space. I do. More specifically, if the hand object HO1 and the hand object HO2 satisfy predetermined conditions when the hand object HO1 and the hand object HO2 make contact, the object control unit 117 determines the contact state of the hand object HO1 and the hand object HO2. Control is performed so as to be maintained for a predetermined period.

  For example, when the hand object HO1 and the hand object HO2 when the hand object HO1 and the hand object HO2 come into contact with each other are located below a predetermined reference position, the object control unit 117 determines that the talent object and the specific fan object Control to shake hands. The predetermined reference position for the talent object TLO and the fan object FO to shake hands may be any position at which the handshake operation performed by both objects looks natural. The predetermined reference position may be, for example, a position corresponding to the shoulder (shoulder object) of the talent object TLO for the hand object HO1, or a position corresponding to the shoulder of the fan object FO for the hand object HO2. (Shoulder object). The predetermined reference position is not limited to the above, and may be a position corresponding to the chest of the talent object TLO and the fan object FO.

  For example, the predetermined reference position may be set differently between the talent object TLO and the fan object FO. For example, the reference position of the talent object TLO may be a position corresponding to the shoulder, and the reference position of the fan object FO may be a position corresponding to the chest.

  The object control unit 117 may consider other conditions as conditions for executing the handshake operation. For example, among the plurality of finger objects included in each of the hand objects HO1 and HO2, the object control unit 117 determines that the height of the first joint of the middle finger object (the first joint counted from the fingertip) is the second joint of the middle finger ( If the height is lower than the height of the second joint (counted from the fingertip), control may be performed so that the talent object and the specific fan object shake hands. Note that a finger other than the middle finger may be employed among the plurality of finger objects to determine whether or not the condition for performing the handshake operation is satisfied.

  According to this configuration, it is possible to realize a handshake event in which the talent object and the specific fan object can enjoy conversation and the like while shaking hands in the virtual space. Therefore, the relationship between users who communicate with each other through the virtual space can be strengthened, and the quality of the user's cognitive experience can be further improved. Further, by further considering the position of the finger object in addition to the position of the hand object, it is possible to more accurately determine whether or not to perform a handshake operation.

  "When the hand object HO1 and the hand object HO2 make contact" includes before (for example, immediately before) the two objects make contact, when the two objects make contact, or after (for example, immediately after) the two objects make contact. The “predetermined period” is not particularly limited, and may be, for example, several seconds or tens of seconds. The predetermined period can be changed. For example, the predetermined period may be changed by operating the operator terminal 4 by the operator O shown in FIG.

  FIG. 9 is a diagram illustrating an example of a high-touch event in a virtual space according to the embodiment of the present invention. FIG. 9A is a diagram illustrating an example of a high touch operation of a talent and a fan in the real space. FIG. 9B is a diagram illustrating an example of an image displayed on the HMD according to the embodiment of the present invention. As shown in FIG. 9A, when the talent TL turns his or her hand H1 upward in the real space and the fan Fa makes his or her hand H2 upward to perform an operation of making a high touch with both of them. As shown in FIG. 9B, the high touch operation of the talent TL and the fan Fa is reflected on the talent object TLO and the fan object FO1 in the virtual space.

  When the hand object HO1 and the hand object HO2 satisfy a predetermined condition when the hand object HO1 and the hand object HO2 come into contact with each other, the object control unit 117 performs the hand object HO1 and the hand object HO2 immediately after the hand object HO1 and the hand object HO2 come into contact with each other. Control is performed so that the hand object HO2 is separated.

  For example, if the hand object HO1 and the hand object HO2 are located above a predetermined reference position when the hand object HO1 and the hand object HO2 come into contact with each other, the object control unit 117 outputs a talent object, a specific fan object, Is controlled to make a high touch. The predetermined reference position for the high touch of the talent object TLO and the fan object FO may be any position at which the high touch operation performed by both objects looks natural. The predetermined reference position may be, for example, a position corresponding to the shoulder (shoulder object) of the talent object TLO for the hand object HO1, or a position corresponding to the shoulder of the fan object FO for the hand object HO2. (Shoulder object). The predetermined reference position is not limited to the above, and may be a position corresponding to the chest of the talent object TLO and the fan object FO. For example, the predetermined reference position may be set differently between the talent object TLO and the fan object FO. For example, the reference position of the talent object TLO may be a position corresponding to the shoulder, and the reference position of the fan object FO may be a position corresponding to the chest.

  The object control unit 117 may consider other conditions as conditions for executing the handshake operation. For example, if the height of the first joint of the middle finger object is higher than the height of the second joint of the middle finger object, among the plurality of finger objects included in each of the hand objects HO1 and HO2, the object control unit 117 determines that Control may be performed so that a specific fan object makes a high touch. Note that a finger other than the middle finger may be employed among the plurality of finger objects to determine whether or not the condition for performing the handshake operation is satisfied.

  According to this configuration, it is possible to realize a high touch event in which a talent object and a specific fan object can enjoy a conversation while performing a high touch in a virtual space. Therefore, the relationship between users who communicate with each other through the virtual space can be strengthened, and the quality of the user's cognitive experience can be further improved. Further, by further taking into account the position of the finger object in addition to the position of the hand object, it is possible to more accurately determine whether or not to execute the high touch operation.

  FIG. 10 is a diagram illustrating an example of a belt-conveyor-type handshake event in a virtual space according to the embodiment of the present invention. As shown in FIG. 10, in the belt conveyor type handshake event, as if a plurality of fan objects FO0 to FO7 are riding on the belt conveyor, for example, the talent objects FO0 to FO7 are automatically moved in the direction indicated by the dashed arrow A in order. You can shake hands and talk with TLO. Each of the fan objects FO0 to FO7 can communicate when the distance from the talent object TLO becomes a fixed distance L. For example, in the example of FIG. 10, the object specifying unit 114 illustrated in FIG. 5 calculates the distance between the talent object TLO and each of the plurality of fan objects FO0 to FO7, and is located at a fixed distance L from the talent object TLO. The fan object FO1 is specified. The virtual space definition unit 115 defines a virtual space in which the talent object TLO and the fan object FO1 can communicate. Note that the distance L can be changed as appropriate.

  According to this configuration, the communication between the talent object and the fan object can be automated, the load on the operator can be reduced, and the number of fans that can communicate with the talent can be increased.

  According to the embodiment described above, a fan object that can communicate with a talent object is specified, and a virtual space including the talent object and a virtual viewpoint of the specified fan object is defined. Then, in the defined virtual space, the talent object and at least a part of the fan objects are controlled based on the expression information. Therefore, it is possible to prevent other fans (fan objects) from being disturbed when communicating with each other. Therefore, it is possible to further strengthen the relationship between users who communicate with each other via the virtual space, and to improve the quality of the user's cognitive experience.

(Other embodiments)
Each of the above embodiments is intended to facilitate understanding of the present invention, and is not to be construed as limiting the present invention. The present invention can be changed / improved (for example, combining the embodiments, omitting some of the configurations of the embodiments) without departing from the spirit thereof, and the present invention also includes equivalents thereof. It is.

  As shown in FIGS. 3B and 8B, fan identification information (for example, text information T1, T2, T3 of the fan name) is displayed near the fan object associated with the fan in the virtual space. You may. Specifically, the obtaining unit 116 illustrated in FIG. 5 obtains the identification information of the fan, and the object control unit 117 determines the fan corresponding to the fan object in the vicinity of the fan object in the virtual space based on the obtained identification information. Place identification information. The fan identification information may be stored in the database 112 in advance, or may be separately input by the operator O or the like.

  In the example of FIG. 1, in the virtual space providing system 100, the VR computer 1 is commonly used by a plurality of HMDs 5, and usually, a separate virtual space can be defined for each different fan. Note that a different VR computer may be used for each of the plurality of HMDs 5. The virtual space providing system 100 also provides a virtual space image based on a virtual space including a talent object TLO associated with the talent TL and a specified virtual viewpoint of the fan object FO1 associated with the fan Fa. However, the virtual space providing system 100 provides a virtual space image further including a fan object of a fan different from the fan Fa in response to a request from the operator O, the staff Sa, the talent TL, or the fan F (for example, It is possible to switch the displayed virtual space image).

  The virtual space providing system 100 includes a VR computer 1, an operator terminal 4, a monitoring terminal 6, a local video distribution terminal 7, a local video display terminal 8, and a talent terminal 9. However, the virtual space providing system 100 may include a device that integrates two or more of these devices. For example, the local image display terminal 8 and the talent terminal 9 used by the talent TL may be a single terminal device.

  In the above embodiments, communication between one talent TL and one or more fans F has been described, but the present invention is not limited to these forms. For example, the present invention is also applicable to communication between a plurality of talents TL and a plurality of fans F. That is, when a plurality of talent objects TLO and a plurality of fan objects FO are arranged in a certain virtual venue, for example, in the virtual venue, a virtual space X for communication between the talent X object and the fan X object is provided. In addition, it is possible to provide a virtual space Y different from the virtual space X for a talent Y object different from the talent X object and a fan Y object different from the fan X object to communicate with each other.

  As described above, the operations (expression information) of the talent TL and the fan F are acquired using the cameras provided around each of the talent TL and the fan F, but are not limited to this mode. The operations of the talent TL and the fan F may be acquired using a controller (not shown) that can be held by each of the talent TL and the fan F. The controller may be configured to be attachable to a part of the body or clothing of the talent TL or the fan F. The controller may be a glove-type controller corresponding to the hands of the talent TL or the fan F, or a stick-type controller. For example, a controller connected to the VR computer 1 shown in FIG. 1 by wire or wirelessly includes a motion sensor. For example, the controller may receive, from the talent TL or the fan F, an operation for controlling the position or movement of at least a part of the object placed in the virtual space. More specifically, the controller is attached to, for example, the hand of the talent TL or the fan F, and detects the movement of the hand of the talent TL or the fan F. For example, a motion sensor provided in the controller detects the number of rotations of the hand or the rotation speed. The detection result is transmitted to the VR computer 1 as a detection signal.

DESCRIPTION OF SYMBOLS 1 ... VR computer, 2a, 2b, 2c ... Camera, 3 ... Microphone, 4 ... Operator terminal, 5a, 5b ... HMD, 6 ... Surveillance terminal, 7 ... Local video distribution terminal, 8 ... Local video display terminal, 9 ... Talent Terminal, 10: Earphone monitor, 11: CPU, 12: Memory, 13: Input unit, 14: Output unit, 15: Recording unit, 16: Communication unit, 17: Bus, 30, 400, 900 ... Display unit, 100 ... Virtual space providing system, 111 Information processing unit, 112 Database (DB), 113 Object association unit, 114 Object identification unit, 115 Virtual space definition unit, 116 Acquisition unit, 117 Object control unit, 118 Image generator,

Claims (13)

A virtual space providing system for providing a virtual space,
An object association unit that associates at least one object with each of the plurality of users;
An object identification unit that can communicate with a first object associated with a first user of the plurality of users and identifies a second object associated with a second user of the plurality of users;
A virtual space definition unit that defines a virtual space including the first object and a virtual viewpoint of the specified second object;
An acquisition unit configured to acquire expression information on expressions of the first user and the second user;
Based on the acquired expression information, Bei give a, and an object control unit for controlling in response to said first object and said second object in said representation,
The first object includes a first hand object corresponding to a hand of the first user,
The second object includes a second hand object corresponding to the hand of the second user,
When the first hand object and the second hand object are located below a predetermined reference position when the first hand object and the second hand object come into contact with each other, the object control unit may perform the first hand object. Controlling the contact state between the hand object and the second hand object to be maintained for a predetermined period;
Virtual space providing system. The first object includes a third object corresponding to at least a part of a body of the first user;
The second object includes a fourth object corresponding to at least a part of a body of the second user,
The object control unit may change a contact state between the third object and the fourth object according to an orientation or an inclination of the third object and the fourth object when the third object and the fourth object contact each other. Control,
The virtual space providing system according to claim 1. Prior Symbol object control unit, when the first hand object and the second hand object when the first hand object and the second hand object is in contact is located above a predetermined reference position, the first Controlling the first hand object and the second hand object to be separated immediately after the one hand object and the second hand object are in contact with each other;
Virtual space-providing system according to claim 1 or claim 2. The obtaining unit further obtains identification information of the second user,
The object control unit arranges the identification information of the second user near the second object in the virtual space based on the acquired identification information,
Virtual space-providing system according to any one of claims 1 to 3. The object specifying unit changes a communication period of the first object and the second object in the virtual space based on an instruction from an operator,
Virtual space-providing system according to any one of claims 1 to 4. The object specifying unit specifies the second object based on a distance between the first object and each of the plurality of objects.
Virtual space-providing system according to any one of claims 1 to 5. The object specifying unit changes an order in which a plurality of objects communicate with the first object based on an instruction from an operator.
Virtual space-providing system according to any one of claims 1 to 6. The plurality of objects include an avatar associated with each of the plurality of users,
Virtual space-providing system according to any one of claims 1 to 7. A virtual space provision executed by a computer that provides a virtual space,
Associating at least one object with each of the plurality of users;
Identifying a second object associated with a second user of the plurality of users that is capable of communicating with a first object associated with a first user of the plurality of users;
Defining a virtual space including the first object and a virtual viewpoint of the specified second object;
Acquiring expression information on expressions of the first user and the second user;
Based on the acquired expression information, see contains the steps of: controlling in response to said first object and said second object in said representation,
The first object includes a first hand object corresponding to a hand of the first user,
The second object includes a second hand object corresponding to the hand of the second user,
The controlling may include: if the first hand object and the second hand object are located below a predetermined reference position when the first hand object and the second hand object are in contact with each other, Controlling the contact state between the hand object and the second hand object to be maintained for a predetermined period;
How to provide virtual space. A computer that provides a virtual space
An object association unit that associates at least one object with each of the plurality of users;
An object identification unit that can communicate with a first object associated with a first user of the plurality of users and identifies a second object associated with a second user of the plurality of users;
A virtual space definition unit that defines a virtual space including the first object and a virtual viewpoint of the specified second object;
An acquisition unit configured to acquire expression information on expressions of the first user and the second user;
An object control unit that controls the first object and the second object according to the expression based on the obtained expression information;
And it is made to function,
The first object includes a first hand object corresponding to a hand of the first user,
The second object includes a second hand object corresponding to the hand of the second user,
When the first hand object and the second hand object are located below a predetermined reference position when the first hand object and the second hand object come into contact with each other, the object control unit may perform the first hand object. A program for controlling a contact state between a hand object and the second hand object to be maintained for a predetermined period . A virtual space providing system for providing a virtual space,
An object association unit that associates at least one object with each of the plurality of users;
An object identification unit that can communicate with a first object associated with a first user of the plurality of users and identifies a second object associated with a second user of the plurality of users;
A virtual space definition unit that defines a virtual space including the first object and a virtual viewpoint of the specified second object;
An acquisition unit configured to acquire expression information on expressions of the first user and the second user;
An object control unit that controls the first object and the second object according to the expression based on the acquired expression information,
The first object includes a first hand object corresponding to a hand of the first user,
The second object includes a second hand object corresponding to the hand of the second user,
When the first hand object and the second hand object are located above a predetermined reference position when the first hand object and the second hand object come into contact with each other, the object control unit may perform the first hand object. Immediately after the hand object and the second hand object come into contact with each other, control is performed so that the first hand object and the second hand object are separated from each other.
Virtual space providing system. A virtual space provision executed by a computer that provides a virtual space,
Associating at least one object with each of the plurality of users;
Identifying a second object associated with a second user of the plurality of users that is capable of communicating with a first object associated with a first user of the plurality of users;
Defining a virtual space including the first object and a virtual viewpoint of the specified second object;
Acquiring expression information on expressions of the first user and the second user;
Controlling the first object and the second object according to the expression based on the obtained expression information,
The first object includes a first hand object corresponding to a hand of the first user,
The second object includes a second hand object corresponding to the hand of the second user,
The step of controlling may include, when the first hand object and the second hand object are in contact with each other, the first hand object and the second hand object being located above a predetermined reference position. Immediately after the hand object and the second hand object come into contact with each other, control is performed so that the first hand object and the second hand object are separated from each other.
How to provide virtual space. A computer that provides a virtual space
An object association unit that associates at least one object with each of the plurality of users;
An object identification unit that can communicate with a first object associated with a first user of the plurality of users and identifies a second object associated with a second user of the plurality of users;
A virtual space definition unit that defines a virtual space including the first object and a virtual viewpoint of the specified second object;
An acquisition unit configured to acquire expression information on expressions of the first user and the second user;
An object control unit that controls the first object and the second object according to the expression based on the obtained expression information;
Function
The first object includes a first hand object corresponding to a hand of the first user,
The second object includes a second hand object corresponding to the hand of the second user,
When the first hand object and the second hand object are located above a predetermined reference position when the first hand object and the second hand object are in contact with each other, the object control unit may control the first hand object and the second hand object. Immediately after the hand object and the second hand object come into contact with each other, control is performed so that the first hand object and the second hand object are separated from each other.
program.

Images