WO2019216249A1

WO2019216249A1 - Method for providing virtual space having prescribed content

Info

Publication number: WO2019216249A1
Application number: PCT/JP2019/017727
Authority: WO
Inventors: 義仁近藤; 雅人室橋
Original assignee: 株式会社エクシヴィ
Priority date: 2018-05-07
Filing date: 2019-04-25
Publication date: 2019-11-14
Also published as: JPWO2019216249A1

Abstract

[Problem] To provide a method in which an interaction between a performer and a user can be more effectively promoted in a virtual space. [Solution] In this method for providing, to a plurality of users including a performing user and a viewing user, a virtual space which has prescribed content including a character performed by the performing user, the virtual space is provided and is characterized in that a prescribed action is received from the user, the received prescribed action is stored as history information for each user, the virtual space including prescribed content and user-identifying information is generated, and information generated on the basis of the history information is displayed.

Description

Method for providing a virtual space having predetermined content

The present invention relates to a method for providing a virtual space for a plurality of users. More specifically, the present invention relates to a method for providing a virtual space having predetermined content including a character played by a performer.

Motion capture is a technology that digitally captures the performance of a performer user (hereinafter referred to as “performer”) in real space, and the captured motion is used to represent character motion in computer animation such as video and games. The

In recent years, a method for controlling a character in a virtual space has been devised through movement of a head mounted display (hereinafter referred to as “HMD”) worn by a performer or input by a controller held by the performer.

In addition, content sharing services that deliver content such as live video to users are provided. In such content sharing services, items can be acquired to express positive evaluations of content contributors and performers, A mechanism for encouraging purchases is disclosed. (For example, Patent Document 1)

Japanese Patent Laying-Open No. 2015-090526

The technology disclosed in the above document provides a mechanism for determining the display position in the virtual space of the user avatar according to the amount of items purchased by the user in order to spawn a desire for approval of the user or a sense of competition between users. ing. However, although the display position of the user avatar is displayed at a position closer to the performer according to the present technology, the user's approval desire can be satisfied, but there is no interaction or communication with the performer.

Therefore, an object of the present invention is to provide a method capable of more effectively promoting the interaction between a performer and a user in a virtual space.

According to an embodiment of the present invention, there is provided a method for providing a virtual space having a predetermined content including a character played by a performer user to a plurality of users including a performer user and a viewing user. An action is received, the received predetermined action is stored as history information for each user, a virtual space including predetermined content and information for identifying the user is generated, and information generated based on the history information is displayed. A virtual space characterized by this is provided.

According to the present invention, it is possible to more effectively promote interaction and communication between a performer and a user in a virtual space. This further satisfies the user's desire for approval.

The schematic diagram of the external appearance of the head mounted display 110 concerning 1st Embodiment is shown. The schematic diagram of the external appearance of the controller 210 concerning 1st Embodiment is shown. 1 is a configuration diagram of an HMD system 300 according to a first embodiment. The functional block diagram of HMD110 concerning 1st Embodiment is shown. The functional block diagram of the controller 210 concerning 1st Embodiment is shown. 1 is a functional configuration diagram of an image generation apparatus 310 according to a first embodiment. An example of the virtual space displayed on a user apparatus concerning 1st Embodiment is shown. The other example of the virtual space displayed on a user apparatus concerning 1st Embodiment is shown. The further another example of the virtual space displayed on a user apparatus concerning 1st Embodiment is shown. An example of the user management table concerning 1st Embodiment is shown. An example of the user gift history management table concerning a 1st embodiment is shown. An example of the user comment history management table concerning a 1st embodiment is shown. In 1st Embodiment, the flowchart as an example of the process which provides a user with virtual space is shown. In 1st Embodiment, the flowchart as another example of the process which provides a user with virtual space is shown. In 2nd Embodiment, the flowchart as an example of the process which provides a user with virtual space is shown. In 2nd Embodiment, an example of the virtual space displayed to a user is shown. In 3rd Embodiment, the flowchart as an example of the process which provides a user with virtual space is shown. In 3rd Embodiment, an example of the virtual space displayed to a user is shown. In 4th Embodiment, an example of the virtual space displayed to a user is shown. In 5th Embodiment, an example of the virtual space displayed on a performer is shown. In 6th Embodiment, an example of the virtual space displayed to a user is shown.

<Embodiment 1>
A specific example of a program for controlling a head mounted display system according to an embodiment of the present invention will be described below with reference to the drawings. In addition, this invention is not limited to these illustrations, is shown by the claim, and intends that all the changes within the meaning and range equivalent to the claim are included. In the following description, the same reference numerals are given to the same elements in the description of the drawings, and redundant descriptions are omitted.

FIG. 1 is a schematic view of the appearance of a head mounted display (hereinafter referred to as HMD) 110 according to the present embodiment. The HMD 110 is mounted on the performer's head and includes a display panel 120 so as to be placed in front of the left and right eyes of the performer. As the display panel, an optical transmission type and a non-transmission type display are conceivable. In this embodiment, a non-transmission type display panel capable of providing a more immersive feeling is exemplified. The display panel 120 displays an image for the left eye and an image for the right eye. By using the parallax between both eyes, an image with a stereoscopic effect can be provided to the performer. If the image for the left eye and the image for the right eye can be displayed, the display for the left eye and the display for the right eye can be provided separately, or an integrated display for the left eye and the right eye can be provided. is there.

Furthermore, the housing part 130 of the HMD 110 includes a sensor 140. Although not shown in the figure, the sensor can include, for example, a magnetic sensor, an acceleration sensor, a gyro sensor, or a combination thereof, in order to detect movements such as the head direction and inclination of the performer. The vertical direction of the head of the performer is defined as the Y axis, and among the axes orthogonal to the Y axis, the axis corresponding to the front and rear direction of the performer that connects the center of the display panel 120 and the performer is defined as the Z axis. And the axis corresponding to the left and right direction of the performer is the X axis, the sensor 140 has a rotation angle around the X axis (so-called pitch angle), a rotation angle around the Y axis (so-called yaw angle), Z A rotation angle around the axis (so-called roll angle) can be detected.

In addition, or instead of the sensor 140, the housing unit 130 of the HMD 110 can include a plurality of light sources 150 (for example, infrared light LEDs and visible light LEDs) and is installed outside the HMD 110 (for example, indoors). The detected camera (for example, an infrared light camera or a visible light camera) detects these light sources, so that the position, orientation, and inclination of the HMD 110 in a specific space can be detected. Alternatively, for the same purpose, the HMD 110 may be provided with a camera for detecting a light source installed in the housing part 130 of the HMD 110.

Further, the housing part 130 of the HMD 110 can include an eye tracking sensor. The eye tracking sensor is used to detect the gaze direction and the gazing point of the left and right eyes of the performer. Various types of eye tracking sensors are conceivable. For example, the position of the reflected light on the cornea formed by irradiating the left eye and the right eye with weak infrared light is used as a reference point, and depending on the position of the pupil relative to the position of the reflected light. A method of detecting the line-of-sight direction and detecting the intersection of the left-eye and right-eye line-of-sight directions as a gazing point can be considered.

FIG. 2 is a schematic diagram of the appearance of the controller 210 according to the present embodiment. The controller 210 can support the performer to perform a predetermined input in the virtual space. The controller 210 may be configured as a set of left hand 220 and right hand 230 controllers. The left hand controller 220 and the right hand controller 230 may each include an operation trigger button 240, an infrared LED 250, a sensor 260, a joystick 270, and a menu button 280.

The operation trigger button 240 is arranged as 240a and 240b at positions that are assumed to perform an operation of pulling the trigger with the middle finger and the index finger when the grip 235 of the controller 210 is gripped. A frame 245 formed in a ring shape downward from both side surfaces of the controller 210 is provided with a plurality of infrared LEDs 250, and the position of these infrared LEDs is detected by a camera (not shown) provided outside the controller. Thus, the position, orientation and inclination of the controller 210 in a specific space can be detected.

In addition, the controller 210 can incorporate a sensor 260 in order to detect movements such as the orientation and inclination of the controller 210. Although not illustrated, the sensor 260 can include, for example, a magnetic sensor, an acceleration sensor, a gyro sensor, or a combination thereof. Furthermore, a joystick 270 and a menu button 280 can be provided on the upper surface of the controller 210. The joystick 270 can be moved 360 degrees around the reference point, and is assumed to be operated with the thumb when the grip 235 of the controller 210 is gripped. Similarly, it is assumed that the menu button 280 is operated with the thumb. Further, the controller 210 may incorporate a vibrator (not shown) for applying vibration to the hands of the performer who operates the controller 210. In order to output information such as the position, orientation, and tilt of the controller 210 via a sensor or the like through the input contents of a performer via a button or a joystick, and to receive information from the host computer, the controller 210 Part and a communication part.

The system determines the movement and posture of the performer's hand based on whether the performer holds the controller 210 and operates various buttons and joysticks, and information detected by the infrared LED and sensor. The performer's hand can be displayed and operated.

FIG. 3 is a configuration diagram of the HMD system 300 according to the present embodiment. The HMD system 300 can be configured by, for example, an HMD 110, a controller 210, and an image generation apparatus 310 that functions as a host computer. Furthermore, an infrared camera (not shown) for detecting the position, orientation, inclination, and the like of the HMD 110 and the controller 210 can be added. These devices can be connected to each other by wired or wireless means. For example, each device is equipped with a USB port, and communication can be established by connecting with a cable. In addition, HDMI (registered trademark), wired LAN, infrared, Bluetooth (registered trademark), WiFi (registered trademark) can be established. Communication can also be established by wired or wireless. The image generation apparatus 310 may be an apparatus having a calculation processing function such as a PC, a game machine, or a mobile communication terminal. Further, the image generation apparatus 310 can connect to a plurality of user apparatuses such as 401A, 401B, and 401C via a network such as the Internet, and transmit the generated image in a streaming or download form. Each of the user devices 401A and the like can reproduce an image transmitted by being provided with an Internet browser or an appropriate viewer. Here, the image generation device 310 can directly transmit an image to a plurality of user devices, or can transmit an image via another content distribution server.

FIG. 4 is a functional configuration diagram of the HMD 110 according to the present embodiment. As described in FIG. 1, the HMD 110 may include a sensor 140. As a sensor, although not shown in order to detect movements such as the head direction and inclination of the performer, for example, a magnetic sensor, an acceleration sensor, a gyro sensor, or a combination thereof can be provided. Further, an eye tracking sensor can be provided. The eye tracking sensor is used to detect the gaze direction and the gazing point of the left and right eyes of the performer. Furthermore, in order to detect the movement such as the direction and inclination of the performer's head with high accuracy and to detect the position of the performer's head, an LED 150 such as infrared light or ultraviolet light may be provided. Moreover, the camera 160 for image | photographing the outside scene of HMD can be provided. Further, a microphone 170 for collecting the performer's utterance and a headphone 180 for outputting sound can be provided. Note that the microphone and the headphone can be provided as an independent device from the HMD 110.

Furthermore, the HMD 110 can include, for example, an input / output unit 190 for establishing a wired connection with peripheral devices such as the controller 210 and the image generation device 310, and can include infrared, Bluetooth (registered trademark), and WiFi (registered trademark). The communication unit 115 for establishing a wireless connection can be provided. Information relating to the movement of the performer's head, such as the orientation and tilt of the performer, acquired by the sensor 140 is transmitted to the image generation device 310 by the control unit 125 via the input / output unit 190 and / or the communication unit 115. Although details will be described later, the image generated by the image generation device 310 based on the movement of the performer's head is received via the input / output unit 190 and / or the communication unit 115, and the display unit 120 is displayed by the control unit 125. Is output.

FIG. 5 is a functional configuration diagram of the controller 210 according to the present embodiment. As described in FIG. 2, the controller 210 can be configured as a set of controllers for the left hand 220 and the right hand 230, but in either controller, the operation trigger button 240, the joystick 270, An operation unit 245 such as a menu button 280 can be provided. In addition, the controller 210 can incorporate a sensor 260 in order to detect movements such as the orientation and inclination of the controller 210. Although not illustrated, the sensor 260 can include, for example, a magnetic sensor, an acceleration sensor, a gyro sensor, or a combination thereof. Further, a plurality of infrared LEDs 250 are provided, and the position, orientation, and inclination of the controller 210 in a specific space are detected by detecting the positions of these infrared LEDs with a camera (not shown) provided outside the controller. be able to. The controller 210 can include, for example, an input / output unit 255 for establishing a wired connection with a peripheral device such as the HMD 110 or the image generation device 310, such as infrared rays, Bluetooth (registered trademark), WiFi (registered trademark), or the like. A communication unit 265 for establishing a wireless connection can be provided. Information input by the performer via the operation unit 245 and information such as the orientation and inclination of the controller 210 acquired by the sensor 260 are transmitted to the image generation device 310 via the input / output unit 255 and / or the communication unit 265. .

FIG. 6 is a functional configuration diagram of the image generation apparatus 310 according to the present embodiment. The image generation device 310 stores information related to the head movement of the performer and the movement and operation of the controller acquired by the input information and sensors transmitted from the HMD 110 and the controller 210, performs predetermined calculation processing, and performs image processing. Devices such as a PC, a game machine, and a mobile communication terminal that have a function for generating can be used. The image generation device 310 can include an input / output unit 320 for establishing a wired connection with peripheral devices such as the HMD 110 and the controller 210, for example, infrared, Bluetooth (registered trademark), WiFi (registered trademark), etc. A communication unit 330 for establishing a wireless connection can be provided. Information on the head movement of the performer and the movement and operation of the controller received from the HMD 110 and / or the controller 210 via the input / output unit 320 and / or the communication unit 330 is transmitted to the control unit 340 in the position of the performer, Characters are controlled by executing a control program that is detected as input content including actions such as line of sight, posture, speech, operation, etc., and stored in the storage unit 350 in accordance with the input content of the performer. Is generated. The control unit 340 can be configured by a CPU, but by further providing a GPU specialized for image processing, information processing and image processing can be distributed, and overall processing efficiency can be improved. The image generation apparatus 310 can also communicate with other calculation processing apparatuses and share information processing and image processing with other calculation processing apparatuses.

Further, the control unit 340 of the image generation apparatus 310 is a user input detection unit that detects information about the head movement of the performer, the speech of the performer, and the movement and operation of the controller received from the HMD 110 and / or the controller 210. 610, a character control unit 620 that executes a control program stored in the control program storage unit for a character stored in advance in the character data storage unit 660 of the storage unit 350, and an image based on the character control An image generation unit 630 is included. Here, regarding the movement control of the character, information such as the orientation and tilt of the performer head detected by the HMD 110 or the controller 210 and the movement of the hand is created in accordance with the movement and restriction of the joints of the human body. This is realized by applying the motion of the bone structure by converting the motion of each part of the bone structure and associating the bone structure with the character data stored in advance. In addition, the control unit 340 includes an item receiving unit 640 that receives selection of items to be arranged in the virtual space from other user devices, and includes a comment receiving unit 650 that receives comments. The additional information generation unit 655 generates additional information based on the history information storing the received items and comments. The image generation unit 630 generates a virtual space including the character display area including the additional information and the background image and the user display area as shown in FIG. The screen of the virtual space can be displayed on the display unit of each user apparatus via the network, or can be displayed on the display unit 120 of the HMD 110 worn by the performer. In particular, the screen displayed on the display unit 120 of the HMD 110 worn by the performer can include information that can be displayed only on the performer, in addition to the additional information described above. In addition to the screen of the virtual space, a screen for performing the performer can be displayed on the display unit 120. In the character display area (area displayed on the image display unit 1110 in FIG. 7), the frame rate of the character image and the frame rate of the background image (landscape, still life, etc.) can be set at different speeds. For example, the frame rate of the character image can be set relatively low by setting the frame rate of the character image to 8 fps and the frame rate of the background image to 30 fps. In this manner, the motion of the character image can be shown without a sense of incongruity by setting the frame rate close to the animation that is not the existing 3DCG.

The storage unit 350 stores information related to the character, such as character attributes, in addition to the character image data, in the character data storage unit 660 described above. In addition, the control program storage unit 670 stores a program for controlling the movement and facial expression of a character in the virtual space and a program for generating a virtual space including content such as a character and a user avatar. The streaming data storage unit 680 stores the image generated by the image generation unit 630. The virtual space image stored as the stream data can be simultaneously distributed together with the live image in response to a user request. The storage unit 350 includes an item data storage unit 685 that stores data related to items. Furthermore, the storage unit 350 includes a user data storage unit 690 that stores information related to the user and a user avatar. The image generation apparatus according to the present embodiment is characterized by not only transmitting an image that is a virtual space to a plurality of user apparatuses, but also receiving items and comments from the user apparatuses. It should be noted that all or part of the functions of the image generation apparatus 310 are specialized only for generating an image, a separate content service server is provided, the content service server transmits an image to the user apparatus, and It is also possible to have a function of receiving items and comments from the user device.

FIG. 7 is a diagram illustrating an example of the virtual space 1100 displayed on the user device according to the present embodiment. As shown in FIG. 7, the user device can display an image of a virtual space including the character 1120 in an image display unit 1110 such as a viewer for displaying an image embedded in a built-in web browser. The character 1120 arranged in the virtual space is a movement such as the tilt and orientation of the performer's head, the utterance content of the performer, or the tilt of the controller 210 via the HMD 110 and / or the controller 210 attached to the performer as a performer. It is possible to operate based on user input such as a movement such as a direction and an operation content of a performer via the controller 210. Further, in the virtual space 1110, an area 1130 for displaying a plurality of user avatars serving as audiences is provided. Furthermore, the virtual space 1100 includes a comment input unit 1140 for a user to input a comment and a post, and a user for a gift item. And a gift item selection unit 1150 for posting. The comment input and posted by the user is displayed in a predetermined area 1160 such as a balloon displayed in the vicinity of the corresponding user avatar position in FIG. 7, for example.

An example of processing for providing a virtual space to a user in the present embodiment will be described with reference to FIGS. 8 and 13. First, the item receiving unit 640 or the comment receiving unit 650 of the control unit 340 receives a predetermined action (posting a gift item or a comment) from the user device (S101). Next, the item reception unit 640 or the comment reception unit 650 of the control unit 340 stores the received gift item or comment in the user data storage unit 690 (S102). More specifically, the action received in the user management table shown in FIG. 10 is stored as history information, and if the received action includes a gift item post, the points consumed to purchase the item (pt ). As the history information, the user gift history management table shown in FIG. 11 can be updated. In FIG. 11, for example, when user A purchases a stuffed animal as a gift item, the user gift history management table stores information such as event date, posted gift (stuffed animal), consumed pt (50 pt), and the like. Update the table. Also, the user comment history table shown in FIG. 12 can be updated as history information. In FIG. 12, for example, when the user A posts a comment “I will come again”, the table is updated by storing the event date and time and the content of the posted comment in the user comment history management table.

Returning to the description of FIG. 13, the additional information generation unit 655 of the control unit 340 then stores the user management table, the user gift history management table, and / or the user comment history table stored in the user data storage unit 690. Reference is made (S103). Next, the additional information generation unit 655 generates additional information related to the predetermined user (S104). For example, when a predetermined action is received from the user, the additional information generation unit 655 generates information associated with the user avatar corresponding to the user A as illustrated in FIG. As an example of the content of the information, information such as “user name: Yoshi”, “latest history: box (post)”, “consumption pt: 1500”, and “comment: I will come back” can be generated. The image generation unit 630 generates an image provided to the user from the generated additional information together with elements (for example, a character and a user avatar) constituting another virtual space (S105). As shown in FIG. 8, the additional information is displayed in the vicinity of the user avatar provided to the user and displayed in the virtual space (in FIG. 8, inside the balloon 1170). Here, the generated additional information may be all or a part of the information illustrated in FIG. 8. When the user takes some action, the additional information related to the user is changed to the user. By displaying it in association with the user avatar corresponding to (for example, in the vicinity of the user avatar), the performer can immediately detect information related to the user, and can express words of gratitude and effort to the user. Can do. As the amount of information displayed increases, the performer can refer to the information and put a special word on the user. Alternatively, as additional information, a history of comments and / or gift item postings made by the user over a predetermined period in the past (for example, the past week) can be displayed in a list. Thereby, the performer can trace each past action performed by the user, and can widen the range of communication.

Further, the range in which the additional information is displayed can be an unspecified user that is not limited to a specific user, or can be limited to the performer and its target user from the viewpoint of privacy, or can be displayed only to the performer. By doing so, the performer can refer to the user's information without being known to the user, and can provide a special effect to the user.

Next, another example of processing for providing a virtual space to the user in the present embodiment will be described with reference to FIGS. 9 and 14. First, the image generation unit 690 of the control unit 340 generates an image corresponding to the gift item posted by the user (S201), and the generated gift item image 1180 is a character image in the virtual space 1100 of FIG. 1120 is displayed. Subsequently, the additional information generation unit 655 of the control unit 340 determines whether or not the character 1120 in FIG. 9 has touched the displayed gift item 1180 (S202). More specifically, the positional relationship between the characters 1120 in the virtual space is confirmed by comparing the coordinates of the hand of the character 1120 with the coordinates corresponding to the area where the gift item 1180 is placed. The performer, for example, moves the controller 210 of FIG. 2 up and down, so that the input detection unit 610 of the control unit 340 detects a change in the position of the controller 210 via the acceleration sensor of the controller 210, and the character control unit 620 controls to move the arm of the character 1120 up and down according to the detected change in position, and the performer has the palm position of the tip of the arm of the character 1120 approximately coincident with the position of the gift item 1180. By operating a predetermined button (for example, trigger button 240b) of the controller 210 at the timing, the character control unit 620 can control the character 1120 to hold the gift item 1180.

If it is determined in S202 that the character 1120 has touched the gift item 1180 (“YES” in S202), the additional information generation unit 655 first identifies the user who posted the gift item 1180, and the user's User data is referred to (S203). If it is determined in S202 that the character 1120 has not touched the gift item 1180 (“No” in S202), the process returns to the original process and monitoring is continued. There are several methods for specifying the user who posted the item from the gift item 1180 touched by the character 1120. For example, there is a method for specifying the purchaser by referring to purchase history information of the gift item. When the user who is the purchaser is identified, the additional information generation unit 655 refers to the user management table, the user gift history management table, and / or the user comment history table stored in the user data storage unit 690, Additional information related to the user is generated (S204). For example, when the user ID is “user A”, as shown in FIG. 9, information associated with the user avatar corresponding to the user A is generated. As an example of the content of the information, information such as “user name: Yoshi”, “latest history: box (post)”, “consumption pt: 1500”, and “comment: I will come back” can be generated. The image generation unit 630 generates an image to be provided to the user from the generated additional information together with elements (for example, characters and user avatars) constituting other virtual spaces (S205). As shown in FIG. 9, the additional information is displayed in the vicinity of the user avatar provided to the user and displayed in the virtual space (in FIG. 9, inside the balloon 1170). Other aspects are the same as the example described with reference to FIGS.

In FIG. 8 and FIG. 9, information related to the user is triggered by the fact that the user has taken a predetermined action or that the character has taken a predetermined action on the gift item posted by the user. In addition to the example of displaying, the performer's viewpoint direction in the virtual space and the viewpoint of the performer detected through the eye tracking sensor, which are detected through various sensors of the HMD 110 attached to the performer. An aspect of displaying information related to a user corresponding to a user avatar located in the area is also conceivable.

Second Embodiment
FIG. 15 shows a flowchart as an example of processing for providing a virtual space to a user in the second embodiment. First, the image generation unit 630 of the control unit 340 generates a user avatar image of a predetermined user stored in the user data storage unit 690, and performs a display process (S301). Subsequently, the additional information generation unit 655 refers to the user management table stored in the user data storage unit 690, and confirms information such as the number of logins and the last login date of the user, so that the user can use the service for the first time It is confirmed whether to log in or browse a room provided by the performer (a page on which the performer provides content) (S302). When it is confirmed that the user logs in to the service for the first time or the performer browses a page that provides content (“YES” in S302), as shown in FIG. 16, the additional information generation unit 655 displays an image. The generating unit performs processing for adding a special effect to the user avatar (S303). In the process of S302, when it is confirmed that the user is a second or subsequent login or page viewer (“No” in S302), the process returns to the original process. In the process of S303, as shown in FIG. 16, a display 1270 for notifying that the user has logged in for the first time can be performed. Here, as an example of the special effect, a process for making the user avatar distinguishable from other user avatars such as a process for highlighting the user avatar, a process for blinking, and a process for enlarging the display can be considered. This special effect may be visible only to the performer.

<Third Embodiment>
FIG. 17 shows a flowchart as an example of processing for providing a virtual space to a user in the third embodiment. First, the image generation unit 630 of the control unit 340 generates an image of a predetermined gift item stored in the item data storage unit 685, and performs display processing (S401). Subsequently, the additional information generation unit 655 instructs the image generation unit 630 to apply a special effect to the displayed gift item, and the image generation unit 630 performs processing for adding special processing (S402). Here, as an example of the special effect, a process for attracting the performer's attention to the gift item, such as a process of highlighting a gift item, a process of blinking, or a process of enlarging display, can be considered. This special effect may be visible only to the performer. Further, referring to the user's history information, a special effect can be given to the gift item based on the information that the user has purchased and posted the gift item for the first time.

The additional information generation unit 655 confirms whether the performer has made any reaction to the posted gift item (S403). For example, in FIG. 18, it is confirmed whether the performer has read a comment (for example, “Thank you, user A”) about the ice cream gift item posted by the user. Here, as a confirmation method, a voice recognition technology is used to recognize a word related to a corresponding gift item or a corresponding user name from a voice uttered by a performer, or a gift is given through a character played by the performer. A method of detecting a predetermined action on an item (for example, a character touching a gift item) can be considered.

In the process of S403, when it is detected that the performer has reacted to the gift item (YES in S403), the additional information generation unit 655 cancels the special effect applied to the gift item (S404). Thereby, the performer can sensuously understand that some kind of reaction has been shown to the gift item posted by the user, and can confirm the gift item that has not yet been reacted. In the process of S403, when it is not detected that the performer has reacted to the gift item (YES in S403), the special effect is maintained for the gift item. The invention disclosed in the present embodiment can be similarly applied to comments posted by the user.

<Fourth embodiment>
FIG. 19 shows an example of a virtual space displayed to the user in the fourth embodiment. In the present embodiment, an item image 1470 related to a predetermined product sold in the real world is displayed in the character display area of the virtual space 1400, and information 1480 related to the product is displayed when a predetermined condition is satisfied. Display in the virtual space. For example, when the positional relationship between the character 1410 displayed in the character display area and the item image 1470 becomes a predetermined distance, or there is an interaction between the character 1410 and the item image 1470 (for example, the character is When the item image is grasped), information 1480 related to the product can be displayed. The information 1480 related to the product can be, for example, the product name, the seller, the price, the contents explaining the product, etc., and these information are acquired from the external server via the application program interface. It can also be stored in advance as product related information in the image generating apparatus 310 together with the item image 1470 related to the product.

<Fifth Embodiment>
FIG. 20 shows an example of a virtual space displayed on the performer in the fifth embodiment. A virtual space 1500 shown in FIG. 20 is displayed on the display unit 120 or a monitor (not shown) of the HMD 110 worn by the performer. As shown in FIG. 20A, the virtual space 1500 also includes a service image display area 1550 for displaying the virtual space displayed on the user device 401, a character 1510 displayed in the area 1550, and a predetermined object (for example, , Comment list) 1520 has an operation image display area 1560 for operating. In the operation image display area 1560, an image corresponding to the image arranged in the service image display area is basically arranged. For example, the operation image display area performer moves the finger of the character 1510 by operating the controller 210 while referring to the reference image 1540 for character finger operation. As a character hand / finger operation reference image 1540, an operation hand is moved to a position that is shifted by a predetermined distance from the approximate coordinates of the position of the performer's hand instruction image 1570 acquired via the position of the controller 210 (in the depth direction of the screen). An image 1580 can be displayed. As shown in FIG. 20B, the performer grasps a part (comment 1590) of the object 1530 (comment list) displayed in the operation image display area 1560 while referring to the image 1580 of the operation hand. By performing the operation, the gripping position of the object (comment) of the character 1510 is reflected at the position shifted by the predetermined distance, so that the action of the character gripping the object with the fingertip can be expressed in a more natural form. it can.

In this embodiment, as shown in FIG. 20A, a list 1520 of comments posted by the user is displayed near the character, and a corresponding comment list 1530 is also displayed in the operation image display area 1560. . As shown in FIG. 20B, when the performer operates the reference image 1540 for finger operation and grasps a part of the comment (for example, the comment “Nemu” posted by the user name “hide”) 1590, Of the comment list displayed near the character, the corresponding comment is held by the character. Further, when the performer grasps the comment 1590, additional information regarding the user “Hide” who posted the comment can be displayed. Accordingly, the performer can make a special inquiry to the user while reading the user comment and referring to the information about the user. Here, instead of the additional information, a special effect can be used as an effect based on the content of the comment. For example, in FIG. 20B, when the performer grasps a comment “congratulations” posted by “Rina”, an effect can be used in which fireworks and cakes are displayed on the screen. Also, regarding the display method of the comment list, the comment list 1520 displayed in the service image display area 1550 and the comment list 1530 displayed in the operation image display area 1560 are highlighted, so that the viewing user can see the comment list. The content of the comment list may be understood by the performer (or the character 1510 played by the performer).

FIG. 21 shows an example of a virtual space displayed to the user in the sixth embodiment. In the present embodiment, the gift item 1620 posted from the user to the character 1610 shown in FIG. 21A is changed to another gift item 1630 as shown in FIG. That is, the user purchases a gift item in advance through a purchasing means 1150 as shown in FIG. 7 in exchange for a point, and in FIG. 21A, a gift box 1620 is posted as a gift item at a specified date and time. It can be provided (delivered) and presented as a gift item 1630 (for example, tea) shown in FIG. 21B according to the action of the character 1120 (such as contact with the gift item 1620). it can. At this time, the additional information 1640 indicating the contents of the gift item 1630 and the poster can be displayed to the user at the time of opening.

110 Head Mount Display 115 Communication Unit 120 Display Panel 125 Control Unit 140 Sensor 150 Light Source 160 Camera 170 Microphone 180 Headphone 190 Input / Output Unit 210 Controller 220 Left Hand Controller 230

Right Hand Controller

240a, 240b Trigger Button 245 Operation Unit 250 Infrared LED
255 Input / output unit 260 Sensor 270 Joystick 280 Menu button 290 Frame 310 Image generation device 320 Input / output unit 330 Communication unit 340 Control unit 350

Storage unit

401A, 401B, 401C User device
610 User input detection unit 620 Character control unit 630 Image generation unit 640 Item reception unit 650 Comment reception unit 655 Additional information generation unit 660 Character data storage unit 670 Control program storage unit 680 Streaming data storage unit 685 Item data storage unit 690 User data storage Section 1100 Virtual space 1110 Image display section 1120 Character 1130 User information display area 1140 Comment input section 1150 Item selection section 1160 Additional information display area 1170 Additional information display area 1180 Gift item 1200 Virtual space 1270 Additional information display area 1400 Virtual space 1410 Character 1470 Gift item 1480 Additional information display area 1500 Virtual space 1510 Character 1520 Object 1530 Object
1540 Character finger operation reference image 1550 Service image display area 1560 Operation image display area 1570 Performer hand instruction image 1580 Operation hand image 1590 Comment 1600 Virtual space 1610 Character 1620 Gift item 1630 Gift item 1640 Additional information display area

Claims

A method for providing a virtual space having a predetermined content including a character played by a performer user to a plurality of users including a performer user and a viewing user,
Receive a predetermined action from the user,
Store the received predetermined action as history information for each user,
Generating a virtual space including the predetermined content and information for identifying the user;
A method for providing a virtual space, wherein information generated based on the history information is displayed.
A method for providing a virtual space according to claim 1, comprising:
The method for providing a virtual space, wherein information related to the action is provided to only some of the one or more users.
A method for providing a virtual space according to claim 2, comprising:
The method of providing a virtual space, wherein the some users are performer users.
A method for providing a virtual space according to claim 1, comprising:
The method for providing a virtual space, wherein the predetermined action includes at least one of user participation, comment posting, and item purchase by the user.
A method for providing a virtual space according to claim 1, comprising:
The method for providing a virtual space, wherein the information for identifying the user is a user name.
A method for providing a virtual space according to claim 1, comprising:
The method for providing a virtual space, wherein the information for identifying the user is an avatar image of the user.
A method for providing a virtual space according to claim 1, comprising:
The method of providing a virtual space, wherein the history information is information obtained by accumulating a plurality of actions for each user for a predetermined period.
A method for providing a virtual space according to claim 1, comprising:
A method for providing a virtual space, characterized by displaying information generated based on the history information when the action of the character and the predetermined action act in both directions.
A method for providing a virtual space according to claim 1, comprising:
The method for providing a virtual space, wherein the predetermined action is an item purchase by a user, and a special effect is given to an item purchased by the user based on the history information.
A method for providing a virtual space based on the history information.
A method for providing a virtual space according to claim 1, comprising:
A method for providing a virtual space, comprising: accessing an application used by the user; and displaying information related to the user together with information generated based on the history information.