JPWO2019216249A1 - How to provide a virtual space with given content - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method capable of more effectively promoting an interaction between a performer and a user in a virtual space. A method of providing a plurality of users, including a performer user and a viewing user, with a virtual space having a predetermined content including a character played by the performer user, receiving a predetermined action from the user. It is characterized in that a predetermined received action is stored as history information for each user, a virtual space including a predetermined content and information for identifying a user is generated, and information generated based on the history information is displayed. Provide virtual space. [Selection diagram] FIG. 8

Description

The present invention relates to a method of providing a virtual space for a plurality of users. More specifically, the present invention relates to a method of providing a virtual space having a predetermined content including a character played by a performer.

Motion capture is a technology that digitally captures the movements of performer users (hereinafter referred to as "performers") in real space, and the captured movements are used for computer animation such as movies and expression of character movements in games and the like. To.

Further, in recent years, a method of controlling a character in a virtual space has been devised through the movement of a head-mounted display (hereinafter, “HMD”) worn by a performer and the input by a controller held by the performer.

In addition, content sharing services that deliver content such as live videos to users are provided, and in such content sharing services, it is possible to acquire items to express a positive evaluation of content posters and performers. A mechanism for encouraging purchases is disclosed. (For example, Patent Document 1)

Japanese Unexamined Patent Publication No. 2015-090526

The technology disclosed in the above document provides a mechanism for determining the display position of the user avatar in the virtual space according to the amount of items purchased by the user in order to cultivate a desire for approval of the user or a sense of competition between the users. ing. However, although the present technology can satisfy the user's desire for approval by displaying the display position of the user avatar closer to the performer, there is no interaction or communication with the performer.

Therefore, an object of the present invention is to provide a method capable of more effectively promoting the interaction between the performer and the user in the virtual space.

A method provided by one embodiment of the present invention for providing a plurality of users, including a performer user and a viewing user, with a virtual space having predetermined content including a character played by the performer user is determined by the user. Receives an action, stores the received predetermined action as history information for each user, creates a virtual space including predetermined content and information that identifies the user, and displays the information generated based on the history information. It provides a virtual space characterized by the fact that.

According to the present invention, it is possible to more effectively promote interaction and communication between a performer and a user in a virtual space. As a result, the user's desire for approval is further satisfied.

A schematic view of the appearance of the head-mounted display 110 according to the first embodiment is shown. A schematic diagram of the appearance of the controller 210 according to the first embodiment is shown. The block diagram of the HMD system 300 which concerns on 1st Embodiment is shown. The functional block diagram of the HMD 110 according to the 1st Embodiment is shown. The functional block diagram of the controller 210 which concerns on 1st Embodiment is shown. The functional block diagram of the image generation apparatus 310 which concerns on 1st Embodiment is shown. An example of the virtual space displayed on the user device according to the first embodiment is shown. Another example of the virtual space displayed on the user apparatus according to the first embodiment is shown. Yet another example of the virtual space displayed on the user apparatus according to the first embodiment is shown. An example of the user management table according to the first embodiment is shown. An example of the user gift history management table according to the first embodiment is shown. An example of the user comment history management table according to the first embodiment is shown. In the first embodiment, a flowchart as an example of a process of providing a virtual space to a user is shown. In the first embodiment, a flowchart as another example of the process of providing the virtual space to the user is shown. In the second embodiment, a flowchart as an example of the process of providing the virtual space to the user is shown. In the second embodiment, an example of the virtual space displayed to the user is shown. In the third embodiment, a flowchart as an example of the process of providing the virtual space to the user is shown. In the third embodiment, an example of the virtual space displayed to the user is shown. In the fourth embodiment, an example of the virtual space displayed to the user is shown. In the fifth embodiment, an example of the virtual space displayed on the performer is shown. In the sixth embodiment, an example of the virtual space displayed to the user is shown.

<Embodiment 1>
A specific example of a program for controlling a head-mounted display system according to an embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, and is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims. In the following description, the same elements will be designated by the same reference numerals in the description of the drawings, and duplicate description will be omitted.

FIG. 1 shows a schematic view of the appearance of the head-mounted display (hereinafter referred to as HMD) 110 according to the present embodiment. The HMD 110 is attached to the head of the performer and includes a display panel 120 so as to be arranged in front of the left and right eyes of the performer. As the display panel, an optical transmissive type and a non-transmissive type display can be considered, but in the present embodiment, a non-transmissive type display panel capable of providing a more immersive feeling is exemplified. An image for the left eye and an image for the right eye are displayed on the display panel 120, and a stereoscopic image can be provided to the performer by utilizing the parallax of both eyes. If the image for the left eye and the image for the right eye can be displayed, the display for the left eye and the display for the right eye can be provided separately, or an integrated display for the left eye and the right eye can be provided. is there.

Further, the housing 130 of the HMD 110 includes a sensor 140. The sensor may include, for example, a magnetic sensor, an acceleration sensor, or a gyro sensor, or a combination thereof, for detecting movements such as the orientation and tilt of the performer's head, although not shown. The vertical direction of the performer's head is the Y-axis, and of the axes orthogonal to the Y-axis, the axis that connects the center of the display panel 120 and the performer and corresponds to the front-back direction of the performer is the Z-axis, and the Y-axis and the Z-axis. When the axis corresponding to the left-right direction of the performer is the X-axis, the sensor 140 has a rotation angle around the X-axis (so-called pitch angle), a rotation angle around the Y-axis (so-called yaw angle), and Z. The rotation angle around the axis (so-called roll angle) can be detected.

Further, or instead of the sensor 140, the housing 130 of the HMD 110 may be provided with a plurality of light sources 150 (for example, infrared light LED, visible light LED) and installed outside the HMD 110 (for example, indoors). By detecting these light sources by a camera (for example, an infrared light camera or a visible light camera), the position, orientation, and inclination of the HMD 110 in a specific space can be detected. Alternatively, for the same purpose, the HMD 110 may be provided with a camera for detecting a light source installed in the housing 130 of the HMD 110.

Further, the housing 130 of the HMD 110 may be provided with an eye tracking sensor. Eye tracking sensors are used to detect the gaze direction and gaze point of the performer's left and right eyes. Various types of eye tracking sensors can be considered. For example, the position of the reflected light on the cornea formed by irradiating the left eye and the right eye with weak infrared light is used as a reference point, and the position of the pupil with respect to the position of the reflected light is used as a reference point. A method of detecting the line-of-sight direction and detecting the intersection of the line-of-sight directions of the left eye and the right eye as the gazing point can be considered.

FIG. 2 shows a schematic view of the appearance of the controller 210 according to the present embodiment. The controller 210 can support the performer to make a predetermined input in the virtual space. The controller 210 can be configured as a set of controllers for the left hand 220 and the right hand 230. The left-hand controller 220 and the right-hand controller 230 can each have an operation trigger button 240, an infrared LED 250, a sensor 260, a joystick 270, and a menu button 280.

The operation trigger button 240 is arranged as 240a and 240b at positions assuming that an operation such as pulling the trigger with the middle finger and the index finger is performed when the grip 235 of the controller 210 is gripped. A plurality of infrared LEDs 250 are provided on the frame 245 formed in a ring shape downward from both side surfaces of the controller 210, and the positions of these infrared LEDs are detected by a camera (not shown) provided outside the controller. The position, orientation and tilt of the controller 210 in a specific space can be detected.

Further, the controller 210 can incorporate a sensor 260 in order to detect movements such as the orientation and tilt of the controller 210. Although not shown, the sensor 260 may include, for example, a magnetic sensor, an acceleration sensor, a gyro sensor, or a combination thereof. Further, a joystick 270 and a menu button 280 can be provided on the upper surface of the controller 210. The joystick 270 can be moved in the 360-degree direction about the reference point, and it is assumed that the joystick 270 is operated by the thumb when the grip 235 of the controller 210 is gripped. It is assumed that the menu button 280 is also operated with the thumb. Further, the controller 210 may also include a vibrator (not shown) for giving vibration to the hands of the performer who operates the controller 210. The controller 210 inputs and outputs in order to output information such as the input contents of the performer via a button or a joystick and the position, orientation and tilt of the controller 210 via a sensor or the like, and to receive information from the host computer. It has a unit and a communication unit.

The system determines the movement and posture of the performer's hand based on whether the performer holds the controller 210 and operates various buttons and joysticks, and the information detected by the infrared LED and the sensor, and simulates in the virtual space. The performer's hand can be displayed and operated.

FIG. 3 shows a configuration diagram of the HMD system 300 according to the present embodiment. The HMD system 300 can be composed of, for example, an HMD 110, a controller 210, and an image generator 310 that functions as a host computer. Further, an infrared camera (not shown) or the like for detecting the position, orientation, tilt, etc. of the HMD 110 or the controller 210 can be added. These devices can be connected to each other by wired or wireless means. For example, each device is equipped with a USB port and communication can be established by connecting with a cable. In addition, HDMI (registered trademark), wired LAN, infrared ray, Bluetooth (registered trademark), WiFi (registered trademark) ) Etc. can also be established by wire or wireless. The image generation device 310 may be a device having a calculation processing function such as a PC, a game machine, or a mobile communication terminal. Further, the image generation device 310 can be connected to a plurality of user devices such as 401A, 401B, and 401C via a network such as the Internet, and the generated image can be transmitted in the form of streaming or download. The user device 401A and the like can reproduce the transmitted image by providing an Internet browser or an appropriate viewer. Here, the image generation device 310 can directly transmit an image to a plurality of user devices, or can transmit an image via another content distribution server.

FIG. 4 shows a functional configuration diagram of the HMD 110 according to the present embodiment. As described in FIG. 1, the HMD 110 can include a sensor 140. The sensor may include, for example, a magnetic sensor, an acceleration sensor, a gyro sensor, or a combination thereof, although not shown, in order to detect movements such as the orientation and tilt of the performer's head. In addition, an eye tracking sensor can be provided. Eye tracking sensors are used to detect the gaze direction and gaze point of the performer's left and right eyes. An LED 150 such as infrared light or ultraviolet light can be provided to more accurately detect movements such as the orientation and tilt of the performer's head and to detect the position of the performer's head. In addition, a camera 160 for photographing the outside view of the HMD can be provided. Further, a microphone 170 for collecting the utterances of the performer and a headphone 180 for outputting the voice can be provided. The microphone and headphones can also be provided as a device separate from the HMD 110.

Further, the HMD 110 can include, for example, an input / output unit 190 for establishing a wired connection with peripheral devices such as a controller 210 and an image generation device 310, and can include infrared rays, Bluetooth (registered trademark), and WiFi (registered trademark). ) Etc. A communication unit 115 for establishing a wireless connection can be provided. Information on movement such as the orientation and tilt of the performer's head acquired by the sensor 140 is transmitted by the control unit 125 to the image generation device 310 via the input / output unit 190 and / or the communication unit 115. Although the details will be described later, in the image generation device 310, the image generated based on the movement of the performer's head is received via the input / output unit 190 and / or the communication unit 115, and is received by the control unit 125 by the display unit 120. Is output to.

FIG. 5 shows a functional configuration diagram of the controller 210 according to the present embodiment. As described in FIG. 2, the controller 210 can be configured as a set of controllers for the left hand 220 and the right hand 230, and in any of the controllers, the operation trigger button 240, the joystick 270, and the like. An operation unit 245 such as a menu button 280 can be provided. Further, the controller 210 can incorporate a sensor 260 in order to detect movements such as the orientation and tilt of the controller 210. Although not shown, the sensor 260 may include, for example, a magnetic sensor, an acceleration sensor, a gyro sensor, or a combination thereof. Further, a plurality of infrared LEDs 250 are provided, and a camera (not shown) provided outside the controller detects the position of these infrared LEDs to detect the position, orientation, and inclination of the controller 210 in a specific space. be able to. The controller 210 can include, for example, an input / output unit 255 for establishing a wired connection with peripheral devices such as the HMD 110 and the image generation device 310, such as infrared rays, Bluetooth (registered trademark), WiFi (registered trademark), and the like. A communication unit 265 for establishing a wireless connection can be provided. The information input by the performer via the operation unit 245 and the information such as the orientation and inclination of the controller 210 acquired by the sensor 260 are transmitted to the image generator 310 via the input / output unit 255 and / or the communication unit 265. ..

FIG. 6 shows a functional configuration diagram of the image generation device 310 according to the present embodiment. The image generator 310 stores input information transmitted from the HMD 110 or the controller 210, information on the movement or operation of the performer's head acquired by a sensor or the like, and information on the movement and operation of the controller, performs predetermined calculation processing, and produces an image. Devices such as PCs, game consoles, mobile communication terminals, etc., which have a function for generating can be used. The image generation device 310 can include, for example, an input / output unit 320 for establishing a wired connection with peripheral devices such as the HMD 110 and the controller 210, such as infrared rays, Bluetooth (registered trademark), WiFi (registered trademark), and the like. A communication unit 330 for establishing a wireless connection can be provided. Information regarding the movement of the head of the performer and the movement and operation of the controller received from the HMD 110 and / or the controller 210 via the input / output unit 320 and / or the communication unit 330 is transmitted in the control unit 340 to the position of the performer. It is detected as input content including movements such as line of sight and posture, utterance, operation, etc., and the character is controlled by executing the control program stored in the storage unit 350 according to the input content of the performer, and the image is displayed. Is performed. The control unit 340 can be configured by a CPU, but by further providing a GPU specialized for image processing, information processing and image processing can be dispersed and overall processing efficiency can be improved. The image generation device 310 can also communicate with another calculation processing device and have the other calculation processing device share information processing and image processing.

Further, the control unit 340 of the image generation device 310 is a user input detection unit that detects information regarding the movement of the performer's head, the performance's speech, and the movement and operation of the controller received from the HMD 110 and / or the controller 210. An image is generated based on the character control, the character control unit 620 that executes the control program stored in the control program storage unit, and the character stored in the character data storage unit 660 of the storage unit 350 in advance. It has an image generation unit 630. Here, regarding the control of the movement of the character, information such as the orientation and inclination of the performer's head and the movement of the hand detected via the HMD 110 and the controller 210 was created according to the movement and restrictions of the joints of the human body. It is realized by applying the movement of the bone structure by converting it into the movement of each part of the bone structure and associating the bone structure with the character data stored in advance. In addition, the control unit 340 has an item reception unit 640 that accepts selection of items to be arranged in the virtual space from another user device, and also has a comment reception unit 650 that accepts comments. The additional information generation unit 655 generates additional information based on the history information storing the received items and comments. The image generation unit 630 generates a virtual space including a character display area including additional information and a background image and a user display area as shown in FIG. 7. The screen of the virtual space can be displayed on the display unit of each user device via the network, or can be displayed on the display unit 120 of the HMD 110 worn by the performer. In particular, the screen displayed on the display unit 120 of the HMD 110 worn by the performer can include information displayed only on the performer, including the above-mentioned additional information. Further, in addition to the screen of the virtual space, the screen for operating the performer can be displayed on the display unit 120. Further, in the character display area (the area displayed on the image display unit 1110 in FIG. 7), the frame rate of the character image and the frame rate of the background image (landscape, still life, etc.) can be set at different speeds. For example, the frame rate of the character image can be set to 8 fps and the frame rate of the background image can be set to 30 fps to make the frame rate of the character image relatively low. In this way, by setting the frame rate close to the existing animation that is not 3DCG, the operation of the character image can be shown without discomfort.

The storage unit 350 stores the character image data as well as information related to the character such as character attributes in the character data storage unit 660 described above. Further, the control program storage unit 670 stores a program for controlling the movement and facial expression of the character in the virtual space and a program for generating a virtual space including contents such as the character and the user avatar. The streaming data storage unit 680 stores the image generated by the image generation unit 630. The image of the virtual space stored as stream data can be simultaneously distributed together with the live image at the request of the user. The storage unit 350 has an item data storage unit 685 that stores data related to the item. Further, the storage unit 350 has a user data storage unit 690 for storing information related to the user and the user avatar. A feature of the image generation device in the present embodiment is that it can not only transmit an image that becomes a virtual space to a plurality of user devices, but also accept items and comments from the user devices. It should be noted that all or part of the functions of the image generation device 310 are specialized only for generating an image, and a content service server is separately provided, and the content service server transmits the image to the user device and also. , It can also have functions such as accepting items and comments from the user device.

FIG. 7 is a diagram showing an example of the virtual space 1100 displayed on the user device according to the present embodiment. As shown in FIG. 7, the user device can display an image of the virtual space including the character 1120 on the image display unit 1110 such as a viewer for displaying the image embedded in the built-in web browser. The character 1120 arranged in the virtual space moves such as the tilt and orientation of the performer's head, the utterance content of the performer, or the tilt of the controller 210 via the HMD 110 and / or the controller 210 attached to the performer as a performer. It can operate based on user input such as movements such as or direction, and the operation contents of the performer via the controller 210. Further, in the virtual space 1110, an area 1130 for displaying a plurality of user avatars serving as an audience is provided, and further, in the virtual space 1100, a comment input unit 1140 for the user to input and post a comment, and a gift item for the user. Has a gift item selection unit 1150 to select and post. The comment input and posted by the user is displayed in a predetermined area 1160 such as a balloon displayed in the vicinity of the corresponding user avatar position in FIG. 7, for example.

An example of the process of providing the virtual space to the user in the present embodiment will be described with reference to FIGS. 8 and 13. First, the item receiving unit 640 or the comment receiving unit 650 of the control unit 340 receives a predetermined action (posting of a gift item or comment) from the user device (S101). Next, the item receiving unit 640 or the comment receiving unit 650 of the control unit 340 stores the received gift item or comment in the user data storage unit 690 (S102). More specifically, the action received in the user management table shown in FIG. 10 is stored as history information, and if the received action includes a gift item post, the points consumed for purchasing the item (pt). ) Is updated. As the history information, the user gift history management table shown in FIG. 11 can also be updated. In FIG. 11, for example, when the user A purchases a stuffed animal as a gift item, the user gift history management table stores information such as the event date and time, the posted gift (plush toy), and the consumed pt (50 pt). , Update the table. Further, as the history information, the user comment history table shown in FIG. 12 can be updated. In FIG. 12, for example, when user A posts a comment saying "I will come again", the table is updated by storing the event date and time and the posted comment content in the user comment history management table.

Returning to the description of FIG. 13, the additional information generation unit 655 of the control unit 340 then displays the user management table, the user gift history management table, and / or the user comment history table stored in the user data storage unit 690. Refer to (S103). Next, the additional information generation unit 655 generates additional information related to a predetermined user (S104). For example, when a predetermined action is received from the user, the additional information generation unit 655 generates information associated with the user avatar corresponding to the user A, as shown in FIG. As an example of the content of the information, information such as "user name: Yoshi", "latest history: box (posted)", "consumption pt: 1500", and "comment: will come again" can be generated. The image generation unit 630 generates an image provided to the user by using the generated additional information together with other elements constituting the virtual space (for example, a character or a user avatar) (S105). As shown in FIG. 8, the additional information is displayed in the vicinity of the user avatar provided to the user and displayed in the virtual space (in FIG. 8, inside the balloon 1170). Here, the additional information generated may be all or part of the information illustrated in FIG. 8, and when the user takes some action, the additional information related to the user is obtained by the user. By displaying in association with the corresponding user avatar (for example, in the vicinity of the user avatar), the performer can immediately detect the information related to the user and express gratitude and effort to the user. Can be done. The more information that is displayed, the more the performer can refer to that information and give the user a special word. Alternatively, as additional information, the history of comments and / or gift item postings made by the user over a predetermined period in the past (for example, the past week) can be displayed in a list. As a result, the performer can trace each of the past actions performed by the user, and can broaden the range of communication.

In addition, the range in which the additional information is displayed can be an unspecified user who is not limited to a specific user, can be limited to the performer and its target user from the viewpoint of privacy, or can be displayed only to the performer. By letting the performer refer to the user's information without being known to the user, it is possible to provide the user with a special effect.

Next, another example of the process of providing the virtual space to the user in the present embodiment will be described with reference to FIGS. 9 and 14. First, the image generation unit 690 of the control unit 340 generates an image corresponding to the gift item posted by the user (S201), and the generated image 1180 of the gift item is a character image in the virtual space 1100 of FIG. Displayed with 1120. Subsequently, the additional information generation unit 655 of the control unit 340 determines whether or not the character 1120 in FIG. 9 touches the displayed gift item 1180 (S202). More specifically, by comparing the coordinates of the hand of the character 1120 in the virtual space with the coordinates corresponding to the area where the gift item 1180 is arranged, the positional relationship between the two is confirmed. For example, the performer moves the controller 210 in FIG. 2 up and down, so that the input detection unit 610 of the control unit 340 detects a change in the position of the controller 210 via the acceleration sensor of the controller 210, and the character control unit. The 620 controls the movement of raising and lowering the arm of the character 1120 in response to the change in the detected position, and the performer substantially matches the position of the palm at the tip of the arm of the character 1120 with the position of the gift item 1180. By operating a predetermined button (for example, the trigger button 240b) of the controller 210 at the same timing, the character control unit 620 can control the character 1120 to grab the gift item 1180.

If it is determined in S202 that the character 1120 has touched the gift item 1180 (“YES” in S202), the additional information generation unit 655 first identifies the user who posted the gift item 1180, and the user's Refer to the user data (S203). If it is determined in S202 that the character 1120 has not touched the gift item 1180 (“No” in S202), the process returns to the original process and monitoring is continued. There are several ways to identify the user who posted the item from the gift item 1180 touched by the character 1120. For example, there is a method of referring to the purchase history information of the gift item and identifying the purchaser. When the user who is the purchaser is identified, the additional information generation unit 655 refers to the user management table, the user gift history management table, and / or the user comment history table stored in the user data storage unit 690. Generate additional information related to that user (S204). For example, when the user ID is "user A", as shown in FIG. 9, the information associated with the user avatar corresponding to the user A is generated. As an example of the content of the information, information such as "user name: Yoshi", "latest history: box (posted)", "consumption pt: 1500", and "comment: will come again" can be generated. The image generation unit 630 generates an image provided to the user by using the generated additional information together with other elements constituting the virtual space (for example, a character or a user avatar) (S205). As shown in FIG. 9, the additional information is displayed in the vicinity of the user avatar provided to the user and displayed in the virtual space (in FIG. 9, inside the balloon 1170). Other aspects are the same as the examples described with reference to FIGS. 8 and 13.

In FIGS. 8 and 9, the information related to the user is provided in the wake of the user taking a predetermined action or the character taking a predetermined action on the gift item posted by the user. An example of displaying is shown, but in addition, the line-of-sight direction of the performer in the virtual space and the viewpoint of the performer detected via the eye tracking sensor, which are detected through various sensors of the HMD 110 worn by the performer. It is also possible to display information related to the user corresponding to the user avatar located in.

<Second Embodiment>
FIG. 15 shows a flowchart as an example of the process of providing the virtual space to the user in the second embodiment. First, the image generation unit 630 of the control unit 340 generates an image of a user avatar of a predetermined user stored in the user data storage unit 690 and performs display processing (S301). Subsequently, the additional information generation unit 655 refers to the user management table stored in the user data storage unit 690, and confirms information such as the number of logins of the user and the last login date, so that the user can enter the service for the first time. Confirm whether to log in or browse the room provided by the performer (the page on which the performer provides the content) (S302). When it is confirmed that the user logs in to the service for the first time or browses the page on which the performer provides the content (“YES” in S302), the additional information generation unit 655 may display the image as shown in FIG. Generation unit Performs processing to add a special effect to the user avatar (S303). If it is confirmed that the user is the second or subsequent login or page viewer in the process of S302 (“No” in S302), the process returns to the original process. In the process of S303, as shown in FIG. 16, display 1270 for notifying that the user has logged in for the first time can be performed. Here, as an example of the special effect, a process of highlighting the user avatar, a process of blinking, a process of enlarging and displaying, and a process of making the user avatar distinguishable from other user avatars can be considered. This special effect may be visible only to the performer.

<Third Embodiment>
FIG. 17 shows a flowchart as an example of the process of providing the virtual space to the user in the third embodiment. First, the image generation unit 630 of the control unit 340 generates an image of a predetermined gift item stored in the item data storage unit 685 and performs display processing (S401). Subsequently, the additional information generation unit 655 instructs the image generation unit 630 to apply special effects to the displayed gift item, and the image generation unit 630 performs a process of adding special processing (S402). Here, as an example of the special effect, a process for attracting the performer's attention to the gift item, such as a process for highlighting the gift item, a process for blinking, and a process for enlarging the display, can be considered. This special effect may be visible only to the performer. In addition, it is possible to give a special effect to the gift item based on the information that the user purchased and posted the gift item for the first time by referring to the history information of the user.

The additional information generation unit 655 confirms whether the performer has made any reaction to the posted gift item (S403). For example, in FIG. 18, it is confirmed whether the performer has read some comment (for example, "Thank you, Mr. A") about the ice cream gift item posted by the user. Here, as a confirmation method, a method of recognizing a corresponding gift item or a word related to a corresponding user name from a voice uttered by a performer using voice recognition technology, or a gift via a character played by the performer. A method of detecting that a predetermined action is taken on the item (for example, the character touches the gift item) can be considered.

When it is detected in the process of S403 that the performer has reacted to the gift item (YES in S403), the additional information generation unit 655 releases the special effect applied to the gift item (S404). As a result, the performer can intuitively understand that the gift item posted by the user has shown some reaction, and can confirm the gift item that has not yet responded. If the process of S403 does not detect that the performer has reacted to the gift item (YES in S403), the special effect is sustained for that gift item. The invention disclosed in this embodiment is similarly applicable to comments posted by users.

<Fourth Embodiment>
FIG. 19 shows an example of the virtual space displayed to the user in the fourth embodiment. In the present embodiment, the item image 1470 related to a predetermined product sold in the real world is displayed in the character display area of the virtual space 1400, and when the predetermined condition is satisfied, the information 1480 related to the product is displayed. Realizes display in virtual space. For example, when the positional relationship between the character 1410 displayed in the character display area and the item image 1470 becomes a predetermined distance, or there is an interaction between the character 1410 and the item image 1470 (for example, the character has an interaction). When (grabbing the item image), information related to the product 1480 can be displayed. The information 1480 related to the product can be, for example, the product name, the seller, the price, the content explaining the product, and the like, and these information can be obtained from an external server via the application program interface. Alternatively, it can be stored in advance in the image generation device 310 as product-related information together with the item image 1470 related to the product.

<Fifth Embodiment>
FIG. 20 shows an example of the virtual space displayed on the performer in the fifth embodiment. The virtual space 1500 shown in FIG. 20 is displayed on the display unit 120 of the HMD 110 worn by the performer, a monitor (not shown), or the like. As shown in FIG. 20A, the virtual space 1500 also includes a service image display area 1550 that displays the virtual space displayed on the user device 401, a character 1510 displayed in the area 1550, and a predetermined object (for example,). , Comment list) It has an operation image display area 1560 for operating the 1520. In the operation image display area 1560, basically, an image corresponding to the image arranged in the service image display area is arranged. For example, the display area performer for the operation image operates the fingers of the character 1510 by operating the controller 210 while referring to the reference image 1540 of the character finger operation. As the reference image 1540 for the character finger operation, the operation hand is located at a position deviated from the approximate coordinates of the position of the performer's hand instruction image 1570 acquired via the position of the controller 210 by a predetermined distance (in the depth direction of the screen). The image 1580 can be displayed. As shown in FIG. 20B, the performer grasps a part (comment 1590) of the object 1530 (comment list) displayed in the operation image display area 1560 while referring to the image 1580 of the operation hand. By performing the operation, the gripping position of the object (comment) of the character 1510 is reflected in the position deviated by the predetermined distance, so that the action of the character gripping the object with the fingertip can be expressed in a more natural form. it can.

In the present embodiment, as shown in FIG. 20A, the comment list 1520 posted by the user is displayed in the vicinity of the character, and the corresponding comment list 1530 is also displayed in the operation image display area 1560. .. As shown in FIG. 20 (b), when the performer operates the reference image 1540 for finger operation and grasps a part of the comment (for example, the comment "Nemui" posted by the user name "Hide"), 1590. Of the comment list displayed near the character, the corresponding comment is grasped by the character. Further, as the performer grabs the comment 1590, additional information about the user "hide" who posted the comment can be displayed. This allows the performer to read the user comment aloud and at the same time ask a special question to the user while referring to information about the user. Here, instead of the additional information, a special effect can be used as an effect based on the content of the comment. For example, in FIG. 20B, when the performer grabs the comment "Congratulations" posted by "Rina", it is possible to use an effect in which fireworks and cakes are displayed on the screen. Further, regarding the display method of the comment list, the comment list 1520 displayed in the service image display area 1550 and the comment list 1530 displayed in the operation image display area 1560 are highlighted so that the viewing user can see the comment list. However, the performer (or the character 1510 played by the performer) may be configured so that the content of the comment list can be understood.

FIG. 21 shows an example of a virtual space displayed to the user in the sixth embodiment. In the present embodiment, the gift item 1620 posted by the user to the character 1610 shown in FIG. 21 (a) is changed to another gift item 1630 as shown in FIG. 21 (b). That is, the user purchases the gift item in advance through the purchasing means 1150 as shown in FIG. 7 in exchange for points, and in FIG. 21A, the present box 1620 posts as a gift item at the designated date and time. It is possible to provide an effect of being delivered (delivered to home) and opened as a gift item 1630 (for example, tea) shown in FIG. 21 (b) according to the action of the character 1120 (contact with the gift item 1620, etc.). it can. At this time, the content of the gift item 1630 and the additional information 1640 indicating the poster can be displayed to the user at the time of opening.

110 Head-mounted display 115 Communication unit 120 Display panel 125 Control unit 140 Sensor 150 Light source 160 Camera 170 Microphone 180 Headphone 190 Input / output unit 210 Controller 220 Left-hand controller 230 Right-hand controller 240a, 240b Trigger button 245 Operation unit 250 Infrared LED
255 Input / output unit 260 Sensor 270 Joystick 280 Menu button 290 Frame 310 Image generator 320 Input / output unit 330 Communication unit 340 Control unit 350 Storage unit 401A, 401B, 401C User device
610 User input detection unit 620 Character control unit 630 Image generation unit 640 Item reception unit 650 Comment reception unit 655 Additional information generation unit 660 Character data storage unit 670 Control program storage unit 680 Streaming data storage unit 685 Item data storage unit 690 User data storage unit Part 1100 Virtual space 1110 Image display part 1120 Character 1130 User information display area 1140 Comment input part 1150 Item selection part 1160 Additional information display area 1170 Additional information display area 1180 Gift item 1200 Virtual space 1270 Additional information display area 1400 Virtual space 1410 Character 1470 Gift Item 1480 Additional Information Display Area 1500 Virtual Space 1510 Character 1520 Object 1530 Object
1540 Character hand operation reference image 1550 Service image display area 1560 Operation image display area 1570 Performer's hand instruction image 1580 Operation hand image 1590 Comment 1600 Virtual space 1610 Character 1620 Gift item 1630 Gift item 1640 Additional information display area

Claims (10)

A method of providing a plurality of users, including a performer user and a viewing user, with a virtual space having predetermined content including a character played by the performer user.
Receive a given action from the user
The predetermined action received is stored as history information for each user, and is stored.
A virtual space containing the predetermined content and information identifying the user is generated.
A method of providing a virtual space, characterized in that information generated based on the history information is displayed. The method of providing the virtual space according to claim 1.
A method of providing a virtual space, characterized in that information related to the action is provided only to a part of the one or a plurality of users. The method of providing the virtual space according to claim 2.
A method of providing a virtual space, characterized in that some of the users are performer users. The method of providing the virtual space according to claim 1.
A method of providing a virtual space, wherein the predetermined action includes at least one of user participation, comment posting, and item purchase by the user. The method of providing the virtual space according to claim 1.
A method of providing a virtual space, wherein the information that identifies the user is a user name. The method of providing the virtual space according to claim 1.
A method of providing a virtual space, wherein the information identifying the user is an avatar image of the user. The method of providing the virtual space according to claim 1.
The history information is a method of providing a virtual space, which is information obtained by accumulating a plurality of actions for each user for a predetermined period of time. The method of providing the virtual space according to claim 1.
A method for providing a virtual space, characterized in that information generated based on the history information is displayed when the action of the character and the predetermined action act in both directions. The method of providing the virtual space according to claim 1.
The predetermined action is an item purchase by a user, and a method of providing a virtual space, characterized in that a special effect is given to an item purchased by the user based on the history information.
A method of providing a virtual space based on the history information. The method of providing the virtual space according to claim 1.
A method for providing a virtual space, which comprises accessing an application used by the user and displaying information related to the user together with information generated based on the history information.

Images