JP2017041229A - Method and program for controlling head-mounted display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide content provision means capable of appealing to a user in a timely fashion in response to the user's interest.SOLUTION: A method causes a computer to execute the steps of: generating a virtual space in which a user is immersed and target objects in the virtual space; defining a base position in the virtual space and target object positions where the target objects are arranged; defining a base direction, that is, a direction of the user's line of sight with respect to the base position; determining a gaze direction, that is, a direction of the user's line of sight with respect to the target object; determining a gaze time for which the user has gazed at the target object; determining an adjusted gaze time on the basis of the gaze time, the gaze direction, and the base direction; and presenting predetermined content to the user on the basis of the adjusted gaze time.SELECTED DRAWING: Figure 10

Description

  The present invention relates to a method and a program for causing a computer to provide a virtual space in which a user immerses using a head-mounted display (HMD).

  In Patent Document 1, a land price for each user is set in each place in the virtual space, and an advertisement charge for displaying an advertisement in that place is set by averaging the land prices between the users for the place. A technique is disclosed. The land price is set based on the movement operation history of each user. Patent Document 2 discloses a technique for setting the priority of a display item based on a user's action situation.

Japanese Patent No. 3732168 JP 2014-71811 A

  In the techniques described in Patent Documents 1 and 2, it is necessary to collect a large amount of user information in order to provide appropriate content such as advertisements to the user. Therefore, there is room for improvement in approaching the user's interest in a timely manner.

According to the present invention, there is provided a method for providing a virtual space in which a user is immersed, the step of generating a virtual space in which the user is immersed, and a target object in the virtual space, and a reference in the virtual space. Defining a position and a target object position where the target object is arranged; defining a reference direction that is the user's line-of-sight direction with respect to the reference position; and the user's line-of-sight direction with respect to the target object Identifying a gaze direction, identifying a gaze time when the user gazes the target object, identifying an adjusted gaze time based on the gaze time, the gaze direction, and the reference direction And a predetermined container for the user based on the adjusted gaze time. How to execute the steps of providing a tool, a, is obtained.

  ADVANTAGE OF THE INVENTION According to this invention, the content provision means which can approach a user's interest timely can be provided.

1 is a diagram illustrating an HMD system according to an embodiment of the present invention. An orthogonal coordinate system in a three-dimensional space defined around the head of a user wearing an HMD is shown. It is an XYZ space figure which shows an example of virtual space. It is a figure which shows an example of the method of determining a gaze direction. It is a block diagram which shows the function of the control circuit part for implement | achieving the function of a HMD system. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is the YZ plane view which looked at the visual field area from the X direction. It is the XZ plane figure which looked at the visual field area from the Y direction. It is a figure for demonstrating the relationship between a launcher space, 1st virtual space, and 2nd virtual space. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a figure for demonstrating the 1st virtual space and the 2nd virtual space. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a YZ plane view explaining the first deviation amount with respect to the reference direction of the gaze direction. It is a XZ plane figure explaining the 2nd shift amount with respect to the reference direction of a gaze direction. It is a figure which shows the state before being watched of the advertisement in a visual field area | region. It is a figure which shows the state after being watched of the advertisement in a visual field area | region. It is a figure which shows a user action management table. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a figure for demonstrating launcher space. It is a figure which shows a priority management table. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a figure for demonstrating launcher space.

[Description of Embodiment of the Present Invention]
First, the contents of the embodiment of the present invention will be listed and described. A method and a program for controlling a head mounted display system according to an embodiment of the present invention have the following configuration.
(Item 1)
A method for providing an immersive virtual space for a computer,
Generating a virtual space in which the user is immersed, and a target object in the virtual space;
Defining a reference position in the virtual space and a target object position where the target object is arranged;
Defining a reference direction that is the line-of-sight direction of the user with respect to the reference position;
Identifying a gaze direction that is a gaze direction of the user with respect to the target object;
Identifying a gaze time when the user gazes at the target object;
Identifying an adjusted gaze time based on the gaze time, the gaze direction, and the reference direction;
Providing the user with predetermined content based on the adjusted gaze time.
According to the method of this item, the content related to the advertisement is provided to the user based on the time when the user gazes at the target object such as the advertisement. At that time, the gaze time is adjusted according to the relationship between the gaze direction of the user and the reference direction set in the virtual space. Accordingly, it is possible to provide a content providing unit that can easily determine an object that the user has shown interest in and can approach in a timely manner.
(Item 2)
The method according to item 1, wherein the adjusted gaze time is calculated by adjusting the gaze time based on a shift amount of the gaze direction with respect to the reference direction.
According to the method of this item, when the user looks away from the reference direction and looks closely at the target object, it is estimated that the user is highly interested in the target. Accordingly, it is possible to provide a content providing unit that can approach the user's interest in a timely manner.
(Item 3)
The shift amount is a first shift amount that is a shift amount of the gaze direction with respect to the reference direction in a vertical plane, and a second shift amount that is a shift amount of the gaze direction with respect to the reference direction in a horizontal plane perpendicular to the vertical plane. Including displacement
The method according to Item 2, wherein the adjusted gaze time is calculated by assigning a predetermined weight to the first deviation amount and the second deviation amount.
According to the method of this item, it is possible to place an emphasis on actions that are more difficult for the user to do spontaneously.
(Item 4)
Item 3. The method according to Item 3, wherein the weighting is largely applied to the first deviation amount.
According to the method of this item, content that can approach the user's interest in a timely manner by placing emphasis on the user's behavior that makes it difficult for the user to voluntarily move the line of sight in the vertical plane. Means may be provided.
(Item 5)
Defining a field of view of the user based on the line of sight of the user;
Identifying a first object and a second object that are a plurality of the target objects included in the field-of-view area;
Changing the display of the second object to be associated with the first object if the adjusted gaze time of the user for the first object exceeds a predetermined threshold; The method according to any of items 1 to 4.
According to the method of this item, when the user gazes at a predetermined target, other objects in the user's field of view are changed so as to relate to the gazed object. Accordingly, it is possible to provide a content providing unit that can approach the user's interest in a timely manner.
(Item 6)
Providing the virtual space to the user by outputting to a head mounted display;
Based on the adjusted gaze time, set the priority of the content provided to the user,
Based on the priority, identify extended content related to the content,
The method according to any one of items 1 to 5, wherein when the head mounted display is not attached to the user, a download request for downloading the extended content is output.
According to the method of this item, the extended content (for example, a trial version of a game) related to the advertisement can be downloaded when the head mounted display is not used. As a result, the extended content can be provided without giving the user who is using the head-mounted display a sense of incongruity such as deterioration of the communication status.
(Item 7)
And further comprising the step of generating a launcher space,
The method of claim 6, wherein the launcher space is associated with the virtual space and the extended content, and the content is displayed according to the priority.
According to the method of this item, by displaying objects such as advertisements reflecting the user's interest in a certain virtual space as a list in the launcher space, it is possible to prevent a loss of immersion in the user's virtual space. .
(Item 8)
Generating a second virtual space in which the user is immersed;
Generating a second target object related to the first target object arranged in the first virtual space in the second virtual space;
Identifying a total adjusted gaze time obtained by adding the adjusted gaze time for the first target object and the adjusted gaze time for the second target object by the user;
The method according to any one of items 1 to 7, further causing a step of providing predetermined content to the user based on the gaze time.
According to the method of this item, a user can propose a more appropriate advertisement by adding up the actions related to the user's line-of-sight history in a plurality of contents.
(Item 9)
A program for causing a computer to execute any one of items 1 to 8.

[Details of the embodiment of the present invention]
A specific example of a method and a program according to an embodiment of the present invention will be described below with reference to the drawings. In addition, this invention is not limited to these illustrations, is shown by the claim, and intends that all the changes within the meaning and range equivalent to the claim are included. In the following description, the same reference numerals are given to the same elements in the description of the drawings, and redundant descriptions are omitted.

  FIG. 1 shows an HMD system 100 including an HMD 110 according to the present embodiment. The HMD system 100 includes an HMD 110 that is worn on the user's head, a control circuit unit 120, a motion sensor 130, and a gaze sensor 140.

The HMD 110 includes a display 112 that is a non-transmissive display device, a sensor unit 114, and a gaze sensor 140. The control circuit unit 120 displays a right-eye image and a left-eye image on the display 112, thereby providing a three-dimensional image using parallax between both eyes as a virtual space. By arranging the display 112 in front of the user's eyes, the user can be immersed in the virtual space. The virtual space includes a background, various objects that can be operated by the user, menu images, and the like.

  The display 112 may include a right-eye sub-display that provides a right-eye image and a left-eye sub-display that provides a left-eye image. Moreover, as long as the image for right eyes and the image for left eyes can be provided, you may be comprised by one display apparatus. For example, the right-eye image and the left-eye image can be provided independently by switching the shutter so that the display image can be recognized by only one eye at high speed.

  The control circuit unit 120 is a computer connected to the HMD 110 and presents a virtual space on the display 112 by executing a stored predetermined application. In addition, the control circuit unit 120 performs processing so as to operate various objects displayed in the virtual space and to display and control various menu images and the like. The control circuit unit 120 stores a program for controlling execution of such operations. The control circuit unit 120 may not be mounted on the HMD 110 and may be configured as another hardware (for example, a known personal computer or a server computer via a network). In addition, the control circuit unit 120 may implement only a part of functions in the HMD 110 and implement the remaining functions in different hardware.

  The motion sensor 130 detects information related to the position and inclination of the HMD 110. The motion sensor 130 includes a sensor unit 114 and a detection unit 132. The sensor unit 114 may include a plurality of light sources. The light source is, for example, an LED that emits infrared rays. The detection unit 132 is, for example, an infrared sensor, and detects information about the position and angle of the HMD 110 in the real space according to the user's movement by detecting infrared rays from the light source as detection points of the HMD 110 over time. Then, it is possible to determine time-dependent changes in the position and angle of the HMD 110 based on changes over time in the information detected by the detection unit 132, and to detect information related to the movement of the HMD 110.

  Information regarding the position and tilt acquired by the motion sensor 130 will be described with reference to FIG. The XYZ axes are defined around the head of the user wearing the HMD 110. The vertical direction in which the user stands upright is the Y axis, the front-rear direction orthogonal to the Y axis and connecting the center of the display 112 and the user is the Z axis, and the horizontal direction orthogonal to the Y axis and the Z axis is the X axis. Thereby, the temporal change of the position in the XYZ axis direction of the user is acquired. Further, the inclination angle θx (so-called pitch angle), θy (so-called yaw angle), and θz (so-called roll angle) of the HMD 110 around each axis are acquired.

  The motion sensor 130 may be configured by only one of the sensor unit 114 fixed near the display 112 and the detection unit 132. The sensor unit 114 may be a geomagnetic sensor, an acceleration sensor, or an angular velocity (gyro) sensor, and the position and inclination of the HMD 110 (particularly, the display 112) mounted on the user's head using at least one of them. Is detected. Thereby, the information regarding the motion of the HMD 110 can be detected. For example, the angular velocity sensor can detect the angular velocities around the three axes of the HMD 110 over time according to the movement of the HMD 110 and determine temporal changes in the angles around the respective axes. In this case, the detection unit 132 is not necessary. The detection unit 132 may include an optical camera. In this case, information related to the movement of the HMD 110 can be detected based on the image information, and the sensor unit 114 is unnecessary.

The function of detecting information related to the position and tilt of the HMD 110 using the motion sensor 130 is referred to as position tracking. The relationship between the position tracking by the motion sensor 130 and the virtual camera 1 arranged in the virtual space 2 will be described with reference to FIG. In order to explain the positional relationship between the virtual camera 1 and the motion sensor 130, the position of the motion sensor 130 is the position of the detection unit 132 when the detection unit 132 is provided, and is the sensor when the detection unit 132 is not provided. The position of the portion 114 is assumed. The virtual camera 1 is disposed inside the virtual space 2, and the motion sensor 130 is virtually disposed outside the virtual space 2 (real space).

The virtual space 2 is formed in a celestial sphere having a plurality of substantially square or substantially rectangular meshes. Each mesh is associated with the spatial information of the virtual space 2, and the visual field region 23 is defined based on this spatial information. In the present embodiment, it is preferable to adjust so that the center point 21 of the celestial sphere is always arranged on the line connecting the virtual camera 1 and the sensor 130 in the XZ plane. For example, when the user wearing the HMD moves and the position of the virtual camera 1 moves in the X direction, the center 21 is positioned on the line segment between the virtual camera 1 and the motion sensor 130. The area is changed. In this case, the position of the virtual camera 1 in the virtual space 2 is fixed, and only the tilt changes. On the other hand, if the position of the virtual camera 1 is moved in conjunction with the movement of the motion sensor 130 in the XYZ directions, the position of the virtual camera 1 in the virtual space 2 is variably set.

  The gaze sensor 140 has an eye tracking function that detects the direction in which the user's right eye and left eye gaze are directed. The gaze sensor 140 preferably includes a right eye sensor and a left eye sensor, and detects the gaze direction in which the user gazes by detecting the direction in which the gaze of the right eye and the left eye is directed respectively. A known sensor having an eye tracking function can be adopted as the gaze sensor 140. For example, the right eye and the left eye are irradiated with infrared light, and the reflected light from the cornea and the iris is acquired, whereby the rotation angle of the eyeball is adjusted. It may be asking.

As shown in FIG. 4, the gaze sensor 140 detects the gaze direction of the right eye and the left eye of the user U. When the user U is looking near, the lines of sight R1 and L1 are detected, and the gazing point N1 that is the intersection of the two is specified. Further, when the user is looking far away, the lines of sight R2 and L2 whose angles to the Z direction are smaller than the lines of sight R1 and L1 are specified. When the gazing point N1 is specified, the line-of-sight direction N0 of the user U is specified. The line-of-sight direction N0 is a direction in which the user U is actually directed by his eyes. The line-of-sight direction N0 is defined as a direction in which a straight line passes through the center of the right eye R and the left eye L of the user U and the center point N1, for example.

  The HMD system 100 may include headphones including a microphone as any element. Thereby, the user can give a voice instruction to a predetermined object in the virtual space. Further, in order to receive a broadcast of a television program on a virtual television in a virtual space, the HMD system 100 may include a television receiver in any element. Moreover, a communication function or the like for displaying an electronic mail or the like acquired by the user may be included. Further, an external controller for inputting various user commands may be included.

  FIG. 5 is a block diagram illustrating functions of the control circuit unit 120 for realizing display processing of the virtual space 2 in the HMD system 100 and various menu displays and object operations displayed in the virtual space 2. . The control circuit unit 120 controls an output image to the display 112 based mainly on inputs from the motion sensor 130 and the gaze sensor 140.

The control circuit unit 120 includes a display control unit 200, an object control unit 300, and a communication control unit. The display control unit 200 includes a virtual space image generation unit 210, an HMD motion detection unit 220, a line-of-sight detection unit 230, a visual field direction identification unit 240, a visual field region determination unit 250, a visual field image generation unit 260, and spatial information. Storage unit 270. The object control unit 300 includes an advertisement specifying unit 310, a gaze time measurement unit 320, an adjustment unit 330, a determination unit 340, a drawing unit 350, an object information storage unit 360, a virtual camera information storage unit 370, a user An action storage unit.

The motion sensor 130 and the gaze sensor 140 are communicably connected to the display control unit 200 and the object control unit 300, and can be connected via a wired or wireless communication interface. The display control unit 200, the object control unit 300, and the communication control unit 400 are communicably connected to the HMD 110 (display 112), and can be connected via a wired or wireless communication interface. The communication control unit 400 can communicate with the external server 160 via the network 150. The spatial information storage unit 270, the object information storage unit 360, the virtual camera information storage unit 370, and the user behavior storage unit 380 are various types for providing output information corresponding to inputs from the motion sensor 130 and the gaze sensor 140 to the display 112. Contains data.

  A processing flow of the HMD system 100 for providing the virtual space 2 will be described with reference to FIGS. The virtual space 2 can be provided by the interaction of the HMD 110 (the gaze sensor 140 and the motion sensor 130) and the control circuit unit 120.

  First, the control circuit unit 120 (virtual space image generation unit 210) generates the celestial spherical virtual space image 22 constituting the virtual space 2 in which the user is immersed (S120-1). When an operation such as movement or tilt is input from the user to the HMD 110 (S110-1), the position and tilt of the HMD 110 are detected by the motion sensor 130 (S130-1). Detection information of the motion sensor 130 is transmitted to the control circuit unit 120, and position information and tilt information of the HMD 110 are received by the HMD motion detection unit 220. Thereby, the visual field direction based on the positional information and inclination information of the HMD 110 is determined (S120-2).

  When the gaze sensor 140 detects the movement of the eyeballs of the left and right eyes of the user (S140-1), the information is transmitted to the control circuit unit 120. The line-of-sight detection unit 230 identifies the direction in which the line of sight of the right eye and the left eye is directed, and the line-of-sight direction N0 is identified (S120-3). The reference line-of-sight specifying unit 240 specifies the field-of-view direction specified by the inclination of the HMD 110 or the user's line-of-sight direction N0 as the reference line of sight 5 (S120-4).

The visual field area determination unit 250 determines the visual field area 23 of the virtual camera 1 in the virtual space 2 (S120-5). As shown in FIG. 3, the visual field region 23 is a part of the virtual space image 22 that constitutes the user's visual field. The visual field area 23 is determined based on the reference line of sight 5, and the reference line of sight 5 is determined based on the position and tilt of the virtual camera 1. 7A is a YZ plane view of the visual field area 23 as viewed from the X direction, and FIG. 7B is an XZ plane view of the visual field area 23 as viewed from the Y direction.

The visual field region 23 is a first region 24 (see FIG. 7A) that is a range defined by the reference line of sight 5 and the YZ cross section of the virtual space image 22, and a range defined by the reference line of sight 5 and the XZ cross section of the virtual space image 22. And a second region 25 (see FIG. 7B). The first region 24 is set as a range including the polar angle α with the reference line of sight 5 as the center. The second region 25 is set as a range including the azimuth angle β with the reference line of sight 5 as the center.

The view image generation unit 260 generates the view image 26 based on the view region 23 (S120-6). The field-of-view image includes two two-dimensional images for the left eye and the right eye, and these are superimposed on the display 112, whereby the virtual space 2 as a three-dimensional image is provided to the user. The HMD 110 receives information about the view image 26 from the control circuit unit 120 and displays the view image 26 on the display 112 (S110-2).

  A specific example of a virtual space provided to the user and a processing flow of the HMD system 100 for providing the virtual space to the user will be described with reference to FIG. 5 and FIG. As shown in FIGS. 8 and 9, the virtual space in the present embodiment includes a launcher space LAU, and a first virtual space VR1 and a second virtual space VR2 associated with the launcher space LAU. A launcher space LAU is first provided to a user who activates the HMD system 100 and wears the HMD 110. The launcher space LAU is generated by the virtual space image generation unit 210 and the visual field image generation unit 260 and is output to the HMD 110 (S120-7).

In the launcher space LAU, an application AP1 for providing the first virtual space VR1 and an application AP2 for providing the second virtual space VR2 are displayed. The user can move to the selected virtual space by inputting a predetermined operation (such as sending a line of sight for a certain period of time or operating a controller) to the applications AP1 and AP2 (S110-3). In the launcher space LAU, an advertisement space AD for displaying advertisements to be described later and trial version spaces TR1 and TR2 for displaying downloaded trial version contents are also displayed.

  An example of the first virtual space VR1 is shown in FIG. When the application AP1 is selected, the virtual space image generation unit 210 generates the virtual space image 22 constituting the first virtual space VR1. In addition, the drawing unit 350 of the object control unit 300 generates various objects with reference to the object information storage unit 360, and draws and arranges them in the first virtual space VR1 (S120-8). The first virtual space VR1 is a content providing entertainment such as a live stage, for example, and the stage S, the character C providing performance on the stage S, and various advertisements OB1 and OB2 are arranged in the first virtual space VR1. .

  The object control unit 300 specifies a reference position BP in the virtual space and a position (target object position) where the advertisements OB1 and OB2 as target objects to be measured are measured as described later. (S120-9). In a state immediately after the first virtual space VR1 is provided to the user, the user is located at the center of the celestial sphere of the first virtual space VR1, and the reference visual line 5 that is the user's visual line direction is directed to the reference position BP ( Reference direction BD described later). In the present embodiment, the view image 26 generated based on the reference direction BD includes a stage S, a character C, and a plurality of first advertisements (first target objects). Hereinafter, the advertisement included in the view image 26 may be referred to as a first advertisement (first object) OB1, and the advertisement not included in the view image 26 may be referred to as a second advertisement (second object) OB2.

  Note that the second virtual space VR2 is also the same as that in FIG. Schematically, when the application AP2 is selected, the virtual space image generation unit 210 generates the virtual space image 22 constituting the second virtual space VR2. In addition, the drawing unit 350 of the object control unit 300 generates various objects with reference to the object information storage unit 360, and draws and arranges them in the second virtual space VR2 (S120-8). Also in the second virtual space VR2, the stage S, the character C, the plurality of second advertisements (second target objects), the first advertisement (first object) OB1 included in the view image 26, and the view image 26 are not included. A second advertisement (second object) OB2 is arranged.

  The user can act freely in the first virtual space VR1 by inputting various movements such as a change in position and inclination to the HMD 110 (S110-4). When the motion sensor 130 detects the movement of the user, the HMD motion detection unit 220 receives this and specifies the position of the user in the virtual space (S120-10). When the motion sensor 130 or the gaze sensor 140 detects a change in the user's reference line of sight 5, the reference line-of-sight specifying unit 240 receives this and specifies the line-of-sight direction in the user's virtual space (S120-11). The object control unit 300 specifies the user's dyeing direction based on the position of the user in the virtual space and the user's reference line of sight 5 from the reference line-of-sight specifying unit 240. Then, the advertisement specifying unit 310 determines whether the line-of-sight direction is directed to any one of the advertisements OB1 and OB2 based on the information regarding the arrangement positions of the various advertisements OB1 and OB2 (S120-12). The advertisement specifying unit 310 specifies the user's line-of-sight direction directed to one of the advertisements OB1 and OB2 as the gaze direction GD. Then, the gaze time measuring unit 320 starts measuring the gaze time, which is the time during which the reference line of sight is sent to the advertisement (S120-13). When the user's gaze direction is directed to a certain range including a predetermined advertisement (a circle, an ellipse, a polygon, etc. including the advertisement), the gaze time measuring unit 320 displays the advertisement. The gaze time is measured assuming that the user is gazing.

  Of the user behavior history as described above, the behavior in which the user gazes at a predetermined advertisement is stored in the user behavior storage unit 370. The user behavior storage unit 370 includes, for example, a user behavior management table T1 illustrated in FIG. The user behavior management table T1 stores information on when and in what application (virtual space) the user gazes at which object and from which position in the virtual space, and the time when the user gazes at the object. .

  Here, the gaze time when the user gazes at the predetermined advertisement may be stored as the adjusted gaze time based on the processing flow shown in FIG. FIG. 12A is a diagram illustrating a reference direction BD-V and a gaze direction GD-V in a vertical plane (YZ plane) viewed from the user. FIG. 12B is a diagram illustrating a reference direction BD-H and a gaze direction GD-H in a horizontal plane (XZ plane) viewed from the user. First, as described above, the reference direction BD is specified based on the reference position BP in the first virtual space VR1 and the position of the user (the position of the virtual camera 1) (S120-14). At this time, the reference direction BD specifies the reference direction BD-V in the vertical plane (YZ plane) and the reference direction BD-H in the horizontal plane (XZ plane). In the present embodiment, since the character C as the leading role of the content is arranged on the stage S, the user's reference line of sight BD-V is set to face a little information. Further, the gaze direction GD is specified based on the advertisement position OB1 and the user position in the first virtual space VR1 (S120-15). At this time, the gaze direction GD specifies the gaze direction GD-V in the vertical plane (YZ plane) and the gaze direction GD-H in the horizontal plane (XZ plane).

  Next, the object control unit 300 obtains a deviation amount between the gaze direction GD and the reference direction BD (S120-16). In the present embodiment, first shift amounts DV1 and (S120-17), which are shift amounts (angle differences) of the gaze direction GD-V with respect to the reference direction BD-V in the vertical plane, and the gaze direction GD-H in the horizontal plane. The second displacement amount DV2 that is the displacement amount (angle difference) with respect to the reference direction BD-H is obtained (S120-18).

  Then, the adjustment unit 330 calculates the adjusted gaze time by adding the first deviation amount DV1 (angle: radians) and the second deviation amount DV2 (angle: radians) to the gaze time (S120-20).

  β1 is a coefficient for weighting the first deviation amount DV1. β2 is a coefficient for weighting the second deviation amount DV2. The adjustment unit 330 sets β1 and β2 to predetermined values in advance (S120-19), thereby placing an emphasis on the action history that the user has watched an advertisement that has taken actions that are less likely to be spontaneous. Can be adjusted.

  In the present embodiment, it is preferable that the weighting coefficient β1 for the first deviation amount DV1 is larger than the weighting coefficient β2 for the second deviation amount DV2. It is estimated that the user feels more stress on the head movement in the vertical direction than the head movement in the horizontal direction. In this way, it is possible to approach the user's interests in a timely manner by placing importance on the behavior of the user moving his / her line of sight in the vertical plane, which is a behavior that is more difficult for the user to voluntarily.

DV1 and DV2 are values of −π or more and π or less. Further, DV1 and DV2 are raised to the power (square or higher) and applied to the calculation of the adjusted gaze time. As a result, the adjusted gaze time can be calculated with a greater emphasis on the action of the user gazing at the advertisement until the user turns away from the reference direction. The adjusted gaze time is stored in the user behavior management table T1 of the user behavior storage unit 370.

  The determination unit 340 determines whether or not the adjusted gaze time has exceeded a predetermined threshold (S120-21). In this embodiment, when the adjusted gaze time is 5 seconds or more, the user interprets that the user has shown interest in the advertisement, and the threshold is set to 5 seconds. If the adjusted gaze time exceeds a predetermined threshold, it is accepted that the user gazes at the advertisement (S120-22).

  FIG. 13A and FIG. 13B show an example of processing when the user is accepted as having watched the advertisement. FIG. 13A shows a state before being accepted. FIG. 13B shows the state after being accepted. In a state before being accepted, three types of advertisements A, B, and C are displayed in the advertisement OB1 in the view field area 23. When the user's gaze on the advertisement OB1 on which the advertisement A is displayed is received, the drawing unit 350 changes the display of the advertisement OB1 on which the advertisement B is displayed to the advertisement A-1 related to the advertisement A and the advertisement C The display of the advertisement OB1 on which is displayed is changed to the advertisement A-2 related to the advertisement A. Thereby, when a user gazes at a predetermined object, other objects in the user's field of view are changed so as to relate to the gaze object. This makes it possible to approach the user's interest in a timely manner.

  As described above, when the adjusted gaze time for the predetermined advertisement is specified in the first virtual space VR1 (S120-23), an instruction to end the first virtual space VR1 is input from the user to the HMD 110 ( S110-5) is shown in FIG. In this case, the determination unit 340 refers to the user behavior storage unit 370 and specifies the priority of each advertisement (S120-24). The priority of the advertisement is specified by the total value of the adjusted gaze time. As shown in FIG. 14, the priority of the advertisement at the time when the first virtual space VR1 is finished is specified as A-2> A> A-1. Is done.

  The determination unit 340 transmits information regarding the priority of the advertisement to the view image generation unit 260. The field-of-view image generation unit 260 generates a launcher space LAU as shown in FIG. 16 based on the information and outputs it to the HMD 110 (S120-25). In the advertisement space AD of the launcher space LAU, a plurality of advertisements are arranged according to priority. The advertisement A-2 determined to have the highest priority may be displayed larger or displayed with some emphasis.

  Subsequently, when the user selects the application AP2 that activates the second virtual space VR2, the second virtual space VR2 is provided to the HMD 110 (S110-6). Also in the second virtual space VR2, the user's gaze time (adjusted gaze time) for the advertisement is specified by the same processing flow as in the first virtual space VR1 (S120-26).

  When the user finishes using the HMD 110 (S110-7), the determination unit 340 specifies the total adjusted gaze time as the total gaze time (S120-27). The total adjusted gaze time is managed in the priority management table T2 shown in FIG. The priority management table T2 is stored in the user behavior storage unit 370.

  The priority management table T2 calculates the total value of adjusted gaze times for each advertisement based on the user behavior management table T1. In the present embodiment, as shown in the user behavior management table T1, the user is watching the advertisement A-2 in the first virtual space VR1 and the second virtual space VR2. At that time, since the user is gazing at the advertisement A-2 from different user positions, the user's interest during wearing of the HMD is obtained more accurately by adding the adjusted gazing time taking into account the user's line-of-sight direction. be able to. Therefore, in the priority management table T2, the total value of the adjusted gaze time is managed.

  The determination unit 340 sets the priority of each advertisement based on the total adjusted gaze time. That is, the higher priority is given in the order of the longer the total adjusted gaze time. At this time, the advertisement A-1 that does not exceed the threshold value of 5 seconds is excluded from the priority setting targets.

In addition, the determination unit 340 determines whether or not there is a trial version content as extended content related to each advertisement for which priority is set. In the present embodiment, since the two trial version spaces TR1 and TR2 are set in the launcher space LAU, the top two advertisements among the advertisements having the trial version contents are specified (ads B and B-1). When the trial version content is specified, the determination unit 340 instructs the communication control unit 400 to output a download request for downloading the trial version content. When receiving the instruction and information notifying that the user has finished using the HMD from the HMD 110, the communication control unit 410 instructs the communication processing unit 420 to output a download request. The communication processing unit 420 connects to the external server 160 via the network 150 and transmits a trial content download request. The external server 160 transmits the trial version content to the communication processing unit 420 via the network 150 based on the download request.

As described above, in this embodiment, the extended content related to the advertisement (for example, a trial version of the game) can be downloaded when the user is not using the HMD 110. As a result, the extended content can be provided without giving the user who is using the HMD 110 a sense of incongruity such as deterioration of the communication status.

  When the user resumes using the HMD 110, the control circuit unit 120 specifies the downloaded trial version content (S120-30). In the present embodiment, it is determined whether or not the download of the trial version content regarding the advertisement B and the advertisement B-1 is completed, and when the download is completed, the trial version content is specified. Then, similarly to the above, the priority management table T2 is referred to, and the priority of each advertisement set when the user uses the previous HMD 110 is specified (S120-31). Thereafter, the field-of-view image generation unit 260 generates the launcher space LAU shown in FIG. 18 and outputs it to the HMD 110 (S120-32). In the advertisement space AD of the launcher space LAU, advertisements up to priority 5 are arranged in order from the top. Further, the advertisement B having the highest priority is greatly highlighted. In addition, the trial version contents related to the advertisement B and the advertisement B-1 are arranged in the trial version spaces TR1 and TR2.

  The user can select any one of the first virtual space VR1, the second trial version content VR2, and the trial version content arranged in the trial version spaces TR1 and TR2 and enjoy the corresponding virtual space. Moreover, it can also instruct | indicate so that the information regarding it may be transmitted to other information terminals (a personal computer, a smart phone, etc.) by selecting an advertisement.

  Note that once the HMD 110 is not attached, it is preferable not to add up to the time when the advertisement was watched when the HMD 110 was attached last time, even if the advertisement is watched in a virtual space that has been immersed thereafter. That is, it is preferable that the process of adding the time when the advertisement is watched is performed only within one HMD 110 wearing time. Accordingly, it is possible to provide a content providing means capable of approaching the user's interest in a timely manner without averaging the user's interest over a certain long period of time.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. Those skilled in the art will appreciate that various modifications of the embodiments can be made without departing from the spirit and scope of the invention as set forth in the appended claims.

  For example, in the above-described embodiment, the priority of the content is determined based on the adjusted gaze time. However, the priority may be determined based on the gaze time before adjustment.

In the above-described embodiment, the advertisements A, A-1, and A-2, which are contents related to each other, are treated as not relevant in calculating the gaze time or the total adjusted gaze time. It may be handled as related. For example, the gaze time or adjusted gaze time of the advertisements A, A-1, and A-2 may be summed, and the priority may be determined by integrating the advertisements A, A-1, and A-2. In this case, when it is determined that the priority of the advertisements A, A-1, and A-2 is high, all the advertisements related to the advertisements A, A-1, and A-2 and the extended content are displayed as launchers. It is good also as displaying on space LAU.

  DESCRIPTION OF SYMBOLS 100 ... Head mounted display (HMD) system, 110 ... HMD, 112 ... Display, 114 ... Sensor part, 120 ... Control circuit part, 130 ... Motion sensor, 132 ... Detection part, 140 ... Gaze sensor, 200 ... Display control part, DESCRIPTION OF SYMBOLS 300 ... Object control part, 400 ... Communication control part, 1 ... Virtual camera, 2 ... Virtual space, 5 ... Reference line of sight, 22 ... Virtual space image, 23 ... View field area, 26 ... View field image, BP ... Reference point, BD ... reference direction, GD ... gaze direction.

Claims (9)

A method for providing an immersive virtual space for a computer,
Generating a virtual space in which the user is immersed, and a target object in the virtual space;
Defining a reference position in the virtual space and a target object position where the target object is arranged;
Defining a reference direction that is the line-of-sight direction of the user with respect to the reference position;
Identifying a gaze direction that is a gaze direction of the user with respect to the target object;
Identifying a gaze time when the user gazes at the target object;
Identifying an adjusted gaze time based on the gaze time, the gaze direction, and the reference direction;
Providing predetermined content to the user based on the adjusted gaze time;
How to run. The method of claim 1, wherein the adjusted gaze time is calculated by adjusting the gaze time based on a shift amount of the gaze direction with respect to the reference direction. The shift amount is a first shift amount that is a shift amount of the gaze direction with respect to the reference direction in a vertical plane, and a second shift amount that is a shift amount of the gaze direction with respect to the reference direction in a horizontal plane perpendicular to the vertical plane. Including displacement
The method of claim 2, wherein the adjusted gaze time is calculated by assigning a predetermined weight to the first deviation amount and the second deviation amount. The method according to claim 3, wherein the weight is largely applied to the first deviation amount. Defining a field of view of the user based on the line of sight of the user;
Identifying a first object and a second object that are a plurality of the target objects included in the field-of-view area;
Changing the display of the second object to be associated with the first object when the adjusted gaze time of the user for the first object exceeds a predetermined threshold;
The method according to claim 1, further comprising: Providing the virtual space to the user by outputting to a head mounted display;
Based on the adjusted gaze time, set the priority of the content provided to the user,
Based on the priority, identify extended content related to the content,
When the head mounted display is not attached to the user, a download request for downloading the extended content is output.
The method according to claim 1. And further comprising the step of generating a launcher space,
The method of claim 6, wherein the launcher space is associated with the virtual space and the extended content, and the content is displayed according to the priority. Generating a second virtual space in which the user is immersed;
Generating a second target object related to the first target object arranged in the first virtual space in the second virtual space;
Identifying a total adjusted gaze time obtained by adding the adjusted gaze time for the first target object and the adjusted gaze time for the second target object by the user;
The method according to claim 1, further comprising: providing predetermined content to the user based on the gaze time. The program which makes the said computer perform the method in any one of Claims 1-8.