JP2017041019A - Head-mounted display system control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a user with various types of content without sacrificing a user's sense of immersion in a virtual space.SOLUTION: A program is configured to make a computer perform the steps of; generating a virtual space for a user to be immersed and a plurality of target objects in the virtual space, and outputting a result to a head-mounted display; detecting a history of actions taken by the user on the target objects in the virtual space; identifying priorities of the plurality of objects based on the history; generating delivery content representing information related to the target objects to be delivered based on the priorities; and outputting the delivery content to a device other than the head-mounted display.SELECTED DRAWING: Figure 20

Description

  The present invention relates to a program that allows a computer to provide a virtual space in which a user immerses using a head-mounted display (HMD).

  In Patent Document 1, a land price for each user is set in each place in the virtual space, and an advertisement charge for displaying an advertisement in that place is set by averaging the land prices between the users for the place. A technique is disclosed. The land price is set based on the movement operation history of each user. Patent Document 2 discloses a technique for setting the priority of a display item based on a user's action situation.

Japanese Patent No. 3732168 JP 2014-71811 A

  In the techniques described in Patent Literatures 1 and 2, since various advertisements are arranged in a virtual space in which the user is immersed, there is a risk of impairing the user's immersion in the virtual space.

According to the present invention, a virtual space in which a user is immersed in a computer, a step of generating a plurality of target objects in the virtual space, and outputting them to a head mounted display; A step of detecting a history of actions taken in the virtual space, a step of identifying priorities of the plurality of target objects based on the history, and information on the target objects are arranged based on the priorities And a program for executing the step of outputting the distribution content to a device different from the head mounted display.

  According to the present invention, various contents can be provided to the user without impairing the user's immersion in the virtual space.

1 is a diagram illustrating an HMD system according to an embodiment of the present invention. An orthogonal coordinate system in a three-dimensional space defined around the head of a user wearing an HMD is shown. It is an XYZ space figure which shows an example of virtual space. It is a figure which shows an example of the method of determining a gaze direction. It is a block diagram which shows the function of the control circuit part for implement | achieving the function of a HMD system. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is the YZ plane view which looked at the visual field area from the X direction. It is the XZ plane figure which looked at the visual field area from the Y direction. It is a figure for demonstrating the relationship between a launcher space, 1st virtual space, and 2nd virtual space. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a figure for demonstrating the 1st virtual space and the 2nd virtual space. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a YZ plane view explaining the first deviation amount with respect to the reference direction of the gaze direction. It is a XZ plane figure explaining the 2nd shift amount with respect to the reference direction of a gaze direction. It is a figure which shows the state before being watched of the advertisement in a visual field area | region. It is a figure which shows the state after being watched of the advertisement in a visual field area | region. It is a figure which shows a user action management table. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a figure for demonstrating launcher space. It is a figure which shows a priority management table. It is a flowchart which shows the process for implement | achieving the function of a HMD system. It is a figure for demonstrating launcher space. It is a figure which shows the HMD system containing an external device. It is a figure which shows an example of a delivery content.

[Description of Embodiment of the Present Invention]
First, the contents of the embodiment of the present invention will be listed and described. A head mounted display system control program according to an embodiment of the present invention has the following configuration.
(Item 1)
On the computer,
A virtual space in which the user is immersed, and a step of generating a plurality of target objects in the virtual space and outputting them to a head mounted display;
Detecting a history of actions taken by the user in the virtual space for the target object;
Identifying priorities of the plurality of target objects based on the history;
Generating distribution content in which information about the target object is arranged based on the priority;
Outputting the distribution content to a device different from the head-mounted display.
According to the program of this item, the content related to the target object that the user has shown interest in the virtual space is transmitted to a device that can be browsed outside the virtual space. Thereby, contents such as advertisements and trial versions can be provided to the user without impairing the user's immersion in the virtual space.
(Item 2)
The distribution content is:
A first link that causes the device to display detailed information about the target object;
Item 1. The program according to Item 1, including a second link for downloading an extended content related to the target object output to the head mounted display.
According to the program of this item, even when the user does not use the head mounted display, the user can download contents such as advertisements and trial versions executed on the head mounted display (item 3).
Receiving a user instruction input to the second link;
Determining whether the head mounted display is worn by a user;
The program according to item 2, wherein when the head mounted display is not attached to a user, a download request for downloading the extended content is output.
According to the program of this item, it is possible to download extended contents such as a trial version when the head mounted display is not used. As a result, the extended content can be provided without giving the user who is using the head-mounted display a sense of incongruity such as deterioration of the communication status.
(Item 4)
The virtual space includes a first virtual space and a second virtual space;
The target object includes a first target object displayed in the first virtual space and a second target object displayed in the second virtual space and related to the first target object,
The program according to any one of Items 1 to 3, wherein the history is a total gaze time obtained by adding up times when the user gazes at the first target object and the second target object.
According to the program of this item, the user can propose more appropriate content by adding up the actions related to the user's line-of-sight history in a plurality of contents.
(Item 5)
In the computer,
Defining a reference position in the virtual space and a target object position where the target object is arranged;
Defining a reference direction that is the line-of-sight direction of the user with respect to the reference position;
Identifying a gaze direction that is a gaze direction of the user with respect to the target object;
Identifying a gaze time when the user gazes at the target object;
Further executing the step of identifying an adjusted gaze time based on the gaze time, the gaze direction, and the reference direction;
The program according to any one of items 1 to 4, wherein the history is the adjusted gaze time.
According to the program of this item, the content related to the advertisement is provided to the user based on the time when the user gazes at the target object such as the advertisement. At that time, the gaze time is adjusted according to the relationship between the gaze direction of the user and the reference direction set in the virtual space. Accordingly, it is possible to provide a content providing unit that can easily determine an object that the user has shown interest in and can approach in a timely manner.
(Item 6)
The program according to item 5, wherein the adjusted gaze time is calculated by adjusting the gaze time based on a shift amount of the gaze direction with respect to the reference direction.
According to the program of this item, when the user looks away from the reference direction and looks closely at the target object, it is estimated that the user is highly interested in the target. Accordingly, it is possible to provide a content providing unit that can approach the user's interest in a timely manner.
(Item 7)
The shift amount is a first shift amount that is a shift amount of the gaze direction with respect to the reference direction in a vertical plane, and a second shift amount that is a shift amount of the gaze direction with respect to the reference direction in a horizontal plane perpendicular to the vertical plane. Including displacement
The program according to item 6, wherein the adjusted gaze time is calculated by assigning a predetermined weight to the first deviation amount and the second deviation amount.
According to the program of this item, it becomes possible to attach importance to actions that are more difficult for the user to do spontaneously.
(Item 8)
Item 7. The program according to Item 7, wherein the weight is given to the first deviation amount.
According to the program of this item, content that can approach the user's interests in a timely manner by placing emphasis on the user's behavior that makes it difficult for the user to voluntarily move the line of sight in the vertical plane Means may be provided.

[Details of the embodiment of the present invention]
A specific example of the program according to the embodiment of the present invention will be described below with reference to the drawings. In addition, this invention is not limited to these illustrations, is shown by the claim, and intends that all the changes within the meaning and range equivalent to the claim are included. In the following description, the same reference numerals are given to the same elements in the description of the drawings, and redundant descriptions are omitted.

  FIG. 1 shows an HMD system 100 including an HMD 110 according to the present embodiment. The HMD system 100 includes an HMD 110 that is worn on the user's head, a control circuit unit 120, a motion sensor 130, and a gaze sensor 140.

The HMD 110 includes a display 112 that is a non-transmissive display device, a sensor unit 114, and a gaze sensor 140. The control circuit unit 120 displays a right-eye image and a left-eye image on the display 112, thereby providing a three-dimensional image using parallax between both eyes as a virtual space. By arranging the display 112 in front of the user's eyes, the user can be immersed in the virtual space. The virtual space includes a background, various objects that can be operated by the user, menu images, and the like.

  The display 112 may include a right-eye sub-display that provides a right-eye image and a left-eye sub-display that provides a left-eye image. Moreover, as long as the image for right eyes and the image for left eyes can be provided, you may be comprised by one display apparatus. For example, the right-eye image and the left-eye image can be provided independently by switching the shutter so that the display image can be recognized by only one eye at high speed.

  The control circuit unit 120 is a computer connected to the HMD 110 and presents a virtual space on the display 112 by executing a stored predetermined application. In addition, the control circuit unit 120 performs processing so as to operate various objects displayed in the virtual space and to display and control various menu images and the like. The control circuit unit 120 stores a program for controlling execution of such operations. The control circuit unit 120 may not be mounted on the HMD 110 and may be configured as another hardware (for example, a known personal computer or a server computer via a network). In addition, the control circuit unit 120 may implement only a part of functions in the HMD 110 and implement the remaining functions in different hardware.

  The motion sensor 130 detects information related to the position and inclination of the HMD 110. The motion sensor 130 includes a sensor unit 114 and a detection unit 132. The sensor unit 114 may include a plurality of light sources. The light source is, for example, an LED that emits infrared rays. The detection unit 132 is, for example, an infrared sensor, and detects information about the position and angle of the HMD 110 in the real space according to the user's movement by detecting infrared rays from the light source as detection points of the HMD 110 over time. Then, it is possible to determine time-dependent changes in the position and angle of the HMD 110 based on changes over time in the information detected by the detection unit 132, and to detect information related to the movement of the HMD 110.

  Information regarding the position and tilt acquired by the motion sensor 130 will be described with reference to FIG. The XYZ axes are defined around the head of the user wearing the HMD 110. The vertical direction in which the user stands upright is the Y axis, the front-rear direction orthogonal to the Y axis and connecting the center of the display 112 and the user is the Z axis, and the horizontal direction orthogonal to the Y axis and the Z axis is the X axis. Thereby, the temporal change of the position in the XYZ axis direction of the user is acquired. Further, the inclination angle θx (so-called pitch angle), θy (so-called yaw angle), and θz (so-called roll angle) of the HMD 110 around each axis are acquired.

  The motion sensor 130 may be configured by only one of the sensor unit 114 fixed near the display 112 and the detection unit 132. The sensor unit 114 may be a geomagnetic sensor, an acceleration sensor, or an angular velocity (gyro) sensor, and the position and inclination of the HMD 110 (particularly, the display 112) mounted on the user's head using at least one of them. Is detected. Thereby, the information regarding the motion of the HMD 110 can be detected. For example, the angular velocity sensor can detect the angular velocities around the three axes of the HMD 110 over time according to the movement of the HMD 110 and determine temporal changes in the angles around the respective axes. In this case, the detection unit 132 is not necessary. The detection unit 132 may include an optical camera. In this case, information related to the movement of the HMD 110 can be detected based on the image information, and the sensor unit 114 is unnecessary.

The function of detecting information related to the position and tilt of the HMD 110 using the motion sensor 130 is referred to as position tracking. The relationship between the position tracking by the motion sensor 130 and the virtual camera 1 arranged in the virtual space 2 will be described with reference to FIG. In order to explain the positional relationship between the virtual camera 1 and the motion sensor 130, the position of the motion sensor 130 is the position of the detection unit 132 when the detection unit 132 is provided, and is the sensor when the detection unit 132 is not provided. The position of the portion 114 is assumed. The virtual camera 1 is disposed inside the virtual space 2, and the motion sensor 130 is virtually disposed outside the virtual space 2 (real space).

The virtual space 2 is formed in a celestial sphere having a plurality of substantially square or substantially rectangular meshes. Each mesh is associated with the spatial information of the virtual space 2, and the visual field region 23 is defined based on this spatial information. In the present embodiment, it is preferable to adjust so that the center point 21 of the celestial sphere is always arranged on the line connecting the virtual camera 1 and the sensor 130 in the XZ plane. For example, when the user wearing the HMD moves and the position of the virtual camera 1 moves in the X direction, the center 21 is positioned on the line segment between the virtual camera 1 and the motion sensor 130. The area is changed. In this case, the position of the virtual camera 1 in the virtual space 2 is fixed, and only the tilt changes. On the other hand, if the position of the virtual camera 1 is moved in conjunction with the movement of the motion sensor 130 in the XYZ directions, the position of the virtual camera 1 in the virtual space 2 is variably set.

  The gaze sensor 140 has an eye tracking function that detects the direction in which the user's right eye and left eye gaze are directed. The gaze sensor 140 preferably includes a right eye sensor and a left eye sensor, and detects the gaze direction in which the user gazes by detecting the direction in which the gaze of the right eye and the left eye is directed respectively. A known sensor having an eye tracking function can be adopted as the gaze sensor 140. For example, the right eye and the left eye are irradiated with infrared light, and the reflected light from the cornea and the iris is acquired, whereby the rotation angle of the eyeball is adjusted. It may be asking.

As shown in FIG. 4, the gaze sensor 140 detects the gaze direction of the right eye and the left eye of the user U. When the user U is looking near, the lines of sight R1 and L1 are detected, and the gazing point N1 that is the intersection of the two is specified. Further, when the user is looking far away, the lines of sight R2 and L2 whose angles to the Z direction are smaller than the lines of sight R1 and L1 are specified. When the gazing point N1 is specified, the line-of-sight direction N0 of the user U is specified. The line-of-sight direction N0 is a direction in which the user U is actually directed by his eyes. The line-of-sight direction N0 is defined as a direction in which a straight line passes through the center of the right eye R and the left eye L of the user U and the center point N1, for example.

  The HMD system 100 may include headphones including a microphone as any element. Thereby, the user can give a voice instruction to a predetermined object in the virtual space. Further, in order to receive a broadcast of a television program on a virtual television in a virtual space, the HMD system 100 may include a television receiver in any element. Moreover, a communication function or the like for displaying an electronic mail or the like acquired by the user may be included. Further, an external controller for inputting various user commands may be included.

  FIG. 5 is a block diagram illustrating functions of the control circuit unit 120 for realizing display processing of the virtual space 2 in the HMD system 100 and various menu displays and object operations displayed in the virtual space 2. . The control circuit unit 120 controls an output image to the display 112 based mainly on inputs from the motion sensor 130 and the gaze sensor 140.

The control circuit unit 120 includes a display control unit 200, an object control unit 300, and a communication control unit. The display control unit 200 includes a virtual space image generation unit 210, an HMD motion detection unit 220, a line-of-sight detection unit 230, a visual field direction identification unit 240, a visual field region determination unit 250, a visual field image generation unit 260, and spatial information. Storage unit 270. The object control unit 300 includes an advertisement specifying unit 310, a gaze time measurement unit 320, an adjustment unit 330, a determination unit 340, a drawing unit 350, an object information storage unit 360, a virtual camera information storage unit 370, a user An action storage unit.

The motion sensor 130 and the gaze sensor 140 are communicably connected to the display control unit 200 and the object control unit 300, and can be connected via a wired or wireless communication interface. The display control unit 200, the object control unit 300, and the communication control unit 400 are communicably connected to the HMD 110 (display 112), and can be connected via a wired or wireless communication interface. The communication control unit 400 can communicate with the external server 160 via the network 150. The spatial information storage unit 270, the object information storage unit 360, the virtual camera information storage unit 370, and the user behavior storage unit 380 are various types for providing output information corresponding to inputs from the motion sensor 130 and the gaze sensor 140 to the display 112. Contains data.

  A processing flow of the HMD system 100 for providing the virtual space 2 will be described with reference to FIGS. The virtual space 2 can be provided by the interaction of the HMD 110 (the gaze sensor 140 and the motion sensor 130) and the control circuit unit 120.

  First, the control circuit unit 120 (virtual space image generation unit 210) generates the celestial spherical virtual space image 22 constituting the virtual space 2 in which the user is immersed (S120-1). When an operation such as movement or tilt is input from the user to the HMD 110 (S110-1), the position and tilt of the HMD 110 are detected by the motion sensor 130 (S130-1). Detection information of the motion sensor 130 is transmitted to the control circuit unit 120, and position information and tilt information of the HMD 110 are received by the HMD motion detection unit 220. Thereby, the visual field direction based on the positional information and inclination information of the HMD 110 is determined (S120-2).

  When the gaze sensor 140 detects the movement of the eyeballs of the left and right eyes of the user (S140-1), the information is transmitted to the control circuit unit 120. The line-of-sight detection unit 230 identifies the direction in which the line of sight of the right eye and the left eye is directed, and the line-of-sight direction N0 is identified (S120-3). The reference line-of-sight specifying unit 240 specifies the field-of-view direction specified by the inclination of the HMD 110 or the user's line-of-sight direction N0 as the reference line of sight 5 (S120-4).

The visual field area determination unit 250 determines the visual field area 23 of the virtual camera 1 in the virtual space 2 (S120-5). As shown in FIG. 3, the visual field region 23 is a part of the virtual space image 22 that constitutes the user's visual field. The visual field area 23 is determined based on the reference line of sight 5, and the reference line of sight 5 is determined based on the position and tilt of the virtual camera 1. 7A is a YZ plane view of the visual field area 23 as viewed from the X direction, and FIG. 7B is an XZ plane view of the visual field area 23 as viewed from the Y direction.

The visual field region 23 is a first region 24 (see FIG. 7A) that is a range defined by the reference line of sight 5 and the YZ cross section of the virtual space image 22, and a range defined by the reference line of sight 5 and the XZ cross section of the virtual space image 22. And a second region 25 (see FIG. 7B). The first region 24 is set as a range including the polar angle α with the reference line of sight 5 as the center. The second region 25 is set as a range including the azimuth angle β with the reference line of sight 5 as the center.

The view image generation unit 260 generates the view image 26 based on the view region 23 (S120-6). The field-of-view image includes two two-dimensional images for the left eye and the right eye, and these are superimposed on the display 112, whereby the virtual space 2 as a three-dimensional image is provided to the user. The HMD 110 receives information about the view image 26 from the control circuit unit 120 and displays the view image 26 on the display 112 (S110-2).

  A specific example of a virtual space provided to the user and a processing flow of the HMD system 100 for providing the virtual space to the user will be described with reference to FIG. 5 and FIG. As shown in FIGS. 8 and 9, the virtual space in the present embodiment includes a launcher space LAU, and a first virtual space VR1 and a second virtual space VR2 associated with the launcher space LAU. A launcher space LAU is first provided to a user who activates the HMD system 100 and wears the HMD 110. The launcher space LAU is generated by the virtual space image generation unit 210 and the visual field image generation unit 260 and is output to the HMD 110 (S120-7).

In the launcher space LAU, an application AP1 for providing the first virtual space VR1 and an application AP2 for providing the second virtual space VR2 are displayed. The user can move to the selected virtual space by inputting a predetermined operation (such as sending a line of sight for a certain period of time or operating a controller) to the applications AP1 and AP2 (S110-3). In the launcher space LAU, an advertisement space AD for displaying advertisements to be described later and trial version spaces TR1 and TR2 for displaying downloaded trial version contents are also displayed.

  An example of the first virtual space VR1 is shown in FIG. When the application AP1 is selected, the virtual space image generation unit 210 generates the virtual space image 22 constituting the first virtual space VR1. In addition, the drawing unit 350 of the object control unit 300 generates various objects with reference to the object information storage unit 360, and draws and arranges them in the first virtual space VR1 (S120-8). The first virtual space VR1 is a content providing entertainment such as a live stage, for example, and the stage S, the character C providing performance on the stage S, and various advertisements OB1 and OB2 are arranged in the first virtual space VR1. .

  The object control unit 300 specifies a reference position BP in the virtual space and a position (target object position) where the advertisements OB1 and OB2 as target objects to be measured are measured as described later. (S120-9). In a state immediately after the first virtual space VR1 is provided to the user, the user is located at the center of the celestial sphere of the first virtual space VR1, and the reference visual line 5 that is the user's visual line direction is directed to the reference position BP ( Reference direction BD described later). In the present embodiment, the view image 26 generated based on the reference direction BD includes a stage S, a character C, and a plurality of first advertisements (first target objects). Hereinafter, the advertisement included in the view image 26 may be referred to as a first advertisement (first object) OB1, and the advertisement not included in the view image 26 may be referred to as a second advertisement (second object) OB2.

  Note that the second virtual space VR2 is also the same as that in FIG. Schematically, when the application AP2 is selected, the virtual space image generation unit 210 generates the virtual space image 22 constituting the second virtual space VR2. In addition, the drawing unit 350 of the object control unit 300 generates various objects with reference to the object information storage unit 360, and draws and arranges them in the second virtual space VR2 (S120-8). Also in the second virtual space VR2, the stage S, the character C, the plurality of second advertisements (second target objects), the first advertisement (first object) OB1 included in the view image 26, and the view image 26 are not included. A second advertisement (second object) OB2 is arranged.

  The user can act freely in the first virtual space VR1 by inputting various movements such as a change in position and inclination to the HMD 110 (S110-4). When the motion sensor 130 detects the movement of the user, the HMD motion detection unit 220 receives this and specifies the position of the user in the virtual space (S120-10). When the motion sensor 130 or the gaze sensor 140 detects a change in the user's reference line of sight 5, the reference line-of-sight specifying unit 240 receives this and specifies the line-of-sight direction in the user's virtual space (S120-11). The object control unit 300 specifies the user's dyeing direction based on the position of the user in the virtual space and the user's reference line of sight 5 from the reference line-of-sight specifying unit 240. Then, the advertisement specifying unit 310 determines whether the line-of-sight direction is directed to any one of the advertisements OB1 and OB2 based on the information regarding the arrangement positions of the various advertisements OB1 and OB2 (S120-12). The advertisement specifying unit 310 specifies the user's line-of-sight direction directed to one of the advertisements OB1 and OB2 as the gaze direction GD. Then, the gaze time measuring unit 320 starts measuring the gaze time, which is the time during which the reference line of sight is sent to the advertisement (S120-13). When the user's gaze direction is directed to a certain range including a predetermined advertisement (a circle, an ellipse, a polygon, etc. including the advertisement), the gaze time measuring unit 320 displays the advertisement. The gaze time is measured assuming that the user is gazing.

  Of the user behavior history as described above, the behavior in which the user gazes at a predetermined advertisement is stored in the user behavior storage unit 370. The user behavior storage unit 370 includes, for example, a user behavior management table T1 illustrated in FIG. The user behavior management table T1 stores information on when and in what application (virtual space) the user gazes at which object and from which position in the virtual space, and the time when the user gazes at the object. .

  Here, the gaze time when the user gazes at the predetermined advertisement may be stored as the adjusted gaze time based on the processing flow shown in FIG. FIG. 12A is a diagram illustrating a reference direction BD-V and a gaze direction GD-V in a vertical plane (YZ plane) viewed from the user. FIG. 12B is a diagram illustrating a reference direction BD-H and a gaze direction GD-H in a horizontal plane (XZ plane) viewed from the user. First, as described above, the reference direction BD is specified based on the reference position BP in the first virtual space VR1 and the position of the user (the position of the virtual camera 1) (S120-14). At this time, the reference direction BD specifies the reference direction BD-V in the vertical plane (YZ plane) and the reference direction BD-H in the horizontal plane (XZ plane). In the present embodiment, since the character C as the leading role of the content is arranged on the stage S, the user's reference line of sight BD-V is set to face a little information. Further, the gaze direction GD is specified based on the advertisement position OB1 and the user position in the first virtual space VR1 (S120-15). At this time, the gaze direction GD specifies the gaze direction GD-V in the vertical plane (YZ plane) and the gaze direction GD-H in the horizontal plane (XZ plane).

  Next, the object control unit 300 obtains a deviation amount between the gaze direction GD and the reference direction BD (S120-16). In the present embodiment, first shift amounts DV1 and (S120-17), which are shift amounts (angle differences) of the gaze direction GD-V with respect to the reference direction BD-V in the vertical plane, and the gaze direction GD-H in the horizontal plane. The second displacement amount DV2 that is the displacement amount (angle difference) with respect to the reference direction BD-H is obtained (S120-18).

  Then, the adjustment unit 330 calculates the adjusted gaze time by adding the first deviation amount DV1 (angle: radians) and the second deviation amount DV2 (angle: radians) to the gaze time (S120-20).

  β1 is a coefficient for weighting the first deviation amount DV1. β2 is a coefficient for weighting the second deviation amount DV2. The adjustment unit 330 sets β1 and β2 to predetermined values in advance (S120-19), thereby placing an emphasis on the action history that the user has watched an advertisement that has taken actions that are less likely to be spontaneous. Can be adjusted.

  In the present embodiment, it is preferable that the weighting coefficient β1 for the first deviation amount DV1 is larger than the weighting coefficient β2 for the second deviation amount DV2. It is estimated that the user feels more stress on the head movement in the vertical direction than the head movement in the horizontal direction. In this way, it is possible to approach the user's interests in a timely manner by placing importance on the behavior of the user moving his / her line of sight in the vertical plane, which is a behavior that is more difficult for the user to voluntarily.

DV1 and DV2 are values of −π or more and π or less. Further, DV1 and DV2 are raised to the power (square or higher) and applied to the calculation of the adjusted gaze time. As a result, the adjusted gaze time can be calculated with a greater emphasis on the action of the user gazing at the advertisement until the user turns away from the reference direction. The adjusted gaze time is stored in the user behavior management table T1 of the user behavior storage unit 370.

  The determination unit 340 determines whether or not the adjusted gaze time has exceeded a predetermined threshold (S120-21). In this embodiment, when the adjusted gaze time is 5 seconds or more, the user interprets that the user has shown interest in the advertisement, and the threshold is set to 5 seconds. If the adjusted gaze time exceeds a predetermined threshold, it is accepted that the user gazes at the advertisement (S120-22).

  FIG. 13A and FIG. 13B show an example of processing when the user is accepted as having watched the advertisement. FIG. 13A shows a state before being accepted. FIG. 13B shows the state after being accepted. In a state before being accepted, three types of advertisements A, B, and C are displayed in the advertisement OB1 in the view field area 23. When the user's gaze on the advertisement OB1 on which the advertisement A is displayed is received, the drawing unit 350 changes the display of the advertisement OB1 on which the advertisement B is displayed to the advertisement A-1 related to the advertisement A and the advertisement C The display of the advertisement OB1 on which is displayed is changed to the advertisement A-2 related to the advertisement A. Thereby, when a user gazes at a predetermined object, other objects in the user's field of view are changed so as to relate to the gaze object. This makes it possible to approach the user's interest in a timely manner.

  As described above, when the adjusted gaze time for the predetermined advertisement is specified in the first virtual space VR1 (S120-23), an instruction to end the first virtual space VR1 is input from the user to the HMD 110 ( S110-5) is shown in FIG. In this case, the determination unit 340 refers to the user behavior storage unit 370 and specifies the priority of each advertisement (S120-24). The priority of the advertisement is specified by the total value of the adjusted gaze time. As shown in FIG. 14, the priority of the advertisement at the time when the first virtual space VR1 is finished is specified as A-2> A> A-1. Is done.

  The determination unit 340 transmits information regarding the priority of the advertisement to the view image generation unit 260. The field-of-view image generation unit 260 generates a launcher space LAU as shown in FIG. 16 based on the information and outputs it to the HMD 110 (S120-25). In the advertisement space AD of the launcher space LAU, a plurality of advertisements are arranged according to priority. The advertisement A-2 determined to have the highest priority may be displayed larger or displayed with some emphasis.

  Subsequently, when the user selects the application AP2 that activates the second virtual space VR2, the second virtual space VR2 is provided to the HMD 110 (S110-6). Also in the second virtual space VR2, the user's gaze time (adjusted gaze time) for the advertisement is specified by the same processing flow as in the first virtual space VR1 (S120-26).

  When the user finishes using the HMD 110 (S110-7), the determination unit 340 specifies the total adjusted gaze time as the total gaze time (S120-27). The total adjusted gaze time is managed in the priority management table T2 shown in FIG. The priority management table T2 is stored in the user behavior storage unit 370.

  The priority management table T2 calculates the total value of adjusted gaze times for each advertisement based on the user behavior management table T1. In the present embodiment, as shown in the user behavior management table T1, the user is watching the advertisement A-2 in the first virtual space VR1 and the second virtual space VR2. At that time, since the user is gazing at the advertisement A-2 from different user positions, the user's interest during wearing of the HMD is obtained more accurately by adding the adjusted gazing time taking into account the user's line-of-sight direction. be able to. Therefore, in the priority management table T2, the total value of the adjusted gaze time is managed.

  The determination unit 340 sets the priority of each advertisement based on the total adjusted gaze time. That is, the higher priority is given in the order of the longer the total adjusted gaze time. At this time, the advertisement A-1 that does not exceed the threshold value of 5 seconds is excluded from the priority setting targets.

In addition, the determination unit 340 determines whether or not there is a trial version content as extended content related to each advertisement for which priority is set. In the present embodiment, since the two trial version spaces TR1 and TR2 are set in the launcher space LAU, the top two advertisements among the advertisements having the trial version contents are specified (ads B and B-1). When the trial version content is specified, the determination unit 340 instructs the communication control unit 400 to output a download request for downloading the trial version content. When receiving the instruction and information notifying that the user has finished using the HMD from the HMD 110, the communication control unit 410 instructs the communication processing unit 420 to output a download request. The communication processing unit 420 connects to the external server 160 via the network 150 and transmits a trial content download request. The external server 160 transmits the trial version content to the communication processing unit 420 via the network 150 based on the download request.

As described above, in this embodiment, the extended content related to the advertisement (for example, a trial version of the game) can be downloaded when the user is not using the HMD 110. As a result, the extended content can be provided without giving the user who is using the HMD 110 a sense of incongruity such as deterioration of the communication status.

  When the user resumes using the HMD 110, the control circuit unit 120 specifies the downloaded trial version content (S120-30). In the present embodiment, it is determined whether or not the download of the trial version content regarding the advertisement B and the advertisement B-1 is completed, and when the download is completed, the trial version content is specified. Then, similarly to the above, the priority management table T2 is referred to, and the priority of each advertisement set when the user uses the previous HMD 110 is specified (S120-31). Thereafter, the field-of-view image generation unit 260 generates the launcher space LAU shown in FIG. 18 and outputs it to the HMD 110 (S120-32). In the advertisement space AD of the launcher space LAU, advertisements up to priority 5 are arranged in order from the top. Further, the advertisement B having the highest priority is greatly highlighted. In addition, the trial version contents related to the advertisement B and the advertisement B-1 are arranged in the trial version spaces TR1 and TR2.

  The user can select any one of the first virtual space VR1, the second trial version content VR2, and the trial version content arranged in the trial version spaces TR1 and TR2 and enjoy the corresponding virtual space. Moreover, it can also instruct | indicate so that the information regarding it may be transmitted to other information terminals (a personal computer, a smart phone, etc.) by selecting an advertisement.

  Note that once the HMD 110 is not attached, it is preferable not to add up to the time when the advertisement was watched when the HMD 110 was attached last time, even if the advertisement is watched in a virtual space that has been immersed thereafter. That is, it is preferable that the process of adding the time when the advertisement is watched is performed only within one HMD 110 wearing time. Accordingly, it is possible to provide a content providing means capable of approaching the user's interest in a timely manner without averaging the user's interest over a certain long period of time.

  Further, as shown in FIGS. 20 and 21, the priority of the target object is determined based on the history of actions taken on the target object in the user's virtual space (measurement / adjustment of gaze time) as described above. After the identification, the distribution content including information on the target object may be output to the device 170 different from the HMD. FIG. 20 is a diagram for explaining the HMD system 100 including the device 170. The device 170 is a device that can communicate with the control circuit unit 120 via the network 150 and various cables. For example, a smartphone, a PDA, a tablet computer, a game console, and a notebook PC can be applied to the device 170, and a CPU, a main memory, an auxiliary memory, a transmission / reception unit, a display unit (touch panel 180), and a display unit (touch panel 180), and Any portable device including an input unit can be provided. The device 170 is communicably connected to the communication control unit 400 of the control circuit unit 120, receives distribution content including various contents, and causes the display unit to display the distribution content. Accordingly, the content related to the target object that the user has shown interest in the virtual space is transmitted to the device 170 that can be browsed outside the virtual space, so that the user's immersion in the virtual space is not impaired. Various contents can be provided.

FIG. 21 shows an example of the distribution content CON output to the device 170. The distribution content CON is output to the device 170 by being push-distributed to a user's email address registered in advance or a dedicated application installed in advance. The distribution content CON includes an advertisement AD as a target object, a first link LN, and a second link TR. In the advertisement space AD, images related to the target object are arranged in order from the top, based on the priority set in the priority management table T2 shown in FIG. 17 set by the action history of the user in the virtual space. The Further, the advertisement B having the highest priority is highlighted.

The first link displays detailed information regarding the advertisement AD. The first link is associated with a URL for displaying detailed information related to the advertisement AD on the display unit of the device 170 when a user's touch operation or the like is received. The second link is associated with a URL for causing the control circuit unit 120 to download the trial version content output to the HMD 110. When receiving the touch operation or the like, the second link causes the communication control unit 400 of the control circuit unit 120 to output a download request so as to download the trial version content from the URL. The control circuit unit 120 downloads the trial version content from the external server 160 via the network 150. Thereby, even when the user is not using the HMD 110, the user can download contents such as advertisements and trial versions executed by the HMD 110.

  It is preferable that the control circuit unit 120 determines whether or not the user is wearing the HMD 110, and if not, outputs a download request for the trial version content. Thereby, it is possible to acquire the trial version content without giving the user who is using the HMD 110 a sense of incongruity such as deterioration of the communication status.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. Those skilled in the art will appreciate that various modifications of the embodiments can be made without departing from the spirit and scope of the invention as set forth in the appended claims.

  For example, in the above-described embodiment, the priority of the content is determined based on the adjusted gaze time. However, the priority may be determined based on the gaze time before adjustment.

In the above-described embodiment, the advertisements A, A-1, and A-2, which are contents related to each other, are treated as not relevant in calculating the gaze time or the total adjusted gaze time. It may be handled as related. For example, the gaze time or adjusted gaze time of the advertisements A, A-1, and A-2 may be summed, and the priority may be determined by integrating the advertisements A, A-1, and A-2. In this case, when it is determined that the priority of the advertisements A, A-1, and A-2 is high, all the advertisements related to the advertisements A, A-1, and A-2 and the extended content are displayed as launchers. It is good also as displaying on space LAU.

  DESCRIPTION OF SYMBOLS 100 ... Head mounted display (HMD) system, 110 ... HMD, 112 ... Display, 114 ... Sensor part, 120 ... Control circuit part, 130 ... Motion sensor, 132 ... Detection part, 140 ... Gaze sensor, 200 ... Display control part, DESCRIPTION OF SYMBOLS 300 ... Object control part, 400 ... Communication control part, 1 ... Virtual camera, 2 ... Virtual space, 5 ... Reference line of sight, 22 ... Virtual space image, 23 ... View field area, 26 ... View field image, BP ... Reference point, BD ... reference direction, GD ... gaze direction, CON ... distributed content, AD ... advertisement space, LN ... first link, TR ... second link.

Claims (8)

On the computer,
A virtual space in which the user is immersed, and a step of generating a plurality of target objects in the virtual space and outputting them to a head mounted display;
Detecting a history of actions taken by the user in the virtual space for the target object;
Identifying priorities of the plurality of target objects based on the history;
Generating distribution content in which information about the target object is arranged based on the priority;
Outputting the delivery content to a device different from the head mounted display;
A program that executes The distribution content is:
A first link that causes the device to display detailed information about the target object;
Including a second link for downloading the extended content related to the target object output to the head mounted display;
The program of claim 1. Receiving a user instruction input to the second link;
Determining whether the head mounted display is worn by a user;
When the head mounted display is not worn by the user, a download request for downloading the extended content is output.
The program according to claim 2. The virtual space includes a first virtual space and a second virtual space;
The target object includes a first target object displayed in the first virtual space and a second target object displayed in the second virtual space and related to the first target object,
The history is a total gaze time obtained by adding up times when the user gazes at the first target object and the second target object.
The program in any one of Claims 1-3. In the computer,
Defining a reference position in the virtual space and a target object position where the target object is arranged;
Defining a reference direction that is the line-of-sight direction of the user with respect to the reference position;
Identifying a gaze direction that is a gaze direction of the user with respect to the target object;
Identifying a gaze time when the user gazes at the target object;
Further executing the step of identifying an adjusted gaze time based on the gaze time, the gaze direction, and the reference direction;
The program according to claim 1, wherein the history is the adjusted gaze time. The program according to claim 5, wherein the adjusted gaze time is calculated by adjusting the gaze time based on a shift amount of the gaze direction with respect to the reference direction. The shift amount is a first shift amount that is a shift amount of the gaze direction with respect to the reference direction in a vertical plane, and a second shift amount that is a shift amount of the gaze direction with respect to the reference direction in a horizontal plane perpendicular to the vertical plane. Including displacement
The program according to claim 6, wherein the adjusted gaze time is calculated by assigning a predetermined weight to the first deviation amount and the second deviation amount. The program according to claim 7, wherein the weighting is largely applied to the first shift amount.