JP7017474B2 - Application control program, application control method and application control system - Google Patents

Description

The present invention relates to an application control program, an application control method, and an application control system for allowing a plurality of users to share a virtual space.

A virtual reality game is known in which an image of a virtual space is displayed and played on a mounted display (hereinafter, HMD: head-mounted display) mounted on a user's head. The HMD is provided with a tracking sensor such as a gyro sensor, and can detect the movement of the user's head. When the user moves the head, the HMD follows the movement of the head to change the image displayed on the display. In this way, the game image follows the movement of the head, so that the user can further concentrate his / her consciousness on the game. In other words, it is possible to increase the immersive feeling in the game.

Further, as one of such virtual reality games, a game played by a plurality of users has also been proposed. Patent Document 1 describes a game in which a plurality of users each wear an HMD and play a game. In this game, the position information of each user wearing the HMD is reflected in the position information of the avatar in the virtual space, and the avatar of each user is drawn in the virtual space.

Japanese Unexamined Patent Publication No. 2014-17776

On the other hand, when a plurality of users try to play a virtual reality game, it is not always possible to prepare the same number of HMDs as the number of players to play. In addition, there may be some users who are reluctant to wear the HMD. Although various situations are expected in this way, a game in which both the user who does not wear the HMD and the user who wears the HMD can participate, and the user who does not wear the HMD contributes more to the application. I can't find it.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an application control program, an application control method, and an application control system that can be played simultaneously by a user who wears a wearable display and a user who does not wear the wearable display. To do.

The application control program that solves the above-mentioned problems is an application control program that outputs an image of a virtual space by using a control unit that outputs an image to a wearable display worn by the first user, and the first user causes the control unit to output an image of the virtual space. The step of outputting the image of the virtual space to the mounted display according to the posture of the user, the step of acquiring the designated position of the virtual space based on the operation of the second user, and the step of acquiring the designated position of the virtual space on the mounted display. The step of outputting the reflected image of the virtual space and the step of outputting the image are executed.

As a result, a user who wears the wearing display and a user who does not wear the wearing display can play at the same time.

With respect to the application control program, it is preferable to have the control unit further execute a step of performing an action in the virtual space reflecting the instruction position in response to an operation by the operation unit by the first user.

As a result, the second user can instruct the position where the action is to be performed by the instructed position.

With respect to the application control program, it is preferable that the step of performing the action moves an object in the virtual space in response to an operation by the first user.

For the application control program, it is preferable that the step of outputting the image of the virtual space reflecting the instruction position is displayed together with the type of the object designated by the second user.

For the application control program, it is preferable to have the control unit further execute a step of outputting an image of the virtual space to the display used by the second user.

Regarding the application control program, it is preferable to display an image overlooking the virtual space in the step of outputting the image of the virtual space to the display used by the second user.

The application control method for solving the above problems is an application control method for outputting an image of a virtual space by using a control unit for outputting an image to a wearable display worn by the first user, and the control unit is the first. The step of outputting the image of the virtual space to the mounted display according to the posture of the user, the step of acquiring the designated position of the virtual space based on the operation of the second user, and the step of acquiring the designated position on the mounted display. A step of outputting the reflected image of the virtual space is performed.

As a result, a user who wears the wearing display and a user who does not wear the wearing display can play at the same time.

In an application control system that solves the above problems, an application control system that outputs an image of a virtual space including a control unit that outputs an image to a wearable display worn by a first user, wherein the control unit is the first. The step of outputting the image of the virtual space to the mounted display according to the posture of the user, the step of acquiring the designated position of the virtual space based on the operation of the second user, and the step of acquiring the designated position of the virtual space on the mounted display. A step of outputting the reflected image of the virtual space is performed.

As a result, a user who wears the wearing display and a user who does not wear the wearing display can play at the same time.

According to the present invention, a user who wears a wearable display and a user who does not wear the wearable display can play the application at the same time.

A conceptual diagram that gives a bird's-eye view of the application control system of the first embodiment. The block diagram functionally showing the application control system of the same embodiment. The conceptual diagram explaining the virtual space and the image visually recognized by a user in the same embodiment. Explanatory drawing of the 2nd virtual space image output to the touch panel display of the same embodiment. Explanatory drawing of the bird's-eye view image output to the touch panel display of the same embodiment. Explanatory drawing of the 2nd virtual space image output to the touch panel display of the same embodiment. Explanatory drawing of the 1st virtual space image output to the wearing display of the same embodiment. Explanatory drawing of the 1st virtual space image output to the wearing display of the same embodiment. An explanatory diagram of a processing procedure for outputting an image to a mounted display in the same embodiment. An explanatory diagram of a processing procedure for stopping a moving object in the same embodiment. Explanatory drawing of the 2nd virtual space image output to the touch panel display of 2nd Embodiment. The explanatory view of the 2nd virtual space image output to the touch panel display of 3rd Embodiment. The explanatory view of the 1st virtual space image output to the wearing display of 4th Embodiment. Explanatory drawing of the first virtual space image output to the touch panel display of the modification. Explanatory drawing of the second virtual space image output to the touch panel display of the modification. An explanatory diagram of a virtual space image output to the mounted display of the modified example, (a) is an image output to the mounted display, and (b) is an image output to the touch panel display.

(First Embodiment)
Hereinafter, an embodiment of the application control program, the application control method, and the application control system will be described with reference to FIGS. 1 to 10. In this embodiment, it is assumed that a game application is provided as an application that shares a virtual space among a plurality of users.

As shown in FIG. 1, the application control system includes a head-mounted display (hereinafter, HMD) 11 as a wearable display, an operation unit (controller) 14, a main display 12, and a touch panel display 13. The HMD 11 is attached to the head of the first user 101. An image of the virtual space is displayed on the HMD 11. The first user 101 searches for an object that moves in the virtual space (hereinafter referred to as a moving object), and performs an action (second action) by operating the operation unit 14 on the found moving object. In the present embodiment, the first user 101 searches for a moving object such as a "ghost" and turns on the operation unit 14 to sniper the moving object.

The images output to the main display 12 and the touch panel display 13 are visually recognized by the second user 102. The second user 102 operates the touch panel display 13 while visually recognizing the images output to the main display 12 and the touch panel display 13 to support the first user 101. The second user 102 may be able to teach the position of the moving object to the first user 101 by a real voice or a voice input through a microphone.

A schematic configuration of the application control system 10 will be described with reference to FIG. The application control system 10 includes an HMD 11, an operation unit 14, a main display 12, a touch panel display 13, and a control device 15 connected to these.

The HMD 11 has an HMD control unit 30, a tracking sensor 31, and a display unit 32. The HMD control unit 30 outputs a signal corresponding to the detection result of the tracking sensor 31 to the control device 15, and inputs various signals such as an image signal from the control device 15. The tracking sensor 31 is a sensor for identifying the point of interest of the first user 101, and detects the position and orientation of the head of the first user 101. The tracking sensor 31 has at least one sensor such as a gyro sensor, an acceleration sensor, and a geomagnetic sensor. As another embodiment, a sensor that directly detects the movement of the eyeball of the first user 101, for example, a line-of-sight detection sensor that incidents near infrared rays into the iris and detects the reflected light, is used as the tracking sensor 31. May be done. The display unit 32 of the HMD 11 displays an image in a predetermined range centered on the point of interest of the first user 101 in the virtual space. That is, the image in a predetermined range is visually displayed to the first user 101. The image of the virtual space output to the display unit 32 of the HMD 11 is hereinafter referred to as a “first virtual space image”.

The operation unit 14 is provided with, for example, an operation button and an operation lever, and is configured so that the first user 101 can operate the operation without visually recognizing it. The operation unit 14 has an acceleration sensor or the like, and can detect a change in the direction from the initial position and the amount of the change. The operation unit 14 outputs a signal indicating on / off of the operation by the second user 102, an amount of change in the direction of the operation unit 14, and the like to the control device 15.

The main display 12 is a display that can be visually recognized by the second user 102. The first virtual space image is displayed on the main display 12 in synchronization with the display unit 32 of the HMD 11.

The touch panel display 13 may be a touch panel display provided in a housing, or may be a portable information terminal such as a tablet or a smartphone. An image obtained by projecting a part of the virtual space (hereinafter referred to as a second virtual space image) is output to the touch panel display 13. In the present embodiment, as the second virtual space image, an image viewed from the viewpoint set at a position above the vertical direction of the first user 101 is used. The initial screen of the second virtual space image displayed on the touch panel display 13 when the game is started is the same as the position of the virtual space projected on the first virtual space image displayed on the display unit 32 of the HMD 11. .. The viewpoint position of the second virtual space image on the initial screen may be the same as or different from the viewpoint position of the first virtual space image on the initial screen. The position where the virtual space is projected onto the second virtual space image is changed and designated according to a predetermined touch operation such as a swipe operation of the second user 102. For example, the swipe operation of the second user 102 changes the projection range of the virtual space to change the second virtual space image, and the swipe operation is stopped, so that the change of the projection range of the virtual space is stopped. The display range of the second virtual space image is specified. The "touch operation" includes, for example, a tap operation, a swipe operation, a flick operation, a pinch-in operation, a pinch-out operation, a touch-and-hold operation, and the like. Further, the touch panel display 13 outputs a signal corresponding to the touch operation of the second user 102 to the control device 15. For example, the touch panel display 13 outputs a signal corresponding to the locus of the swipe operation, a signal corresponding to the contact position, and the like.

The first virtual space image output to the display unit 32 of the HMD 11 and the second virtual space image output to the touch panel display 13 are images in the same virtual space, but the former depends on the posture of the first user 101. The latter changes the display range of the virtual space in response to the touch operation of the second user 102, whereas the display range of the virtual space is changed. Further, the touch panel display 13 displays a bird's-eye view image of the virtual space from a predetermined position.

The control device 15 acquires the signals output from the HMD 11, the operation unit 14, and the touch panel display 13, and manages the progress of the game in the virtual space by the first user 101 and the second user 102.

The control device 15 includes a control unit 16, a storage unit 17, and an image information storage unit 25. The control unit 16 has a calculation unit, a volatile storage unit, and a non-volatile storage unit, and executes a game according to an application control program recorded in the non-volatile storage unit. The storage unit 17 is a storage unit for image processing, and may be a volatile storage unit or a non-volatile storage unit. Game management information 18, attention point information 19, designated display range information 20, object position information 21, and contact position information 22 are recorded in the storage unit 17.

The game management information 18 is information including a user ID, a progress status of the game, and the like, and is updated as the game progresses. The attention point information 19 is information on the position and orientation of the head of the first user 101 detected by the tracking sensor 31, or information indicating the position of the attention point specified from the position and orientation of the head. The attention point information 19 is updated according to the position and orientation of the head of the first user 101. The designated display range information 20 is information indicating a range displayed on the touch panel display 13 by the second user 102 by a touch operation, and is updated with the touch operation of the second user 102. The object position information 21 is information indicating the position of a moving object moving in the virtual space. The control unit 16 randomly moves each of the moving objects according to the application control program. The object position information 21 is updated as the position of the moving object changes. The position of the moving object is synchronized between the first virtual space image output to the HMD 11 and the main display 12 and the second virtual space image output to the touch panel display 13. Further, the contact position information 22 indicates the contact position on the screen of the touch panel display 13, and is updated every time the touch operation of the second user 102 is performed. Further, the image information storage unit 25 records information for drawing an image in the virtual space.

The control unit 16 records the attention point information 19 based on the signal input from the HMD 11 in the storage unit 17. Further, the control unit 16 records the signal corresponding to the designated display range input from the touch panel display 13 in the storage unit 17 as the designated display range information 20.

The control unit 16 acquires image information in a predetermined range centered on the attention point based on the attention point information 19 among the image information recorded in the image information storage unit 25, and uses the HMD 11 and the main display as the first virtual space image. Output to 12. Further, the control unit 16 records the designated display range designated by the touch operation of the second user 102 in the storage unit 17 as the designated display range information 20. Further, the control unit 16 acquires image information from the image information storage unit 25 based on the designated display range information 20, and outputs the image information to the touch panel display 13 as a second virtual space image. Further, the control unit 16 outputs a bird's-eye view image of a part or the whole of the virtual space to the touch panel display 13.

With reference to FIG. 3, the virtual space and the first virtual space image output to the HMD 11 and the main display 12 will be described. The HMD control unit 30 specifies the position and orientation of the head of the first user 101. For example, the position and orientation of the head are centered on the X-axis along the horizontal direction and the pitch θp indicating the rotation direction centered on the X-axis, and the Y-axis and Y-axis along the vertical direction with reference to the initial position. It is calculated as the yaw θy indicating the rotation direction, the Z-axis along the depth direction, and the position of the roll θr indicating the rotation direction about the Z-axis in the three-axis direction and the angle around each axis. The HMD control unit 30 outputs the position in the three-axis direction and the angle around each axis to the control unit 16 as the point of interest information 19.

The control unit 16 specifies the attention point 114 or the line-of-sight direction D in the virtual space 110 based on the attention point information 19. As a method for specifying the point of interest or the direction of the line of sight, a known method according to the type of the tracking sensor 31 or the like can be used. For example, the control unit 16 virtually sets a straight line connecting the left eye position and the right eye position, extends in a direction orthogonal to the straight line, and sets the direction passing through the midpoint between the left eye position and the right eye position as the line-of-sight direction D. .. Further, the control unit 16 can specify the intersection of the line-of-sight direction D and the object (or background) as the point of interest 114. When a tracking sensor that detects the movement of the eyeball or detects the reflected light from the inside of the iris is used, the line-of-sight direction may be specified based on the movement of the eyeball or the reflected light.

The control unit 16 acquires the image information of the attention point 114 and the attention range 111, which is a range based on the line-of-sight direction D, from the image information storage unit 25, and outputs the image information to the display unit 32 of the HMD 11 as a first virtual space image. For example, the attention range 111 has a horizontal horizontal viewing angle α centered on the line-of-sight direction D and a vertical direction when the direction from the position of the viewpoint of the first user 101 toward the point of interest 114 is the line-of-sight direction D. It is a range specified by a vertical viewing angle (not shown) or the like, which is an angle. The horizontal viewing angle α and the vertical viewing angle are not the physical viewing angles of the first user 101, but the angles of rendering that generate the image of the virtual space 110. Further, the control unit 16 also outputs an image corresponding to the acquired image information to the main display 12.

In the first virtual space image displayed on the HMD 11 and the first virtual space image displayed on the main display, the entire virtual space is displayed in the initial state when a predetermined action is not performed on the images. For example, the initial state is a "dark" state, that is, a state in which the brightness and saturation of the image are low. In the first virtual space image in this initial state, the moving object 115 moving in the virtual space is displayed in a hidden state. The hidden state is a state in which the moving object is not displayed at all or is difficult to see. As an example of an unclear display state, a state in which only a part of the moving object 115 is displayed and not entirely displayed, or a state in which at least one of the lightness, saturation, and hue of the moving object 115 is brought close to the background to make it difficult to see. Is. Further, for example, in the virtual space, the second user 102 is displayed as a character 116 such as a drone or an avatar. In the present embodiment, the position of the character 116 maintains the position above the vertical direction of the first user 101.

Further, when the display range is designated by the touch operation of the second user 102 on the touch panel display 13, the control unit 16 acquires the designated display range from the touch panel display 13 and stores the designated display range information 20 as the storage unit 17. Record in. Further, the control unit 16 outputs the designated display range 112 corresponding to the designated display range information 20 to the touch panel display 13 as the second virtual space image in a display mode different from the initial state of the first virtual space image. Specifically, the display mode at this time is a changed state in which at least one of brightness, saturation and hue is different from the initial state, for example, a state in which light is irradiated, that is, the brightness is the initial state. It is a state in which the saturation and hue are different from the initial state. The control unit 16 visually displays the moving object on the second virtual space image based on the object position information 21. As a result, the second user can visually recognize the moving object on the second virtual space image.

Further, when at least a part of the attention range 111 of the first user 101 and the designated display range 112 designated by the second user 102 overlap, the control unit 16 is among the first virtual space images. The overlapping part is displayed in a changed state different from the initial state. Specifically, the overlapping portion is displayed in a state in which at least a part of lightness, saturation, and hue is different from the other portion. Note that FIG. 3 shows a state in which the attention range 111 includes the entire designated display range 112. In the first virtual space image, the light is displayed as if it was emitted from the character 116 itself or a light source possessed by the character 116.

The control unit 16 designates the designated display range 112 designated by the second user 102 in the virtual space 110 (first action), and reflects the first action on the first virtual space image. Specifically, the control unit 16 displays only the moving object 115 included in the overlapping portion (overlapping range in the virtual space) between the attention range and the designated display range 112 in the first virtual space image (visible state). Then, the moving object 115 existing other than the overlapping portion is set to a hidden state (invisible state). Therefore, the first user 101 can visually recognize the moving object 115 only when the attention range 111 and the designated display range 112 overlap and the moving object 115 is included in the overlapping portion.

Further, in the present embodiment, the designated display range 112 designated by the second user 102, in other words, the horizontal viewing angle β of the projection range in the virtual space for generating the second virtual space image is the first virtual space. It is smaller than the horizontal viewing angle α of the projection range in the virtual space to generate. That is, when the horizontal viewing angle β of the second virtual space image is equal to or greater than the horizontal viewing angle α of the first virtual space image, the first user 101 can easily set the designated display range 112 reflected in the first virtual space image. Can be found in. On the other hand, when the horizontal viewing angle β of the second virtual space image is smaller than the horizontal viewing angle α of the first virtual space image, the first user 101 finds the designated display range 112 reflected in the first virtual space image. It becomes difficult. The ratio of the horizontal viewing angle β of the second virtual space image to the horizontal viewing angle α of the first virtual space image may be determined according to the target game balance. In the present embodiment, by making the horizontal viewing angle β of the second virtual space image smaller than the horizontal viewing angle α of the first virtual space image, the contribution of the second user 102 to the game is increased, and the first user 101 And the second user 102 cooperate to advance the game.

Further, the second user 102 can stop the movement of the moving object 115 in the vicinity of the contact position by a predetermined touch operation on the touch panel display 13. At this time, the second user 102 performs a touch-and-hold operation for continuing contact between the finger and an arbitrary position on the screen. When the touch-and-hold operation is performed, the control unit 16 identifies a moving object within a predetermined distance from the contact position as the first action, and stops the movement (movement) of the specified moving object 115. That is, the movement of one nearest moving object 115 is stopped with respect to one contact position. For example, when five moving objects 115 exist in the virtual space 110 and it is desired to stop these movements, a touch-and-hold operation is performed using five fingers.

Further, the first user 101 can perform the second action only on the moving object 115 which is included in the designated display range 112 and becomes visible. In the present embodiment, as the second action, the moving object 115 is sniped. At this time, if the movement of the moving object 115 is stopped by the touch operation of the second user 102, it becomes easy to aim at the moving object 115, and it becomes easy to perform an effective second action. Specifically, when the control unit 16 detects an on operation of the operation unit 14, the control unit 16 emits a ray 118 from the virtual position 117 of the operation unit 14 along the direction of the operation unit 14 to the first virtual space image and the second virtual. Display on a spatial image. Then, when the direction of the ray 118 and the position of the moving object 115 match, the control unit 16 determines that the second action has been effectively performed. Specifically, it is determined that the moving object 115 has been shot, and a score or the like is given to the first user 101. On the other hand, the control unit 16 does not display the action ray 118 for the moving object 115 that is not included in the designated display range 112. Alternatively, even if the action ray 118 is emitted toward the moving object 115 that is not included in the designated display range 112, it may be determined to be invalid. The control unit 16 causes the light ray 118 for action to be displayed on the display unit 32, the main display 12, and the touch panel display 13 of the HMD 11.

Next, the screens displayed on the display unit 32, the main display 12, and the touch panel display 13 of the HMD 11 will be described with reference to FIGS. 4 to 8.
As shown in FIG. 4, the control unit 16 renders the image information of the virtual space recorded in the image information storage unit 25 according to the designated display range 112, and displays an image for displaying the second virtual space image 121. Information is generated, and the generated image information is adjusted for brightness, saturation and hue for the second virtual space image 121, and output to the touch panel display 13. Even if the second virtual space image 121 displays the same range as the first virtual space image, at least a part of the brightness, saturation, and hue is different from the initial state of the first virtual space image. Specifically, the control unit 16 displays the entire second virtual space image 121 in a state of high brightness as if it was irradiated with light. Further, the control unit 16 causes the second virtual space image 121 to display the moving object 115 that moves based on the object position information 21. However, since the moving object 115 moves randomly, if the designated display range 112 is not changed, the initially displayed moving object 115 may move off the screen. The second user 102, for example, performs a swipe operation to change the designated display range 112 to change the screen, and searches for the moving object 115 in order to inform the first user 101 of the position of the moving object 115.

As shown in FIG. 5, the control unit 16 outputs the bird's-eye view image 122 to the touch panel display 13 by acquiring the image information for the bird's-eye view image recorded in the image information storage unit 25 or rendering the image information in the virtual space. .. The bird's-eye view image 122 is an image that gives a bird's-eye view of the entire virtual space, and displays, for example, the position of the character 116. The control unit 16 displays the designated display range 112 specified by the second user 102 in the bird's-eye view image 122 in a changed state as if it was illuminated with light, and displays the other parts in an initial state such as darkness. .. Further, the control unit 16 causes the bird's-eye view image 122 to display the moving object 115.

As shown in FIG. 6, in the second virtual space image 121 displayed on the touch panel display 13, when the second user 102 performs a touch and hold operation, the control unit 16 moves within a predetermined distance from the finger contact position. The object 115 is specified, and the movement of the specified moving object 115 is stopped. In FIG. 6, the control unit 16 stops the movement of the moving object 115 whose relative distance to each of the contact positions P1, P3, and P4 is within a predetermined distance. On the other hand, the control unit 16 does not stop the moving object 115 because the moving object 115 in the vicinity of the contact position P2 cannot be specified, and the moving object 115 on the right side in the figure whose relative distance to the contact position P5 exceeds a predetermined distance , Does not stop its movement.

As shown in FIG. 7, the control unit 16 outputs the first virtual space image 120 to the display unit 32 of the HMD 11 and changes the display range according to the posture of the first user 101. When the control unit 16 determines that the attention range 111 of the first user 101 and the designated display range 112 designated by the second user 102 overlap, it seems that the overlapping portion 125 is irradiated with light. Display in the changed state. Further, the control unit 16 displays the portion of the first virtual space image 120 other than the overlapping portion 125 in an initial state such as darkness. Further, when the moving object 115 is included in the overlapping portion 125, the control unit 16 displays only the moving object 115 included in the overlapping portion 125, and does not display the moving object 115 not included in the overlapping portion 125.

As shown in FIG. 8, when the control unit 16 inputs a signal corresponding to the on operation from the operation unit 14, the HMD 11 displays the first virtual space image 120 reflecting the second action on the first virtual space image 120. Output to unit 32. For example, the control unit 16 emits a light ray 118 from the virtual position 117 of the operation unit 14 in the virtual space along the direction of the operation unit 14 based on the signal output from the operation unit 14 and detecting the direction of the operation unit 14. Display. The control unit 16 enables the second action, sniper, when the moving object 115 is present in the direction of the ray 118. At this time, as described above, if the movement of the moving object 115 is stopped, it becomes easier to aim at the moving object 115, but it is possible to sniper the moving object 115 whose movement is not stopped. Further, for example, the control unit 16 calculates a score according to the number of times a valid action is performed within the time limit, and adds the score to the score information included in the game management information 18.

Next, with reference to FIGS. 9 and 10, the procedure of each process for providing the game to the first user 101 and the second user 102 will be described.
First, with reference to FIG. 9, a procedure for outputting the first virtual space image 120 to the HMD 11 and the main display 12 will be described. The process of outputting the first virtual space image 120 is started when a start condition such as when the operation unit 14 or another device is operated to start the game is satisfied. In parallel with this process, the control unit 16 acquires the coordinates of the designated display range 112 of the touch panel display 13 and records the coordinates as the designated display range information 20 in the storage unit 17.

The control unit 16 acquires the attention point information 19 recorded in the storage unit 17 (step S1). Then, the control unit 16 specifies the attention range 111 based on the attention point information 19. Further, the control unit 16 acquires the designated display range information 20 recorded in the storage unit 17 (step S2). Then, the control unit 16 specifies the designated display range 112 based on the designated display range information 20.

When the attention range 111 and the designated display range 112 are specified, the control unit 16 determines whether or not the attention range 111 and the designated display range 112 overlap each other (step S3). For example, the control unit 16 determines whether or not the ranges overlap based on the range in the coordinate system of the virtual space of the attention range 111 and the range in the coordinate system of the virtual space of the designated display range 112. When the control unit 16 determines that the attention range 111 and the designated display range 112 do not overlap (step S3: NO), the control unit 16 selects the image information corresponding to the attention range 111 among the image information recorded in the image information storage unit 25. The first virtual space image read out and set to the initial state is output to the display unit 32 and the main display 12 of the HMD 11 (step S8).

In step S3, when the control unit 16 determines that the attention range 111 and the designated display range 112 overlap (step S3: YES), whether or not the overlapping portion includes the moving object 115 based on the object position information 21. Is determined (step S4). When the control unit 16 determines that the moving object 115 is not included in the overlapping portion (step S4: NO), the overlapping portion is set to the changed state (step S7), and the first virtual space image is displayed in the display unit 32 of the HMD 11 and the main display. Output to 12 (step S8). As a result, the first virtual space image 120 in which the overlapping portion is changed and the moving object 115 is hidden is displayed on the display unit 32 and the main display 12 of the HMD 11.

In step S4, when the control unit 16 determines that the moving object 115 is included in the overlapping portion between the attention range 111 and the designated display range 112 (step S4: YES), the overlapping portion is set to the changed state (step S5), and the overlapping portion is changed. The moving object 115 included in the portion is set to the display state (step S6). Then, the first virtual space image 120 is output to the display unit 32 and the main display 12 of the HMD 11 (step S8). As a result, the first virtual space image 120 in which the overlapping portion is changed and the moving object 115 is displayed in the overlapping portion is displayed on the display unit 32 and the main display 12 of the HMD 11.

In this way, when the control unit 16 outputs the first virtual space image 120, it returns to step S1 and repeats steps S1 to S8 until the game ends. The control unit 16 causes the first virtual space image 120 to display the light beam 118 when the operation unit 14 is turned on. Further, when the position of the moving object 115 in the display state and the direction of the operation unit 14 match, the control unit 16 determines that the second action has been effectively performed.

Next, with reference to FIG. 10, a process of accepting a touch operation of the second user 102 and stopping the movement of the moving object 115 will be described. The process of stopping the movement of the moving object 115 is started when the start condition such as when the control unit 16 detects the touch and hold operation is satisfied. In parallel with this process, the control unit 16 acquires the coordinates of the designated display range 112 of the touch panel display 13 and records the coordinates as the designated display range information 20 in the storage unit 17.

The control unit 16 acquires the contact position information 22 recorded in the storage unit 17 and specifies the contact position (step S10). Further, the control unit 16 determines whether or not at least one moving object 115 in the vicinity of the contact position has been specified (step S11). Specifically, the control unit 16 specifies the position of the moving object 115 based on the object position information 21. Then, the relative distance between the specified contact position and the position of the moving object 115 is calculated, and it is determined whether or not there is a moving object 115 whose relative distance is within a predetermined distance.

When the control unit 16 determines that there is no moving object 115 whose relative distance from the contact position is within a predetermined distance (step S11: NO), the control unit 16 ends the process without stopping the movement of the moving object 115. Further, when the control unit 16 determines that there is at least one moving object 115 whose relative distance from the contact position is within a predetermined distance (step S11: YES), the control unit 16 stops the movement of the specified moving object. (Step S12).

Then, when the movement of the specified moving object 115 is stopped, the control unit 16 determines whether or not the finger of the second user 102 has been released from the screen of the touch panel display 13 (step S13). When the control unit 16 determines that the finger of the second user 102 has not been released (step S13: NO), the control unit 16 continues to stop the movement of the specified moving object 115 (step S12). When the control unit 16 determines that the finger of the second user 102 has been released (step S13: YES), the control unit 16 resumes the movement of the moving object 115 (step S14), and ends the process of the touch and hold operation. Then, when the start condition is satisfied, the control unit 16 repeats the processes from step S1 to step S13.

As described above, according to the first embodiment, the following effects can be obtained.
(1) In the above embodiment, the control unit 16 of the control device 15 outputs a virtual space image to both the HMD 11 and the touch panel display 13. The second user 102 designates the designated display range 112 by touch operation without wearing the HMD 11, and the control unit 16 specifies the range corresponding to the touch operation in the virtual space as the first action. Further, the first user 101 operates the moving object 115 displayed in the overlapping portion between the attention range 111 and the designated display range 112 by using the operation unit 14, and the control unit 16 responds to the operation. , Performs a second action on the moving object 115. Therefore, in order for the first user 101 to perform the second action, it is premised that the first action by the second user 102 is performed, so that the first user 101 and the second user 102 are in the virtual space. You can work together to advance the game.

(2) In the above embodiment, the control unit 16 specifies the designated display range 112 designated by the second user 102. Then, the control unit 16 reflects the designated display range 112 on the first virtual space image 120. Therefore, the second user 102 can point an arbitrary range to the first user 101 without wearing the HMD 11.

(3) In the above embodiment, when the designated display range 112 and the attention range 111 do not overlap, the control unit 16 outputs the moving object 115 to the HMD 11 in a non-display state, and outputs the designated display range 112 and the attention range to the HMD 11. When the 111 overlaps, the moving object 115 included in the overlapping portion is output to the HMD 11 as a display state. That is, an arbitrary range including the moving object 115 is specified by the second user 102, and by including the range specified by the first user 101 in the attention range 111, the moving object 115 can be visually recognized by the first user. .. Therefore, the first user 101 and the second user 102 can cooperate to advance the game.

(4) In the above embodiment, the control unit 16 sets at least one of the brightness, saturation, and hue of the designated display range 112 in the first virtual space image as the brightness, saturation, and hue of the other range. It is output to HMD 11 in a different state. According to this, the first user 101 wearing the HMD 11 can distinguish between the range specified by the touch panel display 13 and the other range. Further, since the control unit 16 displays the range designated by the second user 102 on the touch panel display 13 as if it was irradiated with light, the control unit 16 surrounds the range with a line and points to the first user 101, etc. It is possible to suppress a decrease in the immersive feeling of the first user 101.

(5) In the above embodiment, the control unit 16 enables the second action only for the moving object 115 included in the overlapping portion between the attention range 111 and the designated display range 112. Therefore, in order to effectively perform the second action, it is a prerequisite that the moving object 115 is included in the designated display range 112 designated by the second user 102, so that the first user 101 and the second user 102 cooperate. You can proceed with the game.

(6) In the above embodiment, the control unit 16 makes the horizontal viewing angle β of the second virtual space image 121 smaller than the horizontal viewing angle α of the first virtual space image 120. Therefore, since the first user 101 needs to perform an operation for the second action in a narrow range in the virtual space, the difficulty level of the second action becomes high, and the support of the second user 102 is required. .. Therefore, the first user 101 and the second user 102 can cooperate to advance the game.

(7) In the above embodiment, the control unit 16 constantly displays the moving object 115 on the touch panel display 13. As a result, in order for the first user 101 to find the moving object 115, it is premised that the second user 102 provides the position information of the moving object 115 by designating the designated display range 112. Therefore, the first user 101 and the second user 102 can cooperate to advance the game.

(8) In the above embodiment, the control unit 16 moves the moving object 115 in the first virtual space image 120 output to the HMD 11. Further, as the first action, the control unit 16 stops the movement of the moving object 115 in the first virtual space image 120 and the second virtual space image 121 output to the touch panel display 13 based on the touch operation of the touch panel display 13. .. According to this, the first user 101 can easily perform the sniper, which is the second action, on the moving object 115 whose movement has been stopped. As a result, the first user 101 and the second user 102 can cooperate to advance the game.

(9) In the above embodiment, the control unit 16 identifies a moving object 115 in the vicinity of the contact point on the screen of the touch panel display 13, and stops the movement only for the moving object 115. That is, the movement of one moving object 115 can be stopped at one contact point. The second user 102 may touch the vicinity of the moving object 115 whose movement is to be stopped, and it is easy to intuitively understand the operation for stopping the movement. Further, when stopping the movement of the plurality of moving objects 115, the same number of contact points as the number of moving objects 115 are required. Therefore, the difficulty of stopping the moving object can be increased according to the number of the moving objects 115 to be stopped.

(10) In the above embodiment, the control unit 16 outputs the bird's-eye view image 122, which is a bird's-eye view of the virtual space, to the touch panel display 13 with the moving object 115 in the display state. Therefore, the second user 102 can grasp the position and state of the moving object 115 included in the virtual space. Therefore, it is easy to include the moving object 115, which is the target of the sniper, which is the second action, in the designated display range 112. Further, the second user 102 can easily grasp the positional relationship between the designated display range 112 and the moving object 115.

(Second Embodiment)
Next, a second embodiment embodying the present invention will be described. Since the second embodiment has a configuration in which the first action in the first embodiment is modified, the same parts are designated by the same reference numerals and detailed description thereof will be omitted.

In the first embodiment, the first action is specified as the designated display range 112, but in the present embodiment, the first action is the selection of the moving object 115 based on the operation of the second user 102. In the present embodiment, the initial state of the first virtual space image 120 output to the HMD 11 may be at least a non-display state of the moving object 115. That is, the first virtual space image 120 of the present embodiment does not have to be in a low-brightness display state such as “darkness”.

As shown in FIG. 11, the moving object 115 is displayed on the second virtual space image 121 displayed on the touch panel display 13. The second user 102 designates the moving object 115 by a touch operation. Although FIG. 11 shows an example in which there is only one contact position P10, there may be a plurality of contact positions.

The control unit 16 acquires the coordinates of the contact position P10 from the touch panel display 13, and determines whether or not there is a moving object 115 whose relative distance to the coordinates of the contact position P10 is equal to or less than a predetermined distance based on the object position information 21. do.

When the control unit 16 determines that there is a moving object 115 whose relative distance to the contact position P10 is equal to or less than a predetermined distance, the moving object 115 is set as a display state on the first virtual space image 120, and the display unit 32 of the HMD 11 and the control unit 16 Output to the main display 12. In this embodiment, unlike the first embodiment, the process of stopping the movement of the moving object 115 in the vicinity of the contact position P10 is not performed.

The control unit 16 continues to display the moving object 115 on the first virtual space image while the relative distance between the contact position P10 and the moving object 115 is maintained at a predetermined distance or less. On the other hand, even if the second user 102 continues the touch operation, if the relative distance between the contact position P10 and the moving object 115 exceeds a predetermined distance as the moving object 115 moves, the control unit 16 moves the moving object 115. Hide it.

As described above, in the present embodiment, since the moving object 115 selected by the second user 102 is displayed on the first virtual space image 120, the work of designating the moving object 115 becomes a finer work. As a result, close cooperation between the first user 101 and the second user 102 is required, and the difficulty level of the game can be made suitable to enhance the interest.

As described above, according to the second embodiment, the following effects can be obtained in addition to the effect of (1) described in the first embodiment.
(11) In the above embodiment, the control unit 16 of the control device 15 specifies the moving object 115 selected by the second user 102. Then, the control unit 16 causes the first virtual space image 120 to display the moving object 115 selected by the second user 102. Therefore, the second user 102 can point the selected moving object 115 to the first user 101 without wearing the HMD 11.

(Third Embodiment)
Next, a third embodiment embodying the present invention will be described. Since the third embodiment has a configuration in which the designated display range in the first embodiment is variable, the same reference numerals are given to the same parts and detailed description thereof will be omitted.

In the first embodiment, the entire range displayed on the touch panel display 13 by the second user 102 by the touch operation is set as the designated display range 112, but in the present embodiment, the designated display range 112 is set by the touch operation of the second user 102. The size and position of the can be changed.

As shown in FIG. 12, the second user 102 performs a pinch-in operation of tracing the screen so as to bring two fingers closer to the designated display range 112, a pinch-out operation of tracing the screen so as to move two fingers away, and the like. Change its size and position by doing. The designated display range 112 is the entire screen in the initial state, and the control unit 16 reduces the designated display range 112 based on the trajectory of the pinch-in operation when the pinch-in operation is performed, and performs the pinch-out operation when the pinch-out operation is performed. The designated display range 112 reduced based on the locus is enlarged. Further, when the two fingers are moved while being in contact with the screen after performing the pinch-in operation or the pinch-out operation, the control unit 16 maintains the size of the designated display range 112 based on the input locus of the operation. As it is, the designated display range 112 is moved within the screen. The light source position (origin) may be moved with the movement of the designated display range 112.

Further, by reducing the designated display range 112, a special effect on the moving object 115 may be generated. For example, the moving object 115 included in the designated display range 112 may not be moved to the outside of the designated display range 112 when the reduction of the designated display range 112 is started. As a result, by reducing the designated display range 112, the moving objects 115 included in the designated display range 112 can be collected in the reduced designated display range 112. When the moving objects 115 are collected in such a narrow range, the first user 101 can easily shoot at many moving objects 115 at once.

When the pinch-in operation is performed, the control unit 16 reduces the designated display range 112 according to the length and angle of the input locus based on the input locus of the touch operation. Further, the control unit 16 outputs to the touch panel display 13 a second virtual space image 121 in which the reduced brightness, saturation, and hue of the designated display range 112 are changed to different states from the other ranges. At this time, the second virtual space image 121 also displays a portion in the initial state, for example, a portion having low lightness or saturation expressing "darkness". At this time, the control unit 16 may display or hide the moving object 115 included in the initial state portion.

Further, the control unit 16 determines whether or not there is an overlapping portion between the attention range 111 of the first user 101 and the designated display range 112, and if there is an overlapping portion, the overlapping portion is changed to a changed state, and other than that. The first virtual space image 120 with the portion of is in the initial state is output to the display unit 32 and the main display 12 of the HMD 11.

The second user 102 touches the touch panel to perform a pinch-out operation to expand the designated display range 112 when the moving object 115 included in the designated display range 112 disappears or decreases due to effective sniping by the first user 101 or the like. This is done on the screen of the display 13. When the pinch-out operation is performed, the control unit 16 inputs the input locus of the touch operation and expands the designated display range 112 according to the length and angle of the input locus. Further, the control unit 16 outputs the second virtual space image 121 with the enlarged designated display range 112 in the changed state and the other parts in the initial state to the touch panel display 13. As a result, the designated display range 112 includes the new moving object 115.

As described above, according to the third embodiment, the following effects can be obtained in addition to the effects (1) to (10) described in the first embodiment.
(12) In the above embodiment, the control unit 16 changes the size and position of the designated display range 112 based on the touch operation on the screen of the touch panel display 13, so that, for example, only a part of the screen can be pointed to.

(Fourth Embodiment)
Next, a fourth embodiment embodying the present invention will be described. Since the fourth embodiment is configured to hide the moving object on the touch panel display and display the moving object on the second virtual space image according to the operation of the operation unit, the same portion as that of the first embodiment. The same reference numerals are given to the above, and detailed description thereof will be omitted.

In the first embodiment, the first action is to specify the designated display range 112 based on the operation of the second user 102, and the second action is to sniper the moving object 115 based on the operation of the first user. In the present embodiment, the first action is the selection of the moving object 115 by the first user 101, and the second action is the attack on the moving object 115 based on the touch operation of the touch panel display 13 of the second user 102.

As shown in FIG. 13, the control unit 16 outputs the first virtual space image 120 in which the moving object 115 is always in the display state to the display unit 32 and the main display 12 of the HMD 11. The operation unit 14 operated by the first user 101 is a controller capable of changing the direction, on operation, and off operation, or a glove type controller. The first user 101 operates the operation unit 14 to select a moving object 115 in the virtual space. Specifically, the control unit 16 changes the direction of the light source 130 such as a flashlight displayed on the first virtual space image 120 by the operation of the operation unit 14, and moves by irradiating the moving object 115 with light. Select object 115. Alternatively, the moving object 115 is captured by using a tool such as a net or a container in the virtual space.

The control unit 16 inputs the signal output from the operation unit 14, and determines whether or not the moving object 115 has been selected based on the object position information 21. When the control unit 16 determines that the moving object 115 is selected, the control unit 16 outputs the second virtual space image 121 with the selected moving object 115 in the display state to the touch panel display 13.

In the second virtual space image 121 output to the touch panel display 13, the moving object 115 is hidden in the initial state. When the moving object 115 selected by the first user 101 is displayed on the second virtual space image 121, the second user touch-operates the moved object 115 selected in the displayed state to perform an attack. When the control unit 16 determines that the attack on the moving object 115 has been performed based on the signal input from the touch panel display 13, the attack on the moving object 115 is reflected in the first virtual space image 120, and the HMD 11 and the main display 12 are used. Output to.

As described above, according to the fourth embodiment, the following effects can be obtained.
(13) In the above embodiment, the first user 101 selects the moving object 115 and puts the selected moving object 115 in the display state on the second virtual space image 121. The second user 102 can attack the moving object 115 displayed in the virtual space without wearing the HMD 11. Therefore, the first user 101 and the second user 102 can cooperate in advancing the game.

(Other embodiments)
It should be noted that each of the above embodiments can be appropriately modified and implemented as follows.
In each of the above embodiments, the control unit 16 sets the position of the character 116 corresponding to the second user 102 to the position above the vertical direction of the first user 101. Instead of this, the position of the character 116 may be changed, for example, based on a touch operation on the screen of the touch panel display 13 by the second user 102.

-In each of the above embodiments, the horizontal viewing angle β of the second virtual space image is made smaller than the horizontal viewing angle α of the first virtual space image. Instead, the horizontal viewing angle β of the second virtual space image is set to be the same as the horizontal viewing angle α of the first virtual space image 120, or is made larger than the horizontal viewing angle α of the first virtual space image. May be good. By doing so, the designated display range 112 can be easily found, which is effective when, for example, it is desired to reduce the difficulty of finding an object. Similarly, the vertical viewing angle of the second virtual space image may be smaller than the vertical viewing angle of the second virtual space image. Alternatively, it may be equal to or larger than the vertical viewing angle of the second virtual space image.

-In each of the above embodiments, the control unit 16 causes the main display 12 to display the attention range of the first user 101 as a first virtual space image. Alternatively or additionally, the control unit 16 may display, for example, the attention range 111 of the first user 101 on the second virtual space image 121. The attention range 111 may be indicated by surrounding it with a line, or at least one of lightness, saturation, and hue may be displayed differently from the background of the second virtual space image 121. In this case, it is possible to omit the main display 12 visually recognized by the second user 102.

-In each of the above embodiments, the control unit 16 outputs a bird's-eye view image 122, which is a bird's-eye view of the virtual space, to the touch panel display 13. However, if it is easy to grasp the position of the object in the virtual space, the display of the bird's-eye view image 122 may be omitted.

In each of the above embodiments, the control unit 16 displays at least one of the brightness, saturation, and hue of the designated display range 112 differently from the brightness, saturation, and hue of the portion other than the designated display range 112. .. In addition to this, as the first action, the designated display range 112 may be displayed by being surrounded by a line displayed on the image, or a mark such as an arrow may be displayed.

In each of the above embodiments, the control unit 16 makes at least one of the brightness, saturation, and hue of the designated display range 112 different from the brightness, saturation, and hue of the portion other than the designated display range 112, and is the first virtual. It was displayed on the spatial image and the second virtual spatial image. In addition to this, the control unit 16 may change and display at least one of the lightness, saturation, and hue of the designated display range 112 with the passage of time. Specifically, when the control unit 16 displays the designated display range 112 with the brightness, saturation, and hue as if it is illuminated by light, the designated display range 112 is darkened with the passage of time. Alternatively, at least one of lightness, saturation and hue may be changed so that light of a color different from the light of the initial state is emitted.

In each of the above embodiments, the control unit 16 stops the movement of the moving object 115 as the first action based on the contact position on the touch panel display 13 by the second user 102. Alternatively or additionally, the control unit 16 may stop the movement of the moving object 115 based on the item selection operation displayed on the screen of the touch panel display 13. For example, when a special item displayed on the screen of the touch panel display 13 is selected, the movement of all the moving objects 115 displayed on the screen of the touch panel display 13 may be stopped. Further, when the item is dragged and dropped to the vicinity of the moving object 115, the movement of the moving object 115 may be stopped.

In each of the above embodiments, the control unit 16 stops the movement of the moving object 115 as the first action based on the contact position on the touch panel display 13 by the second user 102. Alternatively or additionally, the movement of the moving object included in the predetermined range corresponding to one contact position may be stopped. The predetermined range corresponding to one contact position is, for example, a circular range centered on the contact position, a rectangular shape having the contact position as a starting point (corner), a fan shape, or the like. Alternatively, a predetermined range may be selected by swiping or tapping with two fingers.

-In each of the above embodiments, the second action is performed on the moving object 115 that is constantly moving. Alternatively or additionally, a second action may be performed on an object placed at a predetermined fixed position in the virtual space. Examples of such an object are a box, a door, a musical instrument, and the like arranged in a virtual space. In this case, the second action is, for example, an action such as opening the lid of a box, opening a door, or playing a musical instrument.

-In each of the above embodiments, the moving object 115 that is not included in the overlapping portion of the attention range 111 and the designated display range 112 cannot perform the second action, or is invalid even if the second action is performed. And said. Alternatively, the second action may be performed on an object that is not included in the overlapping portion. In this case, for example, the objects included in the overlapping portion may be clearly displayed so as to be easy to aim, and the objects not included in the overlapping portion may be displayed unclearly so as to be easily aimed. Further, when a valid action is performed on an object not included in the overlapping portion, the score may be higher than when a valid action is performed on the object included in the overlapping portion.

In each of the above embodiments, the predetermined object (moving object 115) is hidden in the first virtual space image output to the display unit 32 of the HMD 11 and the main display 12. Instead of this, the object to be the target of the second action may be displayed in the first virtual space image. In this case, for example, the display mode of the overlapping portion of the attention range 111 and the designated display range 112 is displayed in a mode different from the other portions, and the second action is enabled only for the moving object 115 included in the overlapping portion. You may do it. Even in this way, since the second action is effective only for the predetermined object included in the overlapping portion, the first user 101 and the second user 102 can cooperate to advance the game.

Alternatively, the display mode of the object or the virtual space image including the object may be different between the first virtual space image and the second virtual space image. For example, in the second virtual space image, it is possible to enlarge and scale, or the object can be displayed at any angle, but in the first virtual space image, it is impossible to enlarge and scale, or at a certain angle. Only objects can be displayed. Alternatively, the second virtual space image may be a bird's-eye view image of the virtual space including the object, and the bird's-eye view image may not be displayed in the first virtual space image. Alternatively, the second virtual space image can perform predetermined image processing such as transparency and edge extraction, but the first virtual space image may not be able to perform predetermined image processing.

14 and 15 are examples in which the second virtual space image is a bird's-eye view image of a virtual space including an object, and the first virtual space image is an image other than the bird's-eye view image. For example, the game played by the first user 101 and the second user 102 is a puzzle game in which three-dimensional figures are combined. As shown in FIG. 14, the control unit 16 outputs the first virtual space image 120, which is an image of the virtual space including the three-dimensional figure 150, which is an object, to the HMD 11 or the like according to the posture of the first user 101. The solid figure 150 is not particularly limited to a triangular prism, a cone, a spherical shape, or the like, in addition to the cube and the rectangular parallelepiped. The first user 101 operates the operation unit 14 to combine three-dimensional figures. When a predetermined number or more of the three-dimensional figures 150 are combined, it is difficult to include the entire pattern formed by the combination of the three-dimensional figures 150 in the viewing angle of the first virtual space image 120. Therefore, when forming the target pattern of the first user 101, it is difficult to determine at least one of the position where the three-dimensional figure 150 should be arranged and the type of the three-dimensional figure 150 to be arranged.

As shown in FIG. 15, the control unit 16 outputs a bird's-eye view image 122, which is a bird's-eye view of a pattern formed by combining three-dimensional figures 150, to the touch panel display 13. Since the second user 102 can visually recognize the entire pattern on the bird's-eye view image 122, it is possible to easily determine which three-dimensional figure 150 should be arranged at which position.

Further, as shown in FIG. 15, for example, when the second user 102 touches a predetermined position on the screen of the touch panel display 13, the control unit 16 displays a three-dimensional figure on the first virtual space image 120 as shown in FIG. The position where the 150 should be placed is displayed by the designated position mark 151 (first action). Alternatively, in addition to the designated position mark 151, the position where the solid figure 150 should be arranged may be displayed in a mode different from the display mode of the other parts. The control unit 16 arranges the three-dimensional figure 150 at a position based on the operation of the operation unit 14 of the first user 101 (second action).

Further, in a puzzle game in which an action is performed based on the arrangement of the types of the three-dimensional figure 150, for example, the second user 102 selects an arbitrary three-dimensional figure 150 from the display area 153 for selecting the type of the three-dimensional figure 150, and the second user 102 selects an arbitrary three-dimensional figure 150. Touch the position where the solid figure 150 should be placed (see FIG. 15). As shown in FIG. 14, the control unit 16 displays the type of the three-dimensional figure 150 to be arranged by the type identification mark 152 (first action). Further, the position where the solid figure 150 should be arranged may be displayed by the designated position mark 151 or the like. The first user 101 operates the operation unit 14 to arrange the three-dimensional figure 150 pointed to by the type identification mark 152 at the position pointed to by the designated position mark 151.

-In each of the above embodiments, as the first action, the range in the virtual space is designated as the designated display range 112 based on the touch operation of the touch panel display 13. Instead of this, as a first action, a direction or an arbitrary position in the virtual space may be pointed out based on the touch operation of the touch panel display 13.

FIG. 16 shows an example of a screen output in the maze game. As shown in FIG. 16A, the control unit 16 causes the display unit 32 or the like of the HMD 11 to display an image of the three-dimensional maze 160 provided in the virtual space as the first virtual space image 120. The control unit 16 outputs the first virtual space image 120 according to the posture of the user, and the first virtual so as to proceed in the three-dimensional maze 160 based on the operation of the operation unit 14 or the user position detected by the sensor. The spatial image 120 is changed. As shown in FIG. 16B, the control unit 16 causes the touch panel display 13 to display a bird's-eye view image of the three-dimensional maze 160 as the second virtual space image 121. The control unit 16 causes the second virtual space image 121 to display the position 161 of the first user 101. While visually recognizing the second virtual space image 121, the second user 102 traces, for example, the direction in which the first user 101 should go with a finger in accordance with the progress of the first user 101. The control unit 16 acquires an input locus corresponding to the touch operation from the touch panel display 13, and displays the direction instructed by the first virtual space image 120 along the input locus (first action). For example, the direction indicated by the second user 102 may be displayed as light having different brightness, saturation, and hue with respect to the background on the first virtual space image 120, or may be displayed as an index such as an arrow. You may. The first user 101 advances in the three-dimensional maze 160 by operating the operation unit 14 or the like with reference to the direction indicated on the first virtual space image 120.

In each of the above embodiments, in addition to the HMD 11 that detects the position and orientation of the head of the first user 101, a position detection sensor that detects the position of the first user 101 in the real space may be provided. The control unit 16 detects not only the position and orientation of the head of the first user 101 but also the position where the first user 101 stands by the position detection sensor, and the image information of the virtual space according to the position where the first user 101 stands. Is rendered to display the first virtual space image. By doing so, since the first virtual space image is changed with the movement of the first user 101, the immersive feeling can be further enhanced.

-In each of the above embodiments, the game of outputting the image of the virtual space to the HMD 11 and the touch panel display 13 is embodied as a game of sniping a predetermined object. Instead, it may be embodied in other games such as exploration games, card games, racing games, fighting games, simulation games, and role-playing games. Further, the action on the object is not limited to "sniper", and may be other actions such as transforming, throwing, hitting, scattering, dividing, rotating, and moving.

-In each of the above embodiments, an application that outputs an image of a virtual space to the HMD 11 and the touch panel display 13 is embodied in a game. Instead of this, an application that outputs an image of a virtual space to the HMD 11 and a touch panel display 13 is used as a non-game application such as a simulation application and a training application used in fields other than games such as medical fields and education fields. May be good.

Next, the technical ideas that can be grasped from the above-described embodiment and other examples will be added below together with their effects.
[A] In an application control program that outputs an image of a virtual space using a wearable display worn by a first user and a control unit that outputs an image to a display used by the second user, the control unit is used as the first user. The image of the virtual space is output to the mounted display, the display range of the virtual space is determined based on the input operation of the second user, and the determined display is displayed on the display used by the second user. An image of the virtual space in the range is output, the operation of the second user with respect to the virtual space is received, the first action is performed in the virtual space, and the first action is reflected on the mounted display. It is characterized in that it outputs an image of the virtual space and functions as a means for performing a second action in the virtual space in which the first action is reflected in response to an operation by the operation unit by the first user. The application control program to be.
[B] The application control program according to [a], wherein the means for performing the first action is, as the first action, controlling the movement of an object based on an input operation of the second user.
[C] The application control program according to [a] or [b], wherein the means for performing the first action selects an object based on an input operation of the second user as the first action.
[D] The means for further functioning the control unit as a means for moving a moving object that is the target of the second action in the image of the virtual space output to the mounted display, and performing the first action. As the first action, the means for stopping the movement of the moving object based on the operation of the display and outputting the image of the virtual space reflecting the first action is an image to be output to the mounted display. The application control program according to any one of [a] to [c], which stops the movement of the moving object in the above.
[E] The application according to [d], wherein the means for performing the first action identifies the moving object in the vicinity of the contact point on the screen of the display and stops the movement only for the moving object. Control program.
[F] When the means for outputting the image of the virtual space reflecting the first action overlaps the determined display range and the attention range according to the posture of the first user, the means may overlap. A predetermined object included in the overlapping portion is displayed, and if there is a predetermined object not included in the overlapping portion, the predetermined object is hidden and output to the mounted display [a] to The application control program according to any one of [e].
[G] The means for outputting the image of the virtual space reflecting the first action is to set at least one of the brightness, saturation and hue of the determined display range, and the brightness and saturation of the other range. The application control program according to [f], which outputs to the mounted display in a state different from the hue.
[H] The means for performing the second action is effective only for a predetermined object included in the overlapping portion between the determined display range and the attention range according to the posture of the first user. The application control program according to [f] or [g].
[I] The means for outputting an image of the virtual space to the display is described in any one of [f] to [h], which outputs a predetermined object to be the target of the second action in a display state. Application control program.
[J] The means for outputting the image of the virtual space reflecting the first action has the determined display range rather than the horizontal viewing angle of the image of the virtual space output on the mounted display. The application control program according to any one of [a] to [i], which reduces the viewing angle in the horizontal direction.
[K] The means for outputting the image of the virtual space to the display further functions as a means for outputting an image in which the predetermined object to be the target of the second action is displayed in a bird's-eye view of the virtual space. The application control program according to any one of claims [a] to [j].
[L] In an application control method for outputting an image of a virtual space using a wearable display worn by a first user and a control unit that outputs an image to a display used by the second user, the control unit is the first user. The image of the virtual space is output to the mounted display, the display range of the virtual space is determined based on the input operation of the second user, and the determined display is displayed on the display used for the second user. An image of the virtual space in the range is output, the operation of the second user with respect to the virtual space is received, the first action is performed in the virtual space, and the first action is reflected on the mounted display. An application control characterized in that an image of the virtual space is output and a second action is performed in the virtual space in which the first action is reflected in response to an operation in the operation unit by the first user. Method.
[M] In an application control system that outputs an image of a virtual space including a wearable display worn by a first user and a control unit that outputs an image to a display used by the second user, the control unit is the first user. The image of the virtual space is output to the mounted display, the display range of the virtual space is determined based on the input operation of the second user, and the determined display is displayed on the display used by the second user. An image of the virtual space in the range is output, the operation of the second user with respect to the virtual space is received, the first action is performed in the virtual space, and the first action is reflected on the mounted display. An application control characterized in that an image of the virtual space is output and a second action is performed in the virtual space in which the first action is reflected in response to an operation in the operation unit by the first user. system.

11 ... Head-mounted display (HMD) as a mounted display, 12 ... Main display, 13 ... Touch panel display, 14 ... Operation unit, 15 ... Control device, 16 ... Control unit, 17 ... Storage unit, 18 ... Game management information, 19 ... Attention point information, 20 ... Designated display range information, 21 ... Object position information, 22 ... Contact position information, 24 ... Gaze position specifying unit, 25 ... Image information storage unit, 30 ... HMD control unit, 31 ... Tracking sensor, 32 ... display unit, 101 ... first user, 102 ... second user, 110 ... virtual space, 111 ... attention range, 112 ... designated display range, 114 ... attention point, 115 ... moving object, 116 ... character, 118 ... ray, 120 ... 1st virtual space image, 121 ... 2nd virtual space image, 122 ... bird's-eye view image, 125 ... overlapping part.

Claims (4)

It is an application control program that outputs an image of virtual space to the wearable display worn by the first user.
For one or more computers
Depending on the posture of the first user, the mounted display may include objects classified into a plurality of types and arranged at positions to be arranged by the first user, and a pattern in which a plurality of the objects are combined can be formed. The step of outputting the first image of the virtual space and
The display used by the second user is made to output a second image in which the virtual space including the object is overlooked with a viewing angle wider than the viewing angle of the first image, and a display area for selecting the type of the object is provided. Steps to output and
A step of accepting an operation of the second user to select an object of a predetermined type from the types of the objects displayed in the display area, and
A step of accepting the operation of the second user to specify the placement position of the selected object of the predetermined type, and
A step of causing the wearing display to output an image of the virtual space in which the arrangement position designated by the second user and the selected object of the predetermined type are identifiablely displayed is executed .
In the step of outputting the first image, the second image is not output to the mounted display.
An application control program that does not output the first image to the display in the step of outputting the second image . In the virtual space in which the arrangement position designated by the second user and the selected object of the predetermined type are identifiablely displayed, the first user is based on the operation by the operation unit by the first user. The application control program according to claim 1, further executing a step of arranging the selected object at a position designated by the first user. It is an application control method that outputs an image of virtual space to a wearable display worn by a first user.
One or more computers
Depending on the posture of the first user, the mounted display may include objects classified into a plurality of types and arranged at positions to be arranged by the first user, and a pattern in which a plurality of the objects are combined can be formed. The step of outputting the first image of the virtual space and
The display used by the second user is made to output a second image in which the virtual space including the object is overlooked with a viewing angle wider than the viewing angle of the first image, and a display area for selecting the type of the object is provided. Steps to output and
A step of accepting an operation of the second user to select an object of a predetermined type from the types of the objects displayed in the display area, and
A step of accepting the operation of the second user to specify the placement position of the selected object of the predetermined type, and
A step of outputting the image of the virtual space in which the arrangement position designated by the second user and the selected object of the predetermined type are identifiablely displayed on the mounted display is performed.
In the step of outputting the first image, the second image is not output to the mounted display.
An application control method in which the first image is not output to the display in the step of outputting the second image . It is an application control system that has a control unit that outputs an image to a wearable display worn by a first user and outputs an image of a virtual space.
The control unit
Depending on the posture of the first user, the mounted display may include objects classified into a plurality of types and arranged at positions to be arranged by the first user, and a pattern in which a plurality of the objects are combined can be formed. The step of outputting the first image of the virtual space and
The display used by the second user is made to output a second image in which the virtual space including the object is overlooked with a viewing angle wider than the viewing angle of the first image, and a display area for selecting the type of the object is provided. Steps to output and
A step of accepting an operation of the second user to select an object of a predetermined type from the types of the objects displayed in the display area, and
A step of accepting the operation of the second user to specify the placement position of the selected object of the predetermined type, and
A step of outputting the image of the virtual space in which the arrangement position designated by the second user and the selected object of the predetermined type are identifiablely displayed on the mounted display is performed.
In the step of outputting the first image, the second image is not output to the mounted display.
An application control system that does not output the first image to the display in the step of outputting the second image .

Images