JP3413128B2 - Mixed reality presentation method - Google Patents

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a real space and a virtual sky.
The mixed reality space represented by combining
The mixed reality presentation method presented to the user
Relates to a mixed reality presentation method capable of easily indicating an arbitrary position in a mixed reality space .

[0002]

[0003]

In recent years, mixed reality for the purpose of seamless coupling of real space and virtual space (Mixed Reality, M
Research on R) has become popular. MR has been attracting attention as a technique for enhancing VR for the purpose of coexistence of the world of virtual reality (VR), which was conventionally available only in a situation separated from the physical space, and the physical space.

A typical device for realizing mixed reality is a head mounted display (HMD). That is, the mixed reality is realized by synthesizing and displaying the real space and the virtual space on the HMD. As an MR method using the HMD, a semi-transmissive (see-through type) head mounted display (H
There are an optical see-through method of superimposing an image of CG or the like on the MD) and a video see-through method of displaying an image on the HMD after the image of the CG or the like is combined with image data taken by a video camera mounted on the HMD.

[0005] The MR is used as a medical assistance to present to a doctor as if the patient's internal condition is seen through, and
It is expected that new fields will be qualitatively completely different from the VRs used up to now, such as the use of work assistance in which the assembly procedure of products is displayed on the actual product in a factory.

(Game Application) On the other hand, as an example in which a plurality of persons share the same space by using MR and perform operations in cooperation, there is a game as disclosed in, for example, Japanese Patent Laid-Open No. 11-84307. Japanese Patent Laid-Open No. 11-84307 discloses an air hockey device as shown in FIG. 15 as a game device using MR.

In an air hockey game, usually two people face each other across a table, and a puck floated by compressed air supplied from minute holes provided on the table board is hit back by a mallet manually operated by the player, It is a game that puts packs in the goal of.

In the air hockey game to which MR is applied, the puck is superimposed on the real environment as a virtual object and displayed on the HMD of the player to play the air hockey game.

(Structure of Game Device) FIG. 15 is a side view of the MR air hockey game device. A mixed reality air hockey game is a control box (260L, 260L) that two opponents 2000, 3000 use as a mallet with a table 1000 in between.
R) face each other. Two opponents 2000, 30
00 is a head mount display (hereinafter HMD
210L and 210R are attached.

Control box (260L, 260
R) has an infrared ray emitter at its tip, and is a CCD camera 230 provided on the table for detecting infrared rays.
By processing the image acquired by, the position of the two-dimensional plane can be known. If the shape and color of the control box have characteristics, detection by pattern recognition using those characteristics is also possible.

The HMD 210 is, for example, a see-through type. Both opponents 2000, 3000, HMD210L,
Even with the 210R attached, the surface of the table 1000 can be directly observed. A two-dimensional virtual image is input to the HMD 210 from an image processing system (not shown).
Therefore, the opponents 2000 and 3000 will see the two-dimensional image displayed on the display screen of the HMD 210, overlapping with the real space observed through the optical system (not shown in FIG. 15) of the HMD 210.

FIG. 16 shows the mixed reality space that the left player 2000 observes through his HMD 210L.
In FIG. 16, what is recognized as the real space is the player 3000 and the table 1000 who compete with each other, and one's own body in the field of view, especially the hand holding the control box. On the other hand, the opponent goal 1200R and the puck 1500 are displayed in the virtual space.

Then, the two players move the actual control box held by each player so as to hit a virtual pack 1500 displayed by the image processing system, and strike the virtual packs. The position of the virtual puck is calculated by the image processing system and displayed on the HMDs of both players so that the puck can be observed to move while being repelled by the mallet and the surrounding wall surface on the table.

The HMD 210 worn by the player is, for example, the HMD main body disclosed in Japanese Unexamined Patent Publication No. 7-333551, to which a magnetic sensor 220 is attached via a column 221. The configuration in which the magnetic sensor and / or the camera is attached to the HMD is not limited to the video see-through HMD, and even an optical see-through HMD can be adopted for the purpose of accurately detecting the head position and posture. .

Each HMD 210 is fixed to the player's head by a band (not shown). Each player's head has a magnetic sensor (220L, 220R) and a CC.
The D cameras 240 (240L, 240R) are fixed. The field of view of the camera 240 is set in front of the player. In the case of the air hockey game, since the upper surface of the table 1000 is to be seen, the camera 240 also captures an image of the surface of the table 1000. The magnetic sensor 220 (220L, 220R) senses a change in the AC magnetic field generated by the AC magnetic field generation source 250.

When the player turns diagonally downward to see the surface of the table 1000, the view through the HMD 210 shows the surface of the table 1000 and the virtual puck 1 described above.
500, real mallet (control box 26
0), virtual goal 1200 (1200L, 1200
R) is visible. When the player moves his head horizontally within a horizontal two-dimensional plane, or makes a tilting motion, a yaw motion, or a rolling motion, the change is first detected by the magnetic sensor 220.
This is observed as a change in the image captured by the CCD camera 240 as the posture of the head changes.

Then, the detected position change of the head is used to change the display position of the image data of the virtual space displayed on the HMD. Place the marker on the table and use the camera 2
It is also possible to correct the signal indicating the posture change from the marker image captured by 40.

(Instruction Display) On the other hand, when it is desired to indicate a predetermined position or an object in the mixed reality space,
An image formed by combining surfaces, such as a pointing stick used in the real space or a three-dimensional image simulating a simple stick, is used as the pointing stick in the mixed reality space.

[0019]

In the MR air hockey game described above, all the players wear the HMD, and the direction of the line of sight of the opponent player cannot be known. Also, since the mounted device can be seen as it is, it is not realistic.

Further, the displayed virtual space image is only the mallet, the puck, and the goal, and the display contents (display effect) and the image hardly change.

Further, since the information such as the name of the opponent player, the past record, and the current score condition is not displayed, it is difficult to grasp the situation of the game and the level of the opponent player. It was

In addition, there is no suggestion of processing when three or more players share the same mixed reality space.

Further, when pointing to a predetermined position or object in the mixed reality space, since an image simulating a solid object such as a stick is used, it is difficult to grasp the perspective. Furthermore, if a plurality of pointing rods are present in the space, it tends to be complicated, and since they are composed of planes, there is a problem that the overlapping portion is hindered from being visually recognized.

[0024]

[0025]

[0026]

[0027]

The present invention allows the user to save in a mixed reality space.
The perspective image so that you can easily
Is displayed in an easy-to-grasp manner and instructions are displayed
Make sure that the displayed contents are not obstructed.
aimed to.

[0029]

That is, the gist of the present invention is expressed by synthesizing a real space and a virtual space.
Mixed Reality Presentation Method for Presenting Mixed Reality Space to Players
It is a method of acquiring the position / orientation information of the player
Input the image of the space and according to the position / orientation information of the player
Generated virtual image using the model information of the player,
Mixed Reality Presentation Method Combining Images in Real Space and Virtual Images
Modular, operating section operated by player, regular n-square
A mixed reality space containing n parallel lines passing through the vertices of the shape
An instruction image to point to any place in the
Generated as a virtual image according to the position / posture information,
A mixed reality presentation method characterized by synthesizing the image with the image .

[0030]

[0031]

Another subject of the present invention is the composite of the present invention.
The present invention resides in a computer-readable storage medium that stores the reality presentation method as a program executable by the computer .

[0033]

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below with reference to the drawings. Although the present embodiment is a game played by three players sharing the same virtual space, the mixed reality apparatus according to the present invention can be applied to any other application.

FIG. 1 is a diagram showing the configuration of the mixed reality apparatus of the present invention. FIG. 1 shows the configuration when there are three players. The mixed reality apparatus includes player processing units 100 to 300 provided for each player and a control unit 400 connected to each player processing unit.
As the number of players increases or decreases, the number of player processing units connected to the control unit 400 also increases or decreases.

The player processing units 100 to 300 have the same structure, and input / output units (102, 103, 1) such as sensors and display devices mounted on the player's body.
05, 1011, 1021, 1022), a circuit part (101) for generating an image to be displayed on the display device based on the signal processing collected from the sensor and the result of this signal processing and the information of another player supplied from the control unit. , 104, 10
6, 107).

The configuration of the player processing section will be described below with reference to FIG. 1 and FIG. 2 showing the types of input / output devices and the mounting locations of the players in this embodiment. As shown in FIG. 2, in the present embodiment, each player has a head-mounted image display device (hereinafter, referred to as an HMD) 1 which is a display device for combining and displaying a real space and a virtual space on the head.
Wear 05. In the present invention, the HMD may be either a video see-through type or an optical see-through type, but in the following description, a case where a video see-through HMD is used will be described as an example.

Two small video cameras 103 are provided near the eyes of the HMD 105. An image captured by the video camera 103 from a viewpoint substantially the same as the viewpoint of the player is supplied to an image combining unit 106, which will be described later, via the image input unit 104, and is superimposed on the virtual space image and displayed on the HMD 105 of the player. To be done.

Further, a head position / orientation sensor 1011 including, for example, a magnetic sensor is mounted on the player's head. The head position sensor 1011 is, for example, an HMD.
It can be attached using 105. The output signal of the head position sensor 1011 is processed by the head position / orientation measuring unit 101, and the head position and orientation of the player (tilt direction and amount).
Get information on.

On the other hand, the interactive operation input device 102 is mounted on the player's arm. Interactive operation input device 102
Has a position / orientation sensor 1021 for detecting the position and orientation of the mounted part, and a switch (trigger) 1022 that can be turned on / off by the player, so that the player inputs a command by performing a predetermined operation. Device.

In the following description, a plurality of players (three people in this embodiment) share the same mixed reality space, and destroy the enemy while avoiding the attack from the enemy appearing in the mixed reality space. Taking a case where the following commands can be input using the interactive operation input device 102, as an example, in a game in which the number or points of enemies destroyed by the time or the damage caused by the enemy's attack reaches a predetermined amount is competed. explain.

Command 1 (aiming command) A command for displaying an aiming line indicating the aiming position in the virtual space. With the back of the hand up, enter by placing the wrist above the elbow.・ Command 2 (shooting command) Command to shoot at the aiming position indicated by the line of sight. When the aiming line is displayed by the aiming command, the arm (from the elbow to the palm) is reciprocally moved back and forth with a certain acceleration or more to input. -Command 3 (defense command) A command that defends the opponent's attack. With the back of the hand facing the other party, point by pointing your finger upwards. -Command 4 (reset command) A command to be input when the aiming command is input again after the shooting command is input. Input by making the arm hang down.

That is, as a player's operation in the game of this embodiment, normally, command input is repeated in a cycle of aiming command → shooting command → reset command → aiming command, and a defense command is issued as necessary in this cycle. Will be input.

The command input by the interactive operation input device 102 is supplied to the image generator 107.

The image generation unit 107 transfers the player's head position / posture information supplied from the head position / posture measurement unit 101 and the command information supplied from the interactive operation input device 102 to the control unit 400. Further, the head position / posture information of the player and the command information, the head position / posture information of the other player received from the control unit 400, the command information and the model information, the position, the moving direction, the state information of the enemy character, the space A virtual space image to be displayed on the HMD 105 of the corresponding player is created by using the information such as the position and shape of the obstacle arranged at, and is output to the image synthesizing unit 106.

The image synthesizing unit 106 synthesizes the image (real space image) of the video camera 103 attached at the position close to the player's viewpoint and the virtual space image created by the image creating unit 107, and the HMD 105 of the player. Supply.

The control unit 400 includes the mixed reality space management unit 1 to which the player processing units 100 to 300 as described above are connected.
And the physical object position measuring unit 2. The mixed reality space management unit 1 uses the player processing units 100 to 300.
The information regarding the position and posture of the player's head received from the player and the position, posture and command information of the interactive operation input device 102 are distributed, and the generation, disappearance, control and shooting commands of the enemy character displayed in the virtual space are used. Performs game processing such as shooting hit determination. Enemy character's model, position, movement direction, and state (whether it was destroyed)
Information is also distributed to all users who are connected together with the information of each player.

Further, as shown in FIG. 3, in order to adjust the difficulty level of the game, a real object 31 that may be an obstacle to shooting.
~ 33 are arranged, the physical objects 31 to 3
The information about the shape and position of 3 also includes the mixed reality space management unit 1
Managed by.

FIG. 3A is a perspective view of a game field (mixed reality space) seen from a player, and FIG. 3B is a top view thereof. In FIG. 3, three physical space objects 31 to 3 are placed on a table 30 (in physical space).
The case where 3 is arranged as an obstacle is shown. In the present embodiment, since the aiming command input operation using the interactive operation input device 102 is an operation of raising the wrist higher than the elbow as described above, the height is about the same as the waist in a general figure. Although the physical space object is arranged on the table 30 that it has, it goes without saying that the table is unnecessary depending on the command input operation by the interactive operation input device.

In FIG. 3, physical space objects 31 and 32 are fixed, and 33 is movable. If there is a movable physical space object, the difficulty level of the game can be dynamically changed. For example, if the movable physical object 33 moves to the left and right at random speed, the shooting of the enemy character that is the target becomes more difficult than when only the static physical objects 31 and 32 are arranged. The movement control of the movable reality object may be performed by the mixed reality space management unit 1 or by another control circuit. In this case, the model of the movable object is managed by the mixed reality space management unit 1, and the position thereof is measured by the physical object position measuring unit 2 using the object position sensor 331 provided on the movable object 33.

In the mixed reality apparatus as described above, the control unit 400 is the server, and the player processing units 100 to 300.
It can be realized by a client = server system in which (the circuit portion of) is a client. It is possible to flexibly deal with the increase and decrease of the players by distributing the processing related to each player among the clients. More specifically, the player processing unit is a general-purpose computer device having a video input / output function and a signal receiving function from various sensors, and the control unit 400 can also communicate with each player processing unit and an object position measuring unit 2. It can be realized by a general-purpose computer device having a function of receiving a measurement signal from.

However, since it is necessary to perform the calculation relating to the three-dimensional image display in real time, it is preferable to use a relatively high speed computer device having an accelerator (so-called 3D accelerator) specialized for such calculation. . Further, the control unit 400 and the player processing units 100 to 3
The communication with 00 is also preferably connected by a line with a large capacity such as 100BASE-T. If the capacity of the communication line is small, the processing speed will decrease as the number of players increases.

(Display by Command Input) Next, the display operation by command input of the interactive operation input section will be described with reference to FIG. FIG. 4 is a flowchart showing a display process performed when a shooting command is input. First, the input of the shooting command is determined (step S101), and when the shooting command is input, the shooting display is started (step S103).

Further, it is checked whether or not there is an object in the shooting direction (step S104), and if there is nothing, display is performed until a predetermined range is reached (step S10).
6) After a predetermined time, the display is erased to end the shooting display (step S107). On the other hand, if an object exists within the range, it is detected whether or not it is a target (step S105), and if it is a target, the shooting display is ended (step S108), and the target display is displayed as an explosion. The display is changed or transformed so that the player can recognize that the shot is hit (step S109). On the other hand, when an object other than the target is within the range, the shooting is blocked at that range, and the shooting display is ended (step S10).
7). When shooting an object other than the target, "mis
Display on the game effect such as “s” may be performed.

Whether or not an object is present on the trajectory of shooting, and if so, the position and type of the object may be detected before the shooting display is made, and the display may be made based on the result. .

(Display Superposed on Player) The mixed reality apparatus according to the present invention is characterized in that the virtual space image is displayed superposed on the player and the display content can be dynamically changed according to the conditions. And

A display device such as an HMD is indispensable for the mixed reality apparatus so that the player can visually recognize the virtual space by synthesizing the virtual space and the real space. Further, in order to perform some action in the mixed reality space, a sensor for detecting the action of the player, a switch, etc. are also attached. For example, in the game of the present embodiment, the player's head is equipped with an HMD or the like, and the arm is equipped with an interactive operation input device. Such a device to be mounted on a player (a mounting device) is not so conscious of one player, but when there are a plurality of players, the mounting devices of other players can be seen. Therefore, especially in an application such as a game, there is a case where the player lacks a sense of realism and hinders the player's empathy for the game.

Therefore, in the present invention, at least the wearing device of each player is covered with the virtual space image, thereby enabling an application having a higher entertainment value. Then, by making the displayed virtual space image have appropriate content according to the application, the empathy of the player is promoted, and it is possible to add a new effect by dynamically changing the displayed image.

FIG. 5 is a diagram showing a display example when a virtual space image is superimposed on the player's whole body. A virtual helmet 41 is superimposed on the player's head, and virtual parts 42 to 46 are superimposed on both arms, both legs, and the body, respectively. When such a superimposed display is performed, if the player is managed by a model approximated by a simple shape such as a cylinder or a rectangular parallelepiped as shown in the figure, the processing load of the player processing unit and the mixed reality space management unit can be reduced, and the body shape can be reduced. It can be used in common by different players. If the player's wearing device should be hidden, the virtual image may be superimposed only on the corresponding portion. That is, in the case of this embodiment, the virtual helmet 41 for hiding the HMD
Then, only the virtual part 42 or 43 for hiding the interactive operation input device attached to either the left or right arm is required.

In such display control, for example, the position / orientation information detected by the head position / orientation sensor 1011 of another player or the position / orientation sensor 1021 provided in the interactive operation input device and the aforementioned player model are used. It can be realized by using and drawing a superposed virtual object to generate an image.

As the virtual object to be superimposed on the arm to which the interactive operation input device is attached, the virtual object can be simply an object covering the arm portion, but by setting it as an object representing the function in the game, it is possible to hide the attached device. The effect can be obtained. In other words, when a virtual object that hides the interactive operation input device includes an object such as a weapon used in the game in the virtual object (or hides the attached device with an object such as a weapon), it is possible to select and use a plurality of weapons. As described above, the player can easily check the weapon currently in use by looking at his / her arm, and can determine what weapon another player has selected.

Further, the virtual object to be displayed is changed according to the command input by the interactive operation input device,
It can also be transformed. FIG. 6 is a diagram showing an example of changing a virtual object from a weapon to a shield when the above-mentioned defense command is input. That is, in the state of FIG. 6A which is a reset command input operation, when a defense command input operation is performed in which the back of the hand is placed outside and the fingertip is directed upward in preparation for an attack from an enemy or the approach of the enemy (FIG. )), The weapon 47 is no longer displayed, and a virtual object including the shield 47 'is displayed instead (FIG. 6 (c)).

FIG. 6 shows the case where the player who inputs the defense command is seen from another player, but the player himself who inputs the defense command also sees the virtual object of the shield '47 from the back side. You can feel as if you are defending with a shield. FIG. 7 shows a display example when the virtual object to be displayed is changed depending on the selected weapon type (sword and gun).

As the condition for changing the display contents, not only the player's active game operation of command input, but also the position information of the position change input unit due to the movement of the player can be used. FIG. 8 is a diagram showing an example in which the display is changed by using the player's line-of-sight direction information.

In the example of FIG. 8, the position and size of virtual eyes displayed in the virtual helmet and the accompanying display effect are changed. Specifically, the virtual eyes are moved according to the player's line-of-sight direction (see FIG. 8).
(A) and (b)) are considered. By adopting such a display change, the player can recognize the direction in which another player is gazing. Further, the virtual eye display is enlarged as the player's line-of-sight direction approaches the direction of the other player (FIG. 8 (c)), or when the line-of-sight direction and the direction of the other player coincide. Increase the display brightness of the eyes or make them blink (Fig. 8
(D)) etc. are considered. By adopting such a display change, it becomes possible for another player to recognize the player who is looking at himself, and in the game in which the aim is set as in the present embodiment, the line-of-sight direction and the aim direction are normally set. Because they are close together, they are useful information for developing a game strategy.

In order to make such a display change, for example, as shown in FIG. 9, a shift angle a between the direction of the player 1 whose display is to be changed with respect to the player 2 and the line-of-sight direction.
Can be realized by obtaining the size of the player from the position information of each player and the line-of-sight information of the player 1, and changing the display according to the change of the angle a.

Further, since the player generally looks at the sighting direction, the display control as shown in FIG. 8 is performed based on the information on the sighting direction by the interactive operation input device without detecting the sight line direction. It can also be configured as follows.

Specific display control will be described with reference to FIG. FIG. 10 shows (a) → (c) of FIG. 8 based on the conditions shown in FIG. 9 using the line-of-sight information and the position information.
It is a flow chart which shows processing when displaying-(d). First, the head position / posture information and the viewpoint position information of the other player (player 1 in FIG. 9) and oneself (player 2 in FIG. 9) are acquired (step S20).
1, S202). Next, the direction in which the distance to the other player is the shortest is detected (step S203).

Then, the line-of-sight direction of the other player is obtained (step S204), and the angle difference (angle a in FIG. 9) from the direction obtained in step S203 is obtained (step S20).
5). This angle difference is compared with a table of the angle difference and the virtual eye size stored in advance in the image generation unit 107 to acquire the virtual eye size (step S20).
6).

When the size of the virtual eye is acquired, the size of the virtual eye that is displayed in a superimposed manner is updated and displayed again (step S207). Then, it is checked whether or not the angle difference is less than or equal to a predetermined value (step S208), and if it is less than or equal to the predetermined angle, the effect of FIG. 8D is additionally displayed (step S209).

The mixed reality space management unit 1 manages the virtual object and the model of the player displayed in a superimposed manner. The model of the virtual object is a three-dimensional model, and the appearance changes depending on the viewpoint position. That is, the side of the virtual helmet can be seen when looking at the player facing sideways.

(Instruction Display) FIG. 11 is a diagram showing an example of an instruction display for setting an aim in this embodiment.
When it is necessary to point to one point on a space shared by a plurality of players, it is necessary to be clearly recognized not only by the person who points, but also by other players. Furthermore, if the instruction display is large or thick, the field of view becomes very complicated and the progress of the game may be hindered. Also, since a single point in the space is designated, it is difficult to grasp the sense of distance.

In the present invention, instruction display is performed using a plurality of parallel lines arranged at equal intervals. In FIG. 11 (a), the virtual tip part 43 of the virtual part 43 displayed over the interactive operation input device worn by the player is started.
The case where three parallel lines 91 to 93 having a predetermined length and made into dots are used as an instruction display is shown. Parallel lines 91 to 93 are shown in FIG.
As shown in (b), they are arranged so as to pass through the vertices of an equilateral triangle when viewed from the front. Distance d between parallel lines 91 to 93
May be appropriately determined depending on the size of the space that can be pointed, but an interval of about several centimeters (about 3 to 5 cm) is necessary so that the pointing direction becomes clear as described later.

By performing such an instruction display, a portion overlapping with the instruction display (in virtual space or real space)
Is not hidden by the indication display, and the sense of distance and the pointing direction can be easily recognized. That is, FIGS. 12A to 12C
As shown in FIG. 12, from the player who gives an instruction, FIG.
It is easy to see which point in the space you are pointing to. On the contrary, the other player in the vicinity of the front of the player giving the instruction looks as shown in FIG. 12B, so that it is possible to recognize that the instruction is given toward the front. Further, since it looks like FIG. 12 (c) from the front, the aimed player can recognize it in a game application in which the aiming player is aimed at by the instruction display, for example.

The indication display can be used not only to indicate a point in space but also to make the existence of an object clearer. In FIG. 13, the player can more clearly recognize that there is an object by displaying the instruction display line only up to a position in contact with the object 11 (which may be a virtual object or an object arranged in the real space). .

Further, as shown in FIG. 14, if the scales 121 and 122 are added to the line of the instruction display at predetermined intervals, the distance can be grasped more clearly. In this case, the intervals between the scales do not have to be equal, and may be logarithmic scales.

[0077]

Other Embodiments In the above-mentioned embodiment, there is no means for giving physical feedback to the player, but for example, a band for fixing the HMD, a physical unit such as a vibrator for the interactive operation input device, or the like. It is also possible to incorporate a device that gives various stimuli and give feedback to the player in accordance with the game situation. For example, when the enemy's attack is defended or the enemy is destroyed in the vicinity of the player, a more realistic effect can be provided by giving the player vibration as a shock.

Further, the control unit may deliver all the data to all the player processing units, or may deliver only the data necessary for the player processing unit. When all the data is distributed, only the necessary data is selected and used in the player processing section.

Further, in the configuration of the embodiment, the output of the video camera 103 is input only to the image input unit 104, but it is also supplied to the head position / orientation measuring unit 101 for the information correction of the head position / orientation sensor 1011. can do. In this case, if an object whose absolute coordinates do not change, such as a physical space object arranged as a table or an obstacle, is marked with a color that can be easily detected by image processing, the correction is easy.

Further, the number of parallel lines used for indication display is not limited to three, and any number can be selected. However, it is preferable to use three or more lines in order to easily recognize the perspective. On the other hand, if the number of lines is too large, it becomes difficult to see the portion overlapping the instruction display, and if the number of players increases, the complexity increases. Therefore, it may be appropriately determined depending on the size of the space and the number of players.

In the above embodiment, the trigger 1022 is not used, and the command is input by detecting the movement of the interactive operation input device. However, the player uses the trigger 1022 to input various commands. Alternatively, motion detection and trigger on / off may be combined. Further, the interactive operation input device may be incorporated with a switch that is turned on / off according to the player's operation and used as a trigger.

Further, an object of the present invention is to supply a storage medium (or recording medium) recording a program code of software for realizing the functions of the above-described embodiment to a system or apparatus, and to supply a computer of the system or apparatus ( Needless to say, this can also be achieved by the CPU or MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Also,
By executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instructions of the program code.
Needless to say, this also includes the case where the above-mentioned processes perform part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion card inserted in the computer or the function expansion unit connected to the computer, based on the instruction of the program code. Needless to say, this also includes a case where a CPU or the like included in the function expansion card or the function expansion unit performs a part or all of the actual processing and the processing realizes the functions of the above-described embodiments.

[0084]

As described above, the invention according to claim 1
According to Ming, an instruction to indicate an arbitrary position in the mixed reality space
As an image, we show n parallel lines that pass through the vertices of a regular n-sided polygon.
By using it, if the number of instructors increases or the space is small,
Even if the display is instructed, the instruction display hinders the visibility of the original display content.
Never be. Also, because it is composed of lines,
It has the advantage that it is easy to bite, as well as having a small physical load
is there.

Further , according to the invention of claim 2,
Since the operation part of the player is covered with a virtual image,
Both the instruction image and the instruction image can be composed of virtual images.
The advantage is that the alignment can be performed with high accuracy.
is there.

[0086]

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of a mixed reality apparatus according to the present invention.

FIG. 2 is a diagram illustrating a type of a device worn by a player and a mounting site.

FIG. 3 is a diagram illustrating a game according to the embodiment of the present invention.

FIG. 4 is a flowchart illustrating a shooting display process by command input.

FIG. 5 is a diagram showing an example in which a virtual object is superimposed on the player's entire body.

FIG. 6 is a diagram illustrating an example of a case where a display image is changed by command input by an interactive operation input device.

FIG. 7 is a diagram showing an example of a case where a display image is changed according to selection by a player.

FIG. 8 is a diagram showing an example of a case where a display image is changed according to the line-of-sight direction of another player.

9 is a diagram illustrating an example of conditions for realizing the display change of FIG.

10 is a flowchart showing a display process according to the conditions shown in FIG.

FIG. 11 is a diagram illustrating an instruction display method according to the present invention.

FIG. 12 is a diagram showing a case where the instruction display method according to the present invention is used,
It is a figure explaining the change of the appearance according to a display starting point.

FIG. 13 is a diagram showing an example in which the presence of an object is more clearly recognized by using the instruction display method according to the present invention.

FIG. 14 is a diagram showing another example of the instruction display method according to the present invention.

FIG. 15 is a diagram illustrating a conventional hockey game using mixed reality.

FIG. 16 is a diagram illustrating a conventional hockey game using mixed reality.

Claims (6)

(57) [Claims] 1. Combining a real space and a virtual space
A mixed reality that presents the player with a mixed reality space represented by
This is a method of presenting a feeling, in which the player's position / orientation information is acquired, the image in the physical space is input, and a virtual image corresponding to the player's position / orientation information is displayed.
Mixed reality that is generated using model information of a layer and combines the image of the physical space with the virtual image
A sense presentation method, including an operation unit operated by the player , and n parallel lines passing through the vertices of a regular n-sided polygon,
An instruction image for pointing an arbitrary place in the augmented reality space,
Generates a virtual image according to the position / orientation information of the operation unit.
It is characterized in that it is created and combined with the image of the physical space.
Mixed reality presentation method. 2. A virtual image for covering the operation unit is further created.
It is characterized in that it is created and combined with the image of the physical space.
The mixed reality presentation method according to claim 1. 3. The n number of images forming the instruction image
The parallel line starts almost at the tip of the virtual image covering the operation part.
The mixed reality presentation according to claim 2, characterized in that
How to show. 4. A scale is further displayed on the instruction image.
The duplicate according to claim 1, wherein the image is included.
Method of presenting a sense of reality. 5. The mixed reality space is shared by a plurality of players.
The mixed reality sense according to claim 1, wherein the mixed reality sense is provided.
How to show. 6. The method according to claim 1 is implemented in a computer.
A computer that stores a computer program to appear
Computer-readable recording medium.