JP3273017B2 - Image synthesis device and virtual experience device using the same - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image synthesizing apparatus for synthesizing and displaying a real space image and a virtual space image, and a virtual experience device using the same.

[0002]

2. Description of the Related Art In recent years, in the fields of a drive simulator, a flight simulator, a game device, a system kitchen simulated experience system, and the like, various virtual experience devices using a so-called virtual reality system have been proposed. In this type of virtual experience device, how to bring the virtual world closer to the real world is the biggest technical problem.

[0003] As an image display device for experiencing the world called virtual reality, for example, a device called a head mounted display is known. This device called a head-mounted display is configured by attaching a display device such as a liquid crystal display in front of a player so as to cover an operator, for example, a player's field of view in a game. Then, a device called a space sensor is attached to the head-mounted display, thereby detecting three-dimensional information of the player. And
Generate an image according to the detection signal from this space sensor,
By displaying this on the display device of the head-mounted display, it is possible to experience, for example, an operation of casually looking around in a virtual space with the same sense of reality as in a real space.

[0004] However, it has been found that the following problem arises when this device called a head mounted display is used in a driving game, for example.

That is, in this head mounted display,
The entire visual range of the player is covered with a liquid crystal display or the like. Therefore, it is not possible to actually see the steering wheel, gear, operation panel, hands and feet of the player itself, the state of people around the player, and the like in the real world where the player is actually operating. As a result, it has been found that an operation error has occurred in the steering wheel, the gear, and the like, resulting in a poor operation feeling. Further, it has been found that a speedometer, a tachometer, and the like displayed on the operation panel cannot be seen, resulting in a lack of realism.

As a method of preventing this, for example, a method in which the handle, gears, operation panel, and the like are all made into computer graphics and presented to the player can be considered. However, an image reproduced by computer graphics is merely a virtual one, and cannot completely reproduce a steering wheel, a gear, and the like seen in the real world. In particular, in recent years, in this type of driving game, for example, the body of a racing car used in an actual formula race or the like is installed in a game facility so that a player can ride on the actual racing car. There is a tendency to increase the sense of presence and realism. However, depending on the method of reproducing the vehicle body using the computer graphic described above, it is not possible to produce a sense of reality and a sense of reality obtained by installing such an actual vehicle body.

As described above, in a conventional image display apparatus called a head-mounted display, in a case where it is preferable to use a real image that should be actually seen in a real space, the virtual world is converted into the real world. The achievement of technical issues such as getting closer was insufficient.

Further, it has been found that the following problem arises when this device called a head-mounted display is used in, for example, a role playing game.

That is, in this head mounted display,
The space sensor detects only the position of the player and the facing direction. Therefore, for example, when this is used in a battle-type role-playing game, it is not possible to accurately recognize the position and direction of the battle weapon possessed by the player, It has been difficult to accurately and completely determine whether a particular player has defeated an enemy character. As a result, for example, when the fighting weapon is a so-called combat sword, or when the fighting weapon is a so-called combat gun and the target to be aimed at is considerably smaller,
There has been a problem that it is difficult to accurately judge the battle result.

As a result, when a conventional image display device called a head-mounted display is used in, for example, a battle-type role-playing game, the result of the battle cannot be accurately determined. The achievement of technical issues such as approaching the real world was insufficient.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and has been made in view of the above circumstances, and provides an image synthesis apparatus capable of bringing a virtual world closer to the real world by composing and displaying a real space image and a virtual space image. It is an object of the present invention to provide a device and a virtual experience device using the device.

[0012]

According to the present invention, there is provided an image synthesizing apparatus comprising: a first imaging device arranged in a direction of a field of view of an operator;
An apparatus for synthesizing an image reflected in the mirror member that is captured by the stage, a virtual three-dimensional space calculating means for calculating a virtual three-dimensional space representing the game space set by superimposing the real three dimensional space, the first The virtual three-dimensional space is converted to the real three- dimensional space as an image reflected on a mirror member captured by the first imaging unit.
Characterized in that it comprises a and a field image calculation means for image operations of the virtual field image as viewed from the mirror member positioned in the space.

[0013] Here, the image synthesizing apparatus of the present invention comprises:
A second imaging unit that captures a real space image viewed from a mirror member disposed in the real three-dimensional space ; and a display image that includes the virtual view image and the real space image captured by the second imaging unit. Images are synthesized and imaged by the first imaging means.
Display image synthesizing means for generating an image reflected on the mirror member.

Further, a virtual experience device according to the present invention includes the image synthesizing device according to the present invention. Further, the virtual experience device of the present invention is arranged such that a mirror member formed as a side mirror or a rearview mirror is arranged, and a riding body on which an operator rides, and an operation unit provided on the riding body and for the operator to input an operation signal. Image display means mounted so as to cover the field of view of the operator, first image pickup means mounted on the operator for capturing a real space image visible to the operator, and the mirror member
Second imaging means for capturing an image of a real space viewed from above, a space sensor for an operator for detecting three-dimensional information of the operator in a real three-dimensional space, an operation signal from the operation unit, and the space for the operator Based on the detection signal from the sensor,
Calculating means for calculating a virtual three-dimensional space which is set by being superimposed on the three-dimensional space, and performing image calculation of a virtual view image viewed by an operator in the virtual three-dimensional space; and imaging by the virtual view image and the second imaging means anda display image synthesizing means for image synthesis as the display image and a real space image to be, the calculating means, means for image operations of the virtual field image within viewing from the mirror member disposed in said real three dimensional space Including
The display image synthesizing unit is configured to control the virtual view image and the second
And image synthesis as a real space image and the display image captured by the imaging means, characterized in that it comprises a means for generating a rear image reflected in the mirror member which is captured by the first imaging means.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image synthesizing apparatus according to the present embodiment captures an image of a real space which is mounted on an operator so as to cover an operator's field of view and which is mounted on the operator and which is visible to the operator. Imaging means and the operator's 3
An operator calculates a virtual three-dimensional space that is set by being superimposed on the real three-dimensional space based on a detection signal from the operator's space sensor that detects dimensional information and the detection signal from the operator's space sensor. Virtual view image calculating means for calculating an image of a virtual view image seen by an operator, and display image synthesizing means for synthesizing the virtual view image and a real space image captured by the imaging means as a display image. And displaying the display image on the image display means.

Further, the virtual experience device according to the present embodiment has a boarding body on which an operator is boarded, an operation unit provided on the boarding body for inputting an operation signal by the operator, and a field of view of the operator. An image display unit attached to the operator, an imaging unit attached to the operator to capture a real space image visible to the operator, an operator space sensor for detecting three-dimensional information of the operator in a real three-dimensional space, Based on an operation signal from the operation unit and a detection signal from the operator space sensor,
Virtual view image calculation means for calculating a virtual three-dimensional space set by being superimposed on the real three-dimensional space and performing image calculation of a virtual view image seen by an operator in the virtual three-dimensional space; Display image synthesizing means for synthesizing a real space image captured by the means as a display image, displaying the display image on the image display means, and virtually feeling the operation of the vehicle by the operation unit. The feature is that you can experience it.

[0017]

In this case, the virtual view image calculating means includes:
A virtual three-dimensional space calculation unit that calculates a virtual three-dimensional space representing a game space set by being superimposed on the real three-dimensional space based on a predetermined game program and a detection signal from the battle weapon space sensor; It is preferable to include a field-of-view image calculation unit that calculates a virtual field-of-view image viewed from the player based on a detection signal from the space sensor for use.

According to the image synthesizing apparatus according to the present embodiment, the virtual view image in the virtual three-dimensional space calculated based on the operator's space sensor is captured by the imaging means mounted on the operator. The spatial video can be image-synthesized by the display image synthesizing means. As a result, the operator can incorporate the image of the real object in the real space into the virtual view image in the virtual three-dimensional space arbitrarily set by a program or the like, and see the virtual world closer to the real world. Can be experienced.

Further, according to the virtual experience apparatus according to the present embodiment, it is possible to view an image in a real space of a vehicle, an operation unit of the vehicle, and the like by the image synthesizing device.

Therefore, when this is used in, for example, a driving game, a real car on which oneself rides can be freely driven in a virtual space, so that the sense of reality can be greatly improved. Becomes Furthermore, it is possible to see, for example, 360-degree images in all directions, which cannot be seen in the conventional driving game, and to drive while looking at steering wheels, instruments, etc., which cannot be seen in the conventional virtual experience device. This greatly improves operability.

[0022]

In this case, for example, if the game results are calculated for each player and the setting of the virtual tertiary space representing the game space is changed for each player based on the game results, the skill of each player can be improved. It is possible to provide an appropriate game space, and to provide a virtual experience device that does not get tired even when playing repeatedly.

Next, a specific example of this embodiment will be described in detail with reference to the drawings.

1. First Embodiment FIG. 1 shows a block diagram of a first embodiment using the present embodiment. FIG. 2 shows an example of a virtual experience device in which this is applied to a driving game.

In FIG. 2A, a real racing car 30 for a game is installed on a floor 4 in a dome 1 whose inside is entirely painted in blue. Here, as with the inside of the dome 1, the floor 4 is all painted in blue. And this racing car 30
, An operator, for example, a player 50 in a game is on board. It is assumed that the term “operator” includes not only a player in a game but also all persons who use a virtual experience device such as a pilot operating a flight simulator and a player playing sports.

The racing car 30 includes, for example, a vehicle body 36, tires 32, a rear wing 34, a steering wheel 40,
It is configured to include a side mirror 41, an instrument panel 44, a shift gear, an accelerator, a brake (not shown), and the like.
For example, the front tire 32 is formed so as to be freely steered by operating the handle 40 of the player 50, and the rear wing 34 also moves up and down due to the operation of the player or a change in the speed of the racing car 30. It is formed as follows. Then, as described later, the player 50 can see the change of these movements by the video camera 10.

The instrument panel 44 includes, for example, a speedometer, a tachometer, a fuel gauge, and a warning gauge (not shown).
It is configured to change according to the driving state of the player 50. That is, the speedometer and the tachometer change in accordance with the accelerator operation, the brake operation, and the shift gear operation of the player 50, and the fuel gauge indicates the end of the game when the fuel runs out. Further, when a trouble occurs in the engine of the racing car 30 or the like, the warning gauge blinks, and the player 50 can know this by the video camera 10.

An attitude control unit 24 is provided below the racing car 30. The attitude control unit 24 controls the racing car according to the game situation (road surface change, road change, etc.), the steering operation of the player 50, the accelerator operation, and the brake operation. 30 posture changes and acceleration changes are controlled. Thus, the player 50 can experience a change in the posture and a change in the acceleration G,
You will be able to experience a virtual world closer to the real world.

The head mounted body 9 is mounted on the player 50 so as to cover the field of view of the player 50. The head-mounted body 9 includes a video camera 10 and a space sensor 1 for a player.
2. It includes an image display device 20 and a speaker 22.

FIGS. 4A and 4B show an example of the shape of the head-mounted body 9. FIG. 4A shows a video camera 10, a space sensor 12 for a player, and an image display device 2.
0, the wearing body 9 configured by providing the speaker 22 on the helmet 14 is shown. According to this type of mounting,
When the player 50 wears the helmet 14, a world completely isolated from the outside can be created, so that a more realistic virtual reality can be enjoyed. On the other hand, the mounting body 9 shown in FIG. 4B is configured such that the video camera 10, the spatial sensor 12 for the player, the image display device 20, and the speaker 22 are integrally attached to the mounting band 16. Therefore, it is possible to realize a more comfortable wearing feeling. The mounting body used in the present embodiment is not limited to the shapes shown in FIGS. 4A and 4B, and various shapes can be used.

The image display device 20 is mounted in front of the player 50 so as to cover the field of view of the player 50, and converts an image signal sent from the image synthesizing device built in the vehicle body 36 through the connection line 18 into an image. To display. As an image display method in this case, it is desirable to use a small display such as a color liquid crystal display or a small CRT in order to reduce the size of the head mounted body 9 and improve the wearing feeling. In addition, it is desirable that the image be corrected by an optical system so that the image is focused on the eye and the viewing angle is widened and the sense of reality is improved.

Here, the small display may be formed so as to cover the field of view of the player along the shape of the face of the player 50 to obtain a panoramic image effect, or two small displays may be used. Each of them may be formed in front of both eyes of the player 50. In the latter case, it is desirable to form a three-dimensional image by giving an image with parallax to a two-dimensional two-dimensional image given to both eyes. With such a configuration, the size of the object and the distance to the object can be grasped, so that a virtual world closer to the real world can be created.

The video camera 10 is used by the player 50 to view the real world. For example, as shown in FIG. 4, the video camera 10 is set at a position close to the viewpoint position (eye position) of the player 50, and its angle is set. It is also desirable to set the same as the field of view of the player 50. This is because, by setting in this way, the real-world video that is actually viewed from the player 50 can be viewed more comfortably. As a result, for example, the player 50 can check a racing car of another player who is chasing from behind by an operation in which the player 50 actually looks back.
It is possible to create a tense game world.

In this case, however, another video camera is attached to, for example, the rear of the
The user may be configured to be able to switch and view this by the operation of. Further, for example, the racing car 50 may be provided with a side mirror 41, a rear-view mirror, and the like, and a structure such as texture mapping or the like may be used to project an image that should actually be seen behind.

As the imaging means used as the video camera 10, it is desirable to use, for example, a high-resolution and smaller camera, for example, a high-resolution CCD camera, and to use a camera having a function such as automatic focusing. desirable.

The player's space sensors 12, 12 are sensors for detecting three-dimensional information of the player 50. For example, one side is on the ceiling of the dome 1 and the other side is on the player 50's side.
Is attached to the head mounted body 9.

The three-dimensional information of the player 50 here refers to, for example, positional information of the player 50 and view direction information, and obtains a relative positional relationship including the directions of the spatial sensors 12 and 12 in the three-dimensional space. Is detected by However, if the position of the player 50 does not move in the setting of the game or is within the allowable range even if it moves, the three-dimensional information does not necessarily need to include the position information of the player 50. Conversely, when the direction of the field of view of the player 50 is not changed in setting the game, the three-dimensional information does not necessarily include the player 50.
Need not be included. In these cases, subsequent image calculation processing may be performed based on the position or view information set on the game.

As a method for detecting three-dimensional information by the space sensors 12, 12, for example, the following method can be considered. That is, each of the space sensors 12, 12 is constituted by three coils orthogonal to each other. Then, a current is caused to flow through the coil of one of the space sensors 12, and at this time, the positional relationship including the directions of the space sensors 12, 12 is detected from the current value induced in the coil of the other space sensor 12. . As a result, the spatial information of the player 50 is detected.

The method of detecting three-dimensional information by the space sensors 12, 12 is not limited to the method using the above-described dynamic magnetic field, but may be, for example, a method using a static magnetic field or a method using an ultrasonic wave. Is also good.

Next, an image synthesizing method according to the present embodiment will be described below.

In this embodiment, as shown in FIG. 2B, a real space image 100 in a real three-dimensional space captured by a video camera 10 and a virtual view image 102 in a virtual three-dimensional space are synthesized. Thus, a display image 104 is formed. Then, the display image 104 is output to the image display device 20 through the connection line 18 and is actually
Is a view image to be viewed.

In the present embodiment, the image synthesis is performed by blue mat synthesis. In other words, the racing car 30 and its accessories, the player 50 who is himself, etc.
Other than that, the inside of the dome 1 and the floor 4 are all colored blue. By doing so, the real space image 1
At 00, all parts other than the racing car 30, the steering wheel 40, and the player's hand 54 have a blue background. Then, all the pixels of the blue color portion in the real space image 100 are set as empty dots, and the virtual view image 10
2, the display image 104 can be obtained. In this case, for example, the dome 1 has a circuit background mainly seen from the player 50, and the floor 4
Shows the road surface condition of the road on which the racing car 30 is running.

FIG. 1 is a block diagram showing an example of the image synthesizing apparatus in this case.

In FIG. 1, the virtual visual field image calculation device 70
Is for calculating a virtual image that can be viewed by the player 50 in the virtual three-dimensional space, and includes a virtual three-dimensional space calculation unit 72 and a view image calculation unit 76. And
The virtual three-dimensional space calculation unit 72 calculates a virtual three-dimensional space that represents a game space set by being superimposed on the real three-dimensional space. The view image calculation unit 76 calculates the virtual three-dimensional space that has been calculated. Thus, a virtual field-of-view image in a direction that can be seen from the player 50 is calculated.

The program section 74 stores a game calculation program. For example, taking the driving game of the present embodiment as an example, all objects (racing cars, roads, other racing cars, backgrounds visible from the roads, etc.) on the circuit constituting the driving game space are represented by a combination of polyhedrons. The three-dimensional information and associated data of each vertex of the polyhedron are stored as image information. In addition to the above data, the program section 74 also stores a program for controlling the progress of the game and the like, and stores all the circuit information if the user can enjoy a plurality of circuit runs.

Operation signals input by the player 50 through the steering wheel 40, accelerator, brake, and shift gear are as follows:
The data is input to the virtual three-dimensional space calculation unit 72 via the operation unit 38. Then, the operation signal and the program unit 7
4, the position and direction of the racing car 30 in the virtual three-dimensional space are calculated in real time. Then, based on the calculation result, image information of the racing car 30 in the virtual three-dimensional space and all objects on the circuit around the racing car 30 is output to the view image calculation unit 76.

The visual field image calculation unit 76 performs the following coordinate conversion based on the three-dimensional information detected by the space sensor 12 for the player. That is, as shown in FIG. 12, image information calculated by the virtual three-dimensional space calculation unit 72 (for example, an object 17 such as a signboard,
0, 172) from the world coordinate system (XW, YW, ZW) in the virtual three-dimensional space.
The coordinates are converted to a viewpoint coordinate system of 0 (XV, YV, ZV).
Then, the image information of the object 172 such as a signboard passing outside the field of view of the player 50, for example, out of the field of view, is clipped and removed, and the object 170 in the field of view is removed.
Image information is perspectively transformed into a screen coordinate system (XS, YS), that is, a coordinate system that can be actually viewed from the player 50. In this case, the perspective-transformed image information is used as polygons 174, 175, and 17 corresponding to the polyhedron constituting the object.
6 (the back side polygon is omitted). Next, the image information of all the dots in the polygons 174, 175, and 176 is calculated based on the accompanying information such as the color and luminance given for each polygon or each vertex of the polygon, and this is calculated as a virtual view image. The information is output to the display image synthesizing device 80.

In the display image synthesizing device 80, the image synthesis of the virtual visual field image 102 from the visual field image calculation unit 76 and the real space video 100 from the video camera 10 is performed. Various methods can be considered as a method of the image synthesis. In the present embodiment, as described above, the image is synthesized by the method of the blue mat synthesis. FIG. 13 shows details of the display image synthesizing device 80 in this case.

That is, in FIG. 13, an image signal representing the real space image 100 input from the image camera 10 is first passed through a filter 200 in the display image synthesizing device 80 and is divided into three primary color components of RGB. Then, each of these components is A / D converted into 8-bit digital data by the A / D conversion circuit 202, thereby obtaining 24-bit RGB digital data for each pixel. Then, whether or not the 24-bit RGB digital data of each pixel in the real space image 100 matches the 24-bit RGB digital data of blue color painted on the back side of the dome 1 and the floor 4 is determined by whether or not there is an empty dot. The judgment circuit 204 calculates and judges for each pixel.
Then, the result of this determination is written to the empty dot memory 206. The empty dot memory 206 has a configuration of a 1-bit memory corresponding to all pixels of the display image, and empty dot determination data indicating whether or not the pixel is an empty dot is written as 1-bit data for each pixel.

The display image synthesizing device 80 has a built-in field buffer 210 corresponding to each pixel of the display image. Then, the data control unit 208 refers to the empty dot determination data written in the empty dot memory 206, and writes a real space image at each pixel position of the field buffer 210. That is, if the pixel is determined to be a free dot by the free dot determination data, the pixel position in the field buffer 210 is
No real space video is written. Conversely, if it is determined from the empty dot determination data that the pixel is not an empty dot, the 24-bit RGB digital data of the real space video is written as it is.

Next, the data control unit 208 refers to the empty dot determination data written in the empty dot memory 206 and stores the virtual view image calculated by the view image calculation unit 76 at each pixel position in the field buffer 210. Information is overwritten. That is, when it is determined from the empty dot determination data that the pixel is an empty dot, the virtual view image information is written as it is. vice versa,
If it is determined from the empty dot determination data that the pixel is not an empty dot, nothing is written, and a real space image is displayed at this pixel position.

After the above writing, the image information data at each pixel position is read from the field buffer 210 by the data control unit 208. Then, the image information data is output as an image to the image display device 20 through the connection line 18, and the player 50 can see the display image 104 in which the virtual view image 102 is incorporated in the real space image 100 in real time.

It is more preferable that the writing and reading of the image information be performed simultaneously by, for example, configuring the field buffer 210 for two screens.

In the virtual three-dimensional space calculation unit 72, not only the calculation of the image information but also, for example, the sound synthesis unit 7
It also generates an audio signal output from the speaker 22 through the controller 8 and an attitude control signal for controlling the attitude of the racing car 30. That is, according to the game progress situation in the game space, which is calculated based on the game program from the program 74 and the operation signal from the operation unit 38,
Generate more effective voice signal and attitude control signal,
The game's presence is further enhanced.

Further, in the present embodiment, the virtual view image calculation device 70 is divided into a virtual three-dimensional space calculation unit 72 and a view image calculation unit 76, but this is for convenience. This may be configured by a method in which the function of performing the spatial calculation and the function of calculating the view image are integrally performed. That is, as a result, in the virtual three-dimensional space,
As long as a virtual field-of-view image in a direction visible from the player 50 can be obtained, the calculation order, the calculation method, and the like are not limited to the method shown in FIG.

The method of synthesizing the image in the display image synthesizing device 80 is not limited to the above-described method. For example, the image is synthesized using red instead of blue, the image is synthesized using a plurality of colors, Can be used.

The method of obtaining the image information of all the dots in the calculated polygons in the view image calculation unit 76 is not limited to the above-mentioned method, and for example, a method called texture mapping may be used. That is, each vertex of the object in the virtual three-dimensional space is given in advance texture coordinates specifying texture information to be attached. Then, after performing the above-described calculation on the viewpoint coordinate system, the texture information is read from the texture information memory using the texture coordinates as an address, and the texture information is attached to the polygon, thereby obtaining an image of all the dots in the polygon. Compute information.

By calculating image information using such a technique called texture mapping, first,
There is an advantage that the load on the arithmetic processing portion can be significantly reduced. In addition, various unprecedented image effects can be obtained by using this method.
Examples thereof include the following.

That is, the video camera 43 is attached to the side mirror 41 provided on the racing car 30. Then, as shown in FIG. 14, the rear real space image 105 captured by the video camera 43 and the rear virtual view image 106 are image-combined to form a side mirror display image 108.
Is formed, and this side mirror display image 108 is attached to a portion of the side mirror 41 of the display image 104 viewed from the player 50 by using a texture mapping method.
With such a configuration, the side mirror 41 has the configuration shown in FIG.
Similarly to the display image 104 shown in (b), an image in which the rear real space image 105 and the rear virtual view image 106 are combined is displayed. Thus, for example, the racing car 31 of the opponent player chasing from behind.
And the background image in the back and my racing car 30
Can be viewed by the side mirror 41, and a virtual world closer to the real world can be created.

FIG. 15 shows another example of the video effect using the texture mapping method. In this example, as shown in FIG. 15A, the racing car 30 of the player 50 and the racing car 31 of the opponent player 52 are installed in independent dome portions, respectively. A video camera 26 is installed in the dome, and the heads 28 of the players 50 and 52 are photographed from all sides. The photographing data of the player's head 28 is shown in FIG.
As shown in (b), the texture is mapped to the head of the opponent player 52 on the virtual view image 102 by using the texture mapping method. In this way, when the player 50 looks back, the real image of the head of the opponent player can be seen, and the sense of reality of the game can be greatly enhanced. In particular, when the racing car 31 of the opponent player 52 catches up with the own racing car 30 and lines up side by side, it is possible to see a face, etc., which is proud of the victory of the opponent player 52, thereby stimulating each player's consciousness of competition. A game with a sense of reality and tension can be provided.

In FIG. 15, the head 28 of each player is photographed by the video camera 26 from four directions.
It is sufficient that there is video information from at least the front of the player's head 28, and video information from other directions may be the one created by the virtual three-dimensional space calculation unit 70.
Further, the photographing means of the head 28 of each player is not limited to the video camera 26. For example, before starting the game play, the photographing means may be photographed by a still image camera and registered and used.

FIG. 16 shows an example of a simple block diagram for synthesizing an image by the texture mapping method described above.

In other words, when synthesizing images by the texture mapping method, the field-of-view image calculation unit 76 is configured, for example, as shown in FIG.

The image information of each object output from the virtual three-dimensional space calculation unit 72 includes the vertex coordinates VX, VY, and VZ of the polygon constituting each object and the texture vertex coordinates VTX given to each vertex of the polygon. , VTY, and the like. Here, the texture coordinates TX and TY specify texture information to be attached to each polygon. The texture information is stored in the texture storage unit 308 using the texture coordinates TX and TY as addresses. The texture vertex coordinates VTX and VTY represent the texture coordinates at the vertex position of each polygon among the texture coordinates TX and TY.

The vertex coordinates VX, VY, VZ, the texture vertex coordinates VTX, VTY, etc.
Is input to Then, the vertex coordinate transformation unit 300 performs coordinate transformation such as perspective transformation, and outputs the result to the sorting circuit 302. Sorting circuit 30
In 2, the processing priority of each polygon is set. The priority is set so that, for example, a polygon closer to the viewpoint of the player is preferentially processed, and the subsequent processing is performed according to this priority.

The processor unit 304 calculates the display coordinates and texture coordinates T of all the dots in the polygon from the vertex coordinates and texture vertex coordinates of each polygon after the coordinate conversion.
X and TY are required. Then, the obtained texture coordinates TX and TY are written in the field buffer unit 306 using the display coordinates described above as an address.

When displaying an image, the texture coordinates TX and TY are read from the field buffer unit 306, texture information is read from the texture storage unit 308 using the texture coordinates TX and TY as addresses, and a virtual view image is formed. Thereafter, the display image synthesizing device 80 performs image synthesis with the real space video.

With the above configuration, the side mirror display image 108 is displayed on the side mirror 41 by, for example, the following arithmetic processing. Here, the side mirror 41
For example, the image is painted in blue, and an image calculated by the image synthesizing device is set to be displayed at this position.

In this case, first, image synthesis of the side mirror display image 108 shown in FIG. 14 is performed. That is, in FIG. 16, the calculation of the rear virtual view image 106 is performed by the virtual three-dimensional space calculation unit 72 and the view image calculation unit 76. The rear real space image 105 is photographed by the image camera 43. Then, the virtual view image 10 behind this
6 and the rear real space image 105 are synthesized by the display image synthesis device 80, and the side mirror display image 108 is synthesized.
Are image-combined.

Next, this side mirror display image 108
Is returned to the texture storage unit 308 as shown in FIG. Then, in the storage area of the texture storage unit 308, the image information of the side mirror display image 108 is written at the position of the texture coordinate corresponding to the side mirror 41. After that, the display image synthesizing device 80 performs image synthesis of the virtual view image 102 on which the side mirror display image 108 is displayed and the real space image 100 captured by the image camera 10. As a result, an image in which the side mirror display image 108 is displayed can be formed on the side mirror 41 of the display image 104.

When the real image of the head of the opponent player is projected by the configuration shown in FIG. 16, the image photographed by the image camera 26 is directly written into the texture storage unit 308 as shown in FIG. . In other words, the video data captured by the video camera 26 is written in real time at the position of the texture coordinates corresponding to the head of the opponent player in the storage area of the texture storage unit 308. This makes it possible to obtain a display image in which a real image of the head of the opponent player 52 is projected on the virtual field-of-view image 102 viewed from the player 50.

In the case where the still image camera is used, a picture of the opponent player is stored as image data in the texture storage unit 308 when registering before starting the game.

Further, the texture mapping method in the present embodiment is not limited to the structure shown in FIG. 16, and any method of texture mapping can be used.

According to the present embodiment having the above-described configuration, it is possible to provide a game with a sense of realism that could not be experienced in a conventional driving game.

That is, first, in the conventional driving game, a game screen for displaying a circuit or the like is displayed on the player 50.
Because it was set only on the front of the, there was still a lack of realism. On the other hand, in the present embodiment, since the player 50 can see all directions, for example, 360 degrees, the sense of reality of the game is greatly improved.
In particular, for example, when the present embodiment is applied to a driving game by a plurality of players, the player 50 chase by actually looking back or by using the side mirror 41 and the rearview mirror having the above-described configuration. The player of the opponent can be confirmed, and the sense of presence can be further increased. In this case, if the real space image of the opponent player is attached by texture mapping as described above, it is possible to further increase the realism and tension of the game.

In the present embodiment, the player 50
Can ride on a real racing car 30 in a real three-dimensional space, and can run the real racing car 30 freely in a virtual three-dimensional space for games. You will be able to experience the approaching virtual world. That is, the player 50 checks the movements of the racing car 30, the tires 32, the rear wing 34, the opponent player and the like on the side mirror 41 in the actual three-dimensional space while actually checking the racing car 30 with his / her own eyes. Can be operated. Further, since the operation units operated by the player 50, that is, the steering wheel 40, the instrument panel 44, the accelerator, the brake, the shift gear, and the like can be operated while actually confirming with the user's own eyes through the video camera 10 as described above. ,
The operability is greatly improved, and the sense of reality of the game is also greatly improved. Also, in this case, if the control of the attitude control unit 24 and the sound output to the speaker 22 are changed according to the game situation, a game with a more realistic feeling can be provided.

FIG. 3 shows an example of a virtual experience apparatus when the first embodiment is applied to a flight simulator.

In FIG. 3A, a blue mat (hereinafter referred to as a blue mat) is attached to the right window 2 and the left window 3 in the cockpit 45, and the above-described method is applied to the blue mat. Display image 1
04 is fitted. This state is shown in FIG. Here, in FIG. 3B, a virtual view image 102 is fitted to a real space image 100 in the left window 3 direction, and a display image 10 is displayed.
4 is synthesized.

In this case, the image displayed on the display image 104 looks different between the pilot 46 and the instructor 48. This is because the virtual field-of-view image 102 uses the spatial sensor 12 for the player to
This is because the direction of view of each instructor 48 is detected and calculated. Therefore, it is possible to create an image that can be seen by the instructor 48 but cannot be seen by the pilot 46, that is, an image effect that differs depending on the direction of view, and a virtual world with a more realistic feeling can be realized. In this regard, such a video effect cannot be produced by a conventional method, that is, a method of simply providing a CRT display on a window, or a method of displaying a CRT image on a window by a half mirror.

Further, in the present embodiment, the pilot 46
Can simulate the operation of an airplane while viewing the operation panel 47, the control stick 42, the face of the instructor 48, and the like in the cockpit 45 with the video camera 10. Therefore, the operation panel 47 of an airplane, which is generally considered to be complicated and difficult to operate, can be operated by the video camera 10 while actually seeing it with its own eyes, so that operation errors can be dramatically reduced.

That is, for example, according to a system in which all of the operation panel 47, the control stick 42, and the like are formed by image synthesis, the operation feeling for the pilot 46 is poorly realistic. Moreover, according to this method, it is necessary to attach a data glove to the pilot 46, and to detect what operation the pilot 46 has performed on the operation panel 47 and the control stick 42 based on an operation signal from the data glove. As a result, the scale of the system becomes enormous. In this way, the system has a drawback that the operation feeling of the pilot 46 is not good even though the scale is enormous, and there are many operation mistakes of the pilot 46.

In this regard, in this embodiment, the cockpit 45 has the same structure as the cockpit of a real airplane, so that such inconvenience does not occur. Further, in the present embodiment, since the instructor 48 next to the instructor 48 can be viewed, the instruction of the instructor 48 can be accurately grasped.
In addition, an angry face of the instructor 48 when an operation error is made can be seen, so that a more realistic flight simulator can be realized.

2. 5, FIG. 6 shows an example of a virtual experience system applied to Russia Lumpur playing game is shown.

As shown in FIG. 5, this example is different from the first embodiment in that the operation unit 38 is changed to a weapon 58 for fighting against an enemy character. A space sensor 64 is attached, and a virtual three-dimensional space calculation unit 72 has a game result calculation unit 68 built-in.

This role playing game is similar to the one shown in FIG.
The setting as shown in (a), that is, a plurality of players 50, 5, each having a battle weapon 58, for example, a sword 60.
It is set so that the second and other teams form a team and defeat the enemy character 66. Each player wears the head mounted body 9 having the same configuration as that of the first embodiment, and displays the display image 104 displayed on the image display unit 20.
While watching the game play.

The inside of the attraction room 110 is entirely painted in a blue color, so that the above-mentioned blue mat composition can be performed. In addition, the ceiling of the attraction room 110 has 3
A space sensor 12 for detecting dimensional information and a space sensor 64 for detecting three-dimensional information of a sword 60 as a battle weapon are attached. This attraction room 110
The space is substantially the same area as the game space in which the player actually plays the game, and the enemy character 66 is defeated while walking the players 50, 52, and the like in the attraction room 110 by themselves.

In FIG. 6B, a real space image 100
Is an image from the image camera 10 attached to the player 50. As shown in FIG.
At 0, the own hand 54, the second player 52 in front of the user, and the like are displayed with the blue attraction room 110 as a background. A virtual view image 1 created by the same method as in the first embodiment is added to this image.
02 is synthesized to produce a display image 104.
In this case, the virtual view image 102 includes the enemy character 66
In addition, mazes, doors, floors, and the like that constitute the game space are also displayed.

With this configuration, the player 50 can defeat the enemy character 66 while assembling with the second player 52 while watching the display image 104. In this case, communication with the second player 52 is performed through the speaker 22, for example. Also, enemy character 6
The determination as to whether or not 6 has been defeated is performed by a space weapon sensor 64 attached to the sword 60.

That is, the three-dimensional information, for example, the position and direction of the sword 60 are detected by the battle weapon space sensors 64, 64. Then, based on the three-dimensional information of the sword 60, the virtual three-dimensional space calculation unit 72 shown in FIG. 5 accurately calculates whether or not the sword 60 has successfully attacked the enemy character 66. Then, when the game is successful, the enemy character 66 in the game space is erased or the like, and this is displayed on the image display device 20 such as the players 50 and 52 in real time. As a result, each player sets the enemy character 66
Can be confirmed whether or not has fallen, and if it has fallen, the player advances to the next maze to defeat a new enemy character 66. Thus, the skill of such flop Reya 50,52, it is possible to change the game progress of the game space formed, and thus can provide a virtual experience system without getting tired even after repeated play.

In this example , the game score of each player, that is, the number of defeated enemy characters 66 is determined by using the game score calculation unit 68 based on the three-dimensional information from the battle space sensor 64 described above. The result is calculated, and this result is also output to the image display device 20 of each player. This allows
After the game is finished or during the game, the game results of each player, the game results of each team, and the like can be confirmed in real time. As a result, since each player can compete in real time with each player's skill, the fun as a game can be dramatically increased.

In this example , the video camera 10 is used as the video information of the second player 52 as viewed from the player 50.
Was used, but the present embodiment is not limited to this. That is, for example, the second player 5
2 may be displayed using a character synthesized by the virtual three-dimensional space calculation unit 72 without using a real space image. By doing so, it is possible to obtain a video effect such as evolving the second player as the game progresses and defeats the enemy character 66, and the fun of the game can be further increased.

As described above, the following method can be used as a method of changing the real space image of another player into a character image obtained by image synthesis. For example, a battle suit of a different color for each player, that is, a battle suit of a different color from the inside of the attraction room 110 is mounted,
Image synthesis is performed by a method similar to the blue mat method described in the embodiment. That is, the second player is equipped with a red-colored battle suit, and the video camera 10
Only the red color portion is extracted by a method similar to the above-described method, and is set as an empty dot. Next, if a character image that the user wants to be displayed in a superimposed manner on another player is overwritten, a second
An image in which the character image of the player 52 is overwritten can be obtained. After that, only the blue color portion is extracted from the overwritten image, this is set as an empty dot, and the virtual view image 102 on which the enemy character 66, the maze, and the like are displayed is overwritten.
As a result, it is possible for the second player 52 to obtain the display image 104 changed to the character image. In addition,
As a method of changing the image of the second player 52 as described above, for example, the above-described texture mapping method may be used.

In this embodiment , the sword 60 is used as the battle weapon 58. However, this embodiment is not limited to this, and any battle weapon such as a light gun can be used. Then, for example, when using a light gun or the like, since information on whether or not the player has shot the light gun is required,
In this case, as shown in FIG.
This information is input to 2 and will be used for calculating the game space.

FIG. 7A shows a virtual experience apparatus when the present embodiment is applied to a ride type attraction facility.

In this attraction, the player 50,
52 is mounted on a carrier 116, and the carrier 116 moves on a rail 118 laid in the attraction room 110. Various dioramas 114 are provided in the attraction room 110, and a background mat 112 painted in blue is provided inside the attraction room 110. Then, the players 50 and 52
Shoots down the enemy character 66 with the light gun 62 and competes for the game. FIG. 7B shows the state of image synthesis in this case. The method of this image synthesis is the same as that of the first embodiment.

When this example is used for an attraction facility, there are mainly the following advantages.

First, in conventional attraction facilities, once the attraction facility is constructed, it is very difficult to change the contents of the attraction because the facility is large and expensive. For this reason,
It is necessary to take measures to prevent the player from getting tired of playing the same game several times. In this regard, in this example , while the diorama 114 is common,
By changing the virtual image of the game space projected on the background mat 112, this can be easily dealt with. Further, as described above, the virtual three-dimensional space calculation unit 72
By providing the game result calculation unit 68 in the system, this can be dealt with more effectively.

That is, the number of enemy characters 66 shot down by each player is determined in real time by the game score calculation section 68.
The virtual image 102 displayed on the image display device 20 of each player is changed according to the number of the enemy characters 66 shot down, that is, according to the skill of the player. For example, for a player with a high level of skill, the number of enemy characters 66 displayed in the virtual view image 102 is increased, and the displayed enemy character 66 is made strong. Further, a configuration may be adopted in which several kinds of courses of the game space to be displayed on the virtual view image 102 are provided in advance, and one of the courses is selected according to the game result. As a result, a player riding this attraction facility can experience a virtual experience of a different game no matter how many times he or she rides, and can provide an attraction facility that does not get tired even after repeated playing.

Another advantage of the present embodiment is that, despite the fact that the attraction facility that does not get tired as described above can be realized, the real space image can be used for the diorama 114 and the like in the attraction room. The ability to create exciting attractions.

For example, if the transporter 116 is of a so-called roller coaster type, an attraction full of thrilling and speed can be realized when the diorama 114 seen from the player is a real space image. That is, if all the visual images viewed by the user are created by image synthesis, the inherent speed, tension and power of the roller coaster cannot be sufficiently reproduced. In this regard, in this example , the face of another player next to the player can also be viewed as a real space image, and the functions inherent to the jet coaster can be exhibited more.

Also, when the diorama 114 touches the player's hand, it is possible to provide a more realistic attraction when the video is a real space video. Regarding this point, for example, a book as shown in FIG.
Examples are shown in the attractions that apply.

The attraction shown in FIG.
This is an attraction that allows you to experience flying and fighting in a simulated manner.

In this attraction, as shown in FIG. 8A, a plurality of players get into the spacecraft cabin 120 of the spacecraft 134. The spacecraft cabin 120 is made very similar to a real spacecraft, and is provided with, for example, a cockpit and a combat seat. In this case, in particular, the control stick 132 and the operation panel 130 that the player directly touches are
The artillery 144 is elaborately made to look very real.

The players aboard the spacecraft cabin 120, according to their roles, the pilot, the co-pilot,
It is placed in the cockpit, combat seat, etc. as a shooter. And
The pilot 146 and the co-pilot 147 arranged in the cockpit,
The spacecraft 134 is controlled by the control column 132 and the control panel 130 while avoiding the meteorite 140 and the like projected on the cockpit window 122 by the blue mat method described above. In this case, in this example , as described above, the space sensor 12 is attached to each player, the view direction is calculated for each player, and the view image obtained by this calculation is displayed on the image display device 20. As a result, the appearance of the meteorite 140 approaching the spacecraft 134 is determined by the pilot 146, the co-pilot 147,
Since it can be set to look different from the shooter 148, it is possible to provide an attraction that is more realistic and realistic. Further, the pilot 146 and the co-pilot 147
Can operate the spacecraft while operating the control stick 132 and the operation panel 130 which are made very similar to the real spacecraft, so that the player can play as if he were operating a real spacecraft.

The shooters 148, 149 arranged in the battle seat
Is operated by a battle weapon 144 which is a battle weapon.
4. The spacecraft 142 projected on the right window 125 by the blue mat method is shot down. The game score in this case is calculated by the game score calculation unit 68, and is displayed in real time during the game or as a game result of all the crew members after the game is over.

As shown in FIG. 8 (b), the attitude and acceleration G of the spacecraft 134 on which the player rides are controlled by the attitude control unit 138 using hydraulic pressure and the like in accordance with the game progress and the player's operation signals. Thus, the configuration is such that the sense of reality is further increased.

As described above, in this example , since each player can play together in a very near-real spacecraft for different roles, it is possible to provide an attraction full of realism without getting tired.

In the present embodiment described above, the image display unit 20 and the image synthesizing unit (the virtual view image arithmetic unit 70, the display image synthesizing unit 80, etc. in FIG. 5) are arranged at different places, Although this configuration is used for connection, the present embodiment is not limited to this, and it is also possible to adopt a configuration in which the entire image display unit 20 including the image synthesizing unit is incorporated. Alternatively, as shown in FIG. It may be configured to be connected using this.

This walking type attraction is shown in FIG.
(A) As shown in (b), an attraction in which the player 50 equipped with the head-mounted body 9, the two-way radio 160, and the light gun 62 defeats the enemy character 66 while traveling in the maze 150 by himself. It is. Maze 1 like this
In an attraction in which the player goes inside 50, it is desirable that the player's movement has more freedom. Therefore, in this example , the connection with the image synthesizing unit is performed by the two-way wireless device 160. In this case, as the two-way wireless method, for example, infrared rays or the like may be used.

Here, the maze 150 has a window 152, a picture 1
54, an elevator 156, a door 158, and the like. A blue mat is laid on the entire surface of the window 152, and a blue mat is laid on only the portion where the enemy character 66 appears in the picture 154.

The elevator 156 and the door 158 are
The inside is painted blue. Thus, when the player 50 presses the elevator button to open the elevator or turns the door knob to open the door, the configuration is such that the enemy character 66 jumps out of the door. This makes it ideal for attractions in games such as ghost houses. In this case also, in this example , the elevator buttons, door knobs, and the like are genuine, and the player 50 can also operate while watching the real image, so that a more realistic virtual world can be enjoyed. Become.

3. Figure 10, Figure 11 is an example of a virtual experience system applied to Gore Ruch is shown. Although the case where the present invention is applied to golf is shown here, the present embodiment is not limited to this and can be applied to various sports.

As shown in FIG. 10, in the present embodiment, the operation unit 38 of the first embodiment is
86 and is obtained by changing the moving object detection apparatus 88 and 90, it is suitable for ball game with a mainly ball.

[0115] FIG. 11 (a), indoor golf is shown according to the present embodiment. This example relates to a virtual experience apparatus that allows the player 50 to experience a virtual world in the play room 162 with almost the same feeling as real golf.

In FIG. 11A, the play room 16
2 are all painted in blue and the player 50
Is equipped with a head-mounted body 9 having the same configuration as that of the first embodiment. Then, the player 50 makes a shot of the real golf ball 166 using the real golf club 164 as if he were at a real golf course.

In this case, in the play room 162, an image of the golf course created by the virtual three-dimensional space calculation unit 72 is displayed via the image display device 20, and the player 50 is as if the player is at a real golf course. You can have a feeling like that. When the player 50 faces forward, a virtual view image 102 showing a green 162 of the golf course is projected in the direction of the line of sight, as shown in FIG. 11B.

The player 50 moves the golf ball 16
When the shot of No. 6 is performed, the shot golf ball 1
66 is photographed by the ball video cameras 84 and 86. Then, based on the photographed image, the moving object detecting device 8
The coordinates of the center of gravity of the golf ball 166 are calculated by using the data 8 and 90.
Is calculated by estimating the trajectory of. Next, the virtual three-dimensional space calculation unit 72 draws the trajectory of the golf ball on the golf course set in the virtual world based on the estimated trajectory data of the golf ball 166,
And image display on the image display device 20. This allows the player 50 to see the golf ball 166 shot by the player 50 flying toward the green 162 on the display image 104.

Next, a method of detecting a moving object of the golf ball 166 will be described.

The ball video cameras 86 and 89 are for continuously photographing the golf ball 166, and are set at different positions and different photographing angles in the play room 162 as shown in FIG. The photographing data is stored in the data extracting unit 9 in the moving body coordinate detecting units 88 and 90.
2 and 94 are input as frame advance frame data.
In the data extracting units 92 and 94, only the photographing data of the ball is extracted by the background processing. That is, the golf ball 16
The difference between the frame data not shown and the frame data shown in No. 6 is obtained, and this is overwritten in an image memory called a frame buffer. As a result, image data in which only the image of the golf ball 166 is projected can be obtained.

Next, from the obtained image data, two-dimensional coordinates of the position of the golf ball 166, for example, the position of the center of gravity, are obtained in the coordinate detectors 96 and 98, and output to the virtual three-dimensional space calculator 72. .

The virtual three-dimensional space calculation unit 72 obtains three-dimensional coordinates in the virtual three-dimensional space from the two two-dimensional coordinates of the golf ball 166 detected by the moving object detection devices 88 and 90. That is, the virtual three-dimensional space calculation unit 72 previously stores the set positions and set angles of the ball video cameras 84 and 86 in the virtual three-dimensional space, and stores the stored data and the two detected two-dimensional data. From the coordinates, three-dimensional coordinates in a virtual three-dimensional space can be obtained. Then, from the obtained three-dimensional coordinates, the trajectory of the golf ball 166 in the virtual three-dimensional space is calculated using, for example, spline interpolation or the like, and is output to the view image calculation unit 76 together with the background data of the golf course.

Note that the method of estimating the trajectory of the golf ball 166 is not limited to the above, and various methods can be used. For example, this may be estimated from the striking direction and the initial speed at the time of impact, or may be estimated from the sound at the moment of striking.

For example, not only the image viewed from the player 50 but also the image viewed from the green 162 can be displayed on the image display device 20 by, for example, a switching signal of the player 50. That is, to perform this, the viewpoint position may be set on the green 162 when the viewpoint image calculation unit 76 performs the viewpoint conversion. By doing so, the player 50 actually makes the golf ball 166
2 and the golf ball 166 flying on the green 162 can be seen at the same time, and it is possible to enjoy a sense of play that cannot be obtained in actual golf play.

According to the present example having the above configuration, the player 50
Can enjoy golf in the play room 162 as if they were actually at a golf course. In particular,
In a game using such a ball, for example, golf, the player 50 needs to accurately recognize the moment when the golf ball 166 is shot by the club 164.
That is, if the club 164 and the golf ball 166 at the moment of this shot are created by, for example, image synthesis, the sense of play is significantly deteriorated, the sense of reality is reduced, and moreover, accurate sports play can be reproduced. Can not.

On the other hand, in this example , the golf club 1
64, the golf ball 166, the player's own swing, etc., can be seen from the real space image obtained through the image camera 10, so that such a problem does not occur,
It is possible to enjoy a realistic golf play. Therefore, for example, the virtual experience device for ball game practice is optimal.

The present invention is not limited to the above embodiment, and various modifications can be made within the scope of the present invention.

For example, in the present embodiment, the blue mat method is used as a method of synthesizing the virtual visual field image and the real space image, but the image synthesis in the present invention is not limited to this. For example, various methods can be used, such as a method using a color other than blue, a method using multiple colors and combining them to form an image, and a method using texture mapping.

Further, the virtual experience device to which the present invention is applied is not limited to the one shown in the present embodiment, but can be applied to various experience devices. For example, it can be applied to a helicopter flight simulator.

[0130]

[Brief description of the drawings]

1 is a block diagram showing a configuration of a first embodiment.

FIG. 2 is a schematic explanatory diagram showing a case where the first embodiment is applied to a driving game.

FIG. 3 is a schematic explanatory diagram showing a case where the first embodiment is applied to a flight simulator.

FIG. 4 is a schematic explanatory view showing a shape of a head mounted body.

FIG. 5 is a block diagram.

6 is a schematic explanatory diagram showing a case of application to b Lumpur playing game.

7 is a schematic explanatory diagram showing a case of applying the attraction overrided scheme.

8 is a schematic explanatory diagram showing a case of applying the attraction universe ship experience.

9 is a schematic explanatory diagram showing a case of applying the walking type attraction.

FIG. 10 is a block diagram.

11 is a schematic explanatory diagram showing a case of applying the rubber Ruff experience device.

FIG. 12 is a schematic explanatory diagram for describing coordinate conversion in the present embodiment.

FIG. 13 is a schematic block diagram illustrating an example of a configuration of a display image composition device.

FIG. 14 is a schematic explanatory diagram for describing a method of displaying an image on a side mirror.

FIG. 15 is a schematic explanatory diagram illustrating a method of attaching a real space image to a face of an opponent player by a texture mapping method.

FIG. 16 is a schematic block diagram illustrating an example of a configuration for realizing a texture mapping method.

[Explanation of symbols]

 Reference Signs List 9 head-mounted body 10 video camera 12 space sensor 20 image display device 38 operation unit 50 player 58 battle weapon 64 battle weapon sensor 68 game performance calculation unit 70 virtual view image calculation unit 72 virtual 3D space calculation unit 76 view image Calculation unit 80 Display image synthesizing device 84, 86 Ball image camera 88, 90 Moving object detecting device 100 Real space image 102 Virtual field-of-view image 104 Display image

Continuation of the front page (56) References JP-A-3-186287 (JP, A) JP-A-63-131882 (JP, A) JP-A-3-275092 (JP, A) JP-A-2-86285 (JP) , A) JP-A-4-241885 (JP, A) Michael Bajura, et al., "Maging Virtual Obj ects with the Real World": Seeing Ultrasound Imaging. 26, No. 2, p. 203-210 (58) Field surveyed (Int.Cl. ⁷ , DB name) A63F 13/00-13/12 A63G 31/04 G09B 9/00 G06T 15/00

Claims (5)

(57) [Claims] [Claim 1] A disposed toward the viewing direction of the operator
A virtual three-dimensional space calculating means for calculating a virtual three-dimensional space representing a game space set by superimposing the real three-dimensional space on the mirror member captured by the first imaging means; And arranging the virtual three-dimensional space in the real three-dimensional space as an image reflected on a mirror member captured by the first imaging unit.
A field image calculation means for image operations of the virtual field image as viewed from the mirror member which is, actual air as viewed from the mirror member disposed in said real three dimensional space
A second imaging unit that captures an interim video, and an actual image that is captured by the virtual view image and the second imaging unit.
An image is synthesized with the spatial image as a display image,
Display image synthesizing means for generating an image reflected on a mirror member
And an image synthesizing apparatus. 2. The method of claim 1, the mirror member disposed in the real three-dimensional space, the color of the image synthesis other than blue or blue is applied, the display image synthesis means, the first imaging the color data of the real space image of the imaged mirror member with means, said actual sky synthesizing the virtual field image
The empty dot of the interim image is specified, and the empty dot is assigned to the second dot.
Real space video image captured by the image capturing means and the virtual view image
And an image synthesizing apparatus. 3. A virtual experience device comprising the image synthesizing device according to claim 1 . And a mirror member formed as a side mirror or a rearview mirror is arranged, and a rider on which an operator rides; an operation unit provided on the rider, the operator inputs an operation signal; image display means mounted so as to cover the field of view, it is mounted on the operator, a first imaging means for imaging the real space image seen from the operator, a second imaging the real space image as viewed from the mirror member An operator's space sensor for detecting three-dimensional information of the operator in a real three-dimensional space; an operation signal from the operation unit; and a detection signal from the operator's space sensor. Calculating means for calculating a virtual three-dimensional space set by superimposing the virtual three-dimensional space on the basis of the virtual three-dimensional space, and performing image calculation of a virtual view image seen by an operator in the virtual three-dimensional space; The second shooting Anda display image combining means for image synthesis and real space image to be captured as a display image in an image unit, said computing means temporary <br/> within viewing from the mirror member disposed in said real three dimensional space comprising means for image operations of the virtual field image, the display image synthesis means, the image synthesizing a real space image captured by the said the virtual field image the second image pickup device as a display image, shooting the first
Means for generating a rear image reflected on a mirror member captured by the image means. 5. The mirror member according to claim 4, wherein the mirror member arranged in the real three-dimensional space has a blue color.
Alternatively, a color for image synthesis other than blue is applied, and the display image synthesis unit displays a real space image of the mirror member imaged by the first imaging unit.
The real sky which synthesizes the virtual view image from the color data of the image
The empty dot of the interim image is specified, and the empty dot is assigned to the second dot.
Real space video image captured by the image capturing means and the virtual view image
A virtual experience device characterized by combining images with the image.