JP2003296758A - Information processing method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable both an experiencing person and spectators to share information by presenting a compound realistic image to the experiencing person without failure and simultaneously presenting more information about the compound realistic image that experiencing person obtains to the spectators other than the experiencing person. <P>SOLUTION: First image data for generating a first image and second image data showing a second image are stored, the positional attitude of the experiencing person is measured, the first image is generated from the first image data in accordance with the measured positional attitude, the second image corresponding to the second image data is displayed on a second image display part, and a real video including the display image displayed on the second image display part is composited with the generated first image to be presented to the experiencing person. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION This invention relates to providing mixed reality.

[0002]

2. Description of the Related Art In recent years, research on mixed reality for the purpose of seamless connection between a physical space and a virtual space has become popular.

Mixed reality has been attracting attention as a technology for enhancing virtual reality for the purpose of coexistence of the world of virtual reality and the real space, which has been conventionally experienced only in a situation separated from the real space.

Applications of mixed reality include medical assistance for presenting to a doctor as if the patient's inside of the body is seen through, and work assistance for displaying the assembly procedure of the product on the actual product in a factory. There are expectations for new fields that are qualitatively completely different from the virtual reality up to now.

One of the methods for presenting this mixed reality is to use a head-mounted image display device. In this method, a real space image obtained by capturing the real space in real time with a camera or the like, or a mixed reality image in which a landscape of the real space observed through an optical system and an image created by a computer are combined, It is displayed on an image display device mounted on the head of a person who experiences reality (hereinafter referred to as “experienced person”).

By the way, there are situations where it is effective to present the mixed reality state not only to the experienced person but also to people other than the experienced person (hereinafter referred to as "spectators"). For example, in a situation in which an image synthesizer that presents mixed reality is used as an exhibition device, more people can see mixed reality by presenting the mixed reality image to spectators around the viewer. Since the content of the feeling can be grasped, the exhibition effect will be enhanced.

As a method of presenting mixed reality to the spectators, for example, a method of displaying a mixed reality image that the viewer sees on a display for the spectator, or observing from a position other than the viewpoint of the viewer. There is a method for displaying the mixed reality image as a mixed reality image from an objective viewpoint on the display. However, these methods include a defect that the position of the information presented to the experience person does not match the position of the information presented to the audience.

[0008] Further, Japanese Patent Laid-Open No. 5-2411 of Japanese Patent Laid-Open No.
By using the “image information synthesizing method and apparatus” disclosed in No. 23, it is possible to present mixed reality to not only the experienced person but also the spectator. This method and device is a technique for synthesizing a computer image created by a computer for CG creation into a real image taken by a TV camera, and a head-mounted image display device worn by the experience person and the sight line of the experience person. It is characterized in that the real image is displayed on one side and the computer image is displayed on the other side of the screen placed in the direction, and the two are optically combined using a half mirror. Using this technology, the television camera captures a real space in the direction of the viewer's line of sight, one of the image display devices displays a real image captured by the television camera, and the other screen displays a computer. If you display the image,
Since the spectator can observe both the computer image and the real space captured by the television camera, the viewer can grasp the mixed reality that is presented by the composite image of both. However, this method has the disadvantage that the audience can understand the mixed reality, while the experience person is presented with the dual real space in the direction of the observer's line of sight via the half mirror and the TV camera. include.

[0009]

SUMMARY OF THE INVENTION Therefore, the present invention has been made in view of the above situation, and its purpose is to present mixed reality without trouble to a person who uses the technique of the present invention. At the same time, it is to present more information about the mixed reality that the experience person has to the audience other than the mixed reality experience person, and share the information with the experience person and the audience.

[0010]

In order to achieve the above-mentioned object, the present invention holds first image data for generating a first image and second image data showing a second image, and an experience person Of the first image data is generated from the first image data according to the measured position and orientation, and a second image corresponding to the second image data is displayed on the second image display unit. The present invention is characterized by comprising combining a real video including a display image displayed on the second image display unit and the generated first image and presenting the synthesized image to a user.

[0011]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

<First Embodiment> In the first embodiment, the present invention is applied to an image synthesizing apparatus for presenting mixed reality by using a video see-through type image synthesizing method. FIG. 1 is a block diagram showing a schematic configuration of a video composition device according to the first embodiment.

A position / orientation measuring unit 1 is composed of, for example, a magnetic sensor (a magnetic transmitter fixed in the physical space and a magnetic tracker attached to a measurement target). The position / orientation measurement unit 1 measures the position and orientation of the physical space image input unit 5 in the physical space, and sends the position data and orientation data to the image generation unit 4.

Reference numerals 2 and 2'are data sections, which are composed of, for example, hard disks, hold data to be passed to the image generating section 4 and the image generating section 4 ', respectively, and are supplied from the image generating section 4 and the image generating section 4', respectively. In response to the request, appropriate data is provided to the image generation unit 4 and the image generation unit 4 ′.
Send to. The data stored in the data part 2 and the data part 2'is, for example, text data, three-dimensional CG (computer graphics) data, a two-dimensional image, or a real moving image.

Although the data section 2 and the data section 2'are shown as different in FIG. 1, they may be integrated.

The data section 2 and the data section 2'are not limited to hard disks, and may be any medium capable of holding data, such as tape or memory.

Reference numeral 3 denotes an interactive operation input unit, which is composed of, for example, a keyboard, a mouse, a 6-DOF position / orientation sensor, and the like. The interactive operation input unit 3 is operated by a user, acquires the operation information as operation data, and sends this operation data to the image generation unit 4 and the image generation unit 4 ′. This interactive operation input unit 3 is not essential. Further, although FIG. 1 shows one dialogue operation input unit, a plurality of dialogue operation input units may be provided.

Image generators 4 and 4'are, for example, computers.

The image generation unit 4 obtains the position data and the posture data of the photographing viewpoint of the real space image input unit 5 in the real space from the position data and the posture data of the real space image input unit 5 received from the position and posture measurement unit 1. calculate. The image generation unit 4 also generates image data based on the data held in the data unit 2 according to the position data and the posture data. At that time, when the operation data is transmitted from the interactive operation input unit 3, the image data is generated so as to comply with the operation data. This image data is sent to the image synthesizing unit 6. Hereinafter, this image data will be referred to as “image data A”, and an image generated from this will be referred to as “image A”.

The image generating section 4'generates image data based on the data held in the data section 2 '. At that time, when the operation data is transmitted from the interactive operation input unit 3, the image data is generated so as to comply with the operation data.
This image data is sent to the image projection unit 8. Hereinafter, this image data will be referred to as “image data A ′”, and an image generated from this will be referred to as “image A ′”.

In FIG. 1, an image generating section 4 and an image generating section 4 '.
Are shown as different, but as shown in FIG.
These may be integrated.

Reference numeral 5 is a physical space image input section, which comprises, for example, a camera and a video capture card. The physical space image input unit 5 captures an image of the physical space in the viewer's line-of-sight direction. The physical space image input unit 5 also converts the physical space image into a format suitable for the image composition unit 6 and sends it to the image composition unit 6 as physical space image data.

Reference numeral 6 is an image synthesizing unit, which is composed of, for example, a computer. The image synthesizing unit 6 synthesizes the image data A sent from the image generating unit 4 and the physical space image data sent from the physical space image input unit 5 to create synthetic image data. The image composition unit 6 also sends the composite image data to the image display unit 7. Here, the image can be stereoscopically viewed by creating the image for the right eye and the image for the left eye.

An image display unit 7 is composed of, for example, a head-mounted liquid crystal display. The image display unit 7 displays the combined image data sent from the image combining unit 6 as an image. The display light of the composite image enters the observer's viewpoint.

Reference numeral 8 denotes an image projection unit, which is composed of, for example, a video projector. The image projection unit 8 projects the image data A ′ sent from the image generation unit 4 on the image projection surface 9 as an image (image A ′).

Reference numeral 9 denotes an image projection plane, which is composed of, for example, a screen or a real object whose surface is white and plain. Image projection plane 9
The image light of the image A ′ is projected from the image projection unit 8. The image projection plane 9 exists in the physical space captured by the physical space image input unit 5.

One of the features of this embodiment is that an image to be presented to the experience person is displayed on the image display section 7 which is a head-mounted liquid crystal display, and that it exists in the "real space in front of the eyes". It is divided into those displayed on the screen (image projection surface 9).

As a reference for dividing the image displayed on the image display unit and the image displayed on the image projection surface, the distance from the experience person can be considered. The peripheral part close to the experiencer is displayed on the image display unit for three-dimensional presentation in order to realize a more elaborate simulation. On the other hand, an image such as a background far from the experience person does not have a large change in the image due to movement of the viewpoint of the experience person, so that there are few problems even if it is displayed two-dimensionally. Therefore, a two-dimensional image such as a real moving image is used.

For example, a case where the present embodiment is applied to a driving simulator will be described. For example, the scenery from the car window is presented on the image projection surface using a real moving image. Audiences other than the experienced person can also know what kind of virtual space the audience is doing in the driving simulation. In addition, by providing a three-dimensional presentation using the image display unit for the surroundings of the experience person, such as the interior of a car, a higher simulation effect can be given to the experience person.

Further, whether or not stereoscopic viewing is required for presenting an image may be used as a reference. For example, in a driving simulator, the steering wheel, buttons, levers, etc. are displayed on the image display unit in order to present them three-dimensionally, and the two-dimensional objects such as speedometer and car navigation screen and the background are projected in two-dimensional form. Display on the surface.

As described above, in the present embodiment, the image displayed on the image display unit is presented three-dimensionally and is presented two-dimensionally on the image projection plane.

According to the present embodiment, since the image is displayed on the image projection surface 9, the spectator can see the image, and not only the experience person but also the spectator can know the state of the MR space. Further, the experience person can experience by combining the image displayed on the image projection surface 9 and the three-dimensionally presented elaborate stereoscopic image.

The control of this embodiment having the above configuration will be described below. FIG. 3 is a flowchart illustrating the procedure of processing according to the first embodiment.

In step 0, the system boots.

In step S10, the position and orientation measuring unit 1
Measures the position and orientation of the physical space image input unit 5,
The measured position data and posture data are sent to the image generation unit 4.

In step S20, the interactive operation input unit 3
However, the interactive operation input processing is performed, and when there is an operation by the experiencer, the operation data is sent to the image generating unit 4 and the image generating unit 4 ′. If the interactive operation input unit 3 does not exist, this step is not performed.

In step S30, the image generation unit 4 receives the position data and orientation data of the physical space image input unit 5 from the position and orientation measurement unit 1, and from the data, the photographing viewpoint of the physical space image input unit 5 in the physical space. Position data and attitude data of the.

In step S31, the image generation section 4 determines the data section 2 according to the position data and orientation data of the photographing viewpoint of the physical space image input section 5 calculated in step S30.
The image data A is generated using the data held in. In addition, when the operation data is sent from the interactive operation input unit 3, the image generation unit 4 also generates the image data A so as to comply with the operation data. For example, the position of the three-dimensional virtual object presented by the image data A is changed according to the operation data.

In step S31, the image generation unit 4'generates image data A'using the data held in the data unit 2 '. Further, when the operation data is transmitted from the interactive operation input unit 3, the image generation unit 4 ′,
The image data A ′ is generated so as to correspond to the operation data. For example, the actual moving image presented by the image data A ′ is reproduced or rewound according to the operation data.

In step S32, the image generator 4 sends the image data A to the image synthesizer 6, and the image generator 4'sends the image data A'to the image projector 8.

In step S40, the image projection unit 8 projects the image data A'on the image projection plane 9. As a result, the image A ′ is displayed on the image projection surface 9. At this point, the image A'is presented directly to the spectators around the viewer.

In step S50, the physical space image input section 5 captures an image of the physical space in the viewer's line of sight and creates physical space image data from the obtained physical space image. Since the image projection plane 9 exists in the real space imaged by the real space image input unit 5, the person facing the image projection plane 9 faces the image projection plane 9 within the photographing range of the real space image input unit 5.
In the case where is included, the physical space image captured by the physical space image input unit 5 includes the image A ′ projected on the image projection surface 9. This physical space image data is sent to the image synthesis unit 6.

In step S60, the image synthesizing unit 6 synthesizes the image data A received from the image generating unit 4 and the physical space image data received from the physical space image input 5 to generate synthetic image data. This composite image data is sent to the image display unit 7.

In step S70, the image display section 7 displays the composite image data received from the image composition section 6 as a composite image. This composite image is presented to the experience person. Here, since the composite image data is the composite of the image data A and the physical space image data, the physical space image obtained by capturing the physical space and the image A are composited to the experience person in this step S70. Mixed reality will be presented.

By the way, as described above, when the experience person faces the direction in which the image projection plane 9 is located and the image projection plane 9 is within the photographing range of the real space image input unit 5,
The physical space image captured by the physical space image input unit 5 includes the image A ′ projected on the image projection surface 9. Then, in such a case, the image A ′ is also presented to the experience person in step S70. This image A'is also directly presented to the spectator in step S40, so that the image A'is shared between the spectator and the spectator. Since this image A'is a part of the image that constitutes the mixed reality presented to the experience person, the experience person and the spectator eventually share a part of the information about the mixed reality that the experience person obtains. Becomes Moreover, the display positions of the image A'recognized by both are the same. In Figure 4,
The figure explaining the mechanism where an experience person and an audience share a part of information about mixed reality.

In step S80, it is checked whether or not the system is to be terminated. If the system is to be terminated, the system is terminated in step S90.
The process returns to 0 and the above process is repeated.

As described above, according to the first embodiment,
The experience person is presented with mixed reality in which a real space image obtained by photographing a real space in the sight line direction of the experience person and the image A are combined. Then, when the experience person faces the direction in which the image projection plane 9 is located and the image projection plane 9 falls within the photographing range of the real space image input unit 5, the real space photographed in the real space in the sight line direction of the experience person. Mixed reality in which the image, the image A, and the image A ′ are combined is presented without any trouble. At the same time, the scenery and the image A ′ in the real space around the experiencer are presented to the audience at the same position as the position recognized by the experiencer. That is, a part of the image presented to the experience person is also presented to the audience, and the experience person and the audience can share part of the information about the mixed reality that the experience person has, including the position. To be done.

Further, the image displayed on the image projection surface and the image displayed on the image display section may be interlocked with each other so that a higher simulation effect can be given.

For example, by using a timetable, a predetermined image can be displayed on the image display section at a predetermined timing to deal with the problem. Further, it can be realized by changing the image displayed on the image display unit and the image displayed on the image projection surface according to an instruction from the interactive operation input unit.

<Second Embodiment> Next, a second embodiment will be described. The second embodiment is applied to a video synthesizing device that presents mixed reality using an image see-through type image synthesizing method. FIG. 5 is a block diagram showing a schematic configuration of a video composition device according to the second embodiment. The same components as those of the image synthesizer (FIG. 1) according to the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

A position / orientation measuring unit 1 is composed of, for example, a magnetic sensor (a magnetic transmitter fixed in the physical space and a magnetic tracker attached to a measurement target). The position / orientation measurement unit 1 measures the position and orientation of the head of the experience person in the physical space, and sends the position data and orientation data to the image generation unit 4.

Reference numerals 2 and 2'denotes data sections, which are composed of, for example, hard disks, hold data to be passed to the image generating section 4 and the image generating section 4 ', respectively, and are supplied from the image generating section 4 and the image generating section 4', respectively. In response to the request, appropriate data is provided to the image generation unit 4 and the image generation unit 4 ′.
Send to. The data stored in the data part 2 and the data part 2'is, for example, text data, three-dimensional CG (computer graphics) data, a two-dimensional image, or a real moving image.

Although the data part 2 and the data part 2'are shown as different in FIG. 5, they may be integrated.

Reference numeral 3 denotes an interactive operation input unit, which is composed of, for example, a keyboard, a mouse, a 6-DOF position / orientation sensor, and the like. The interactive operation input unit 3 is operated by a user, acquires the operation information as operation data, and sends this operation data to the image generation unit 4 and the image generation unit 4 ′. This interactive operation input unit 3 is not essential. Further, although FIG. 5 shows one dialogue operation input unit, a plurality of dialogue operation input units may be provided.

Image generation units 4 and 4'are, for example, computers.

The image generation unit 4 calculates the position data and posture data of the experience point of the user in the real space from the position data and posture data of the experience person's head received from the position and posture measurement unit 1. The image generation unit 4 also generates image data based on the data held in the data unit 2 according to the position data and the posture data. At that time, when the operation data is transmitted from the interactive operation input unit 3, the image data is generated so as to comply with the operation data.
The image data may be, for example, character data describing the scenery in front of the eyes, or three-dimensional CG data used to simulate a scene in which a virtual building is built in the scenery in front of the eyes. . This image data is displayed on the image display unit 7.
Sent to. Hereinafter, this image data will be referred to as “image data B”, and the image generated from this will be referred to as “image B”.

The image generating section 4'generates image data based on the data held in the data section 2 '. At that time, when the operation data is transmitted from the interactive operation input unit 3, the image data is generated so as to comply with the operation data.
This image data may be, for example, real moving image data that is the background of the mixed reality space, or image data that simulates the surface of a real object. This image data is sent to the image projection unit 8. Hereinafter, this image data will be referred to as “image data B ′”, and an image generated from this will be referred to as “image B ′”.

In FIG. 5, the image generator 4 and the image generator 4 '.
, Are shown as different, but as shown in FIG.
These may be integrated.

An image display unit 7 is composed of, for example, a head-mounted liquid crystal display. The image display unit 7 converts the image data B sent from the image generation unit 4 into an image (image B).
Display as. The display light of this image enters the optical image combining unit 10.

Reference numeral 8 denotes an image projection unit, which is composed of, for example, a video projector. The image projection unit 8 projects the image data B ′ sent from the image generation unit 4 on the image projection surface 9 as an image (image B ′).

Reference numeral 9 denotes an image projection plane, which is composed of, for example, a screen or a real object whose surface is white and plain. Image projection plane 9
The image light of the image B ′ projected from the image projection unit 8 is projected on the screen. This image projection plane 9 exists at a position that can be seen by the experience person.

Reference numeral 10 denotes an optical image synthesizing section, which is composed of, for example, a half mirror. The optical image synthesizing unit 10 displays the display light of the image (image B) displayed by the image display unit 7 and the light in the real space incident from the direction of the line of sight of the experience person when the experience person wears the image display unit 7. To synthesize. This combined light enters the viewpoint of the experience person.

The control of this embodiment having the above configuration will be described below. FIG. 7 is a flowchart illustrating a processing procedure in the embodiment. The same reference numerals are given to the same controls as those of the video synthesizing apparatus according to the first embodiment (FIG. 3), and detailed description thereof will be omitted.

In step 0, the system starts up.

In step S10, the position / orientation measuring unit 1
Measures the position and posture of the head of the experience person and sends the measured position data and posture data to the image generation unit 4.

In step S20, the interactive operation input unit 3
However, the interactive operation input processing is performed, and when there is an operation by the experiencer, the operation data is sent to the image generating unit 4 and the image generating unit 4 ′. If the interactive operation input unit 3 does not exist, this step is not performed.

In step S30, the image generation unit 4 receives the position data and posture data of the head of the experience person from the position and posture measurement unit 1, and calculates the position data and posture data of the experience point of the user in the real space from the data. To do.

In step S31, the image generation unit 4 generates the image data B using the data held in the data unit 2 according to the position data and posture data of the experience point of view calculated in step S30. In addition, the image generation unit 4
When the operation data is transmitted from the interactive operation input unit 3, generates the image data B so as to comply with the operation data. For example, 3 presented by image data B
The position of the dimensional virtual object is changed according to the operation data.

In step S31, the image generation unit 4'uses the data held in the data unit 2'to generate image data B '. Further, when the operation data is transmitted from the interactive operation input unit 3, the image generation unit 4 ′,
The image data B ′ is generated so as to correspond to the operation data. For example, the actual moving image presented by the image data B ′ is reproduced or rewound according to the operation data.

In step S32, the image generation unit 4 sends the image data B to the image display unit 7, and the image generation unit 4 ′ sends the image data B ′ to the image projection unit 8.

In step S40, the image projection unit 8 projects the image data B'on the image projection plane 9. As a result, the image B ′ is displayed on the image projection surface 9. At this point, the image B'is presented directly to the spectators around the viewer.

In step S70, the image display unit 7 displays the image data B received from the image generation unit 4 as an image. The display light of the image B enters the optical image synthesizer 10.

In step S71, the optical image synthesizing device 10 synthesizes the display light of the image displayed by the image display unit 7 and the light of the physical space which is incident from the direction of the line of sight of the experience person. This synthesized light is incident on the viewpoint of the experience person. The display light of the image displayed by the image display unit 7 is the image B, and the light of the real space is the landscape of the real space. Therefore, in this step S71, the experienced person is presented with mixed reality in which the image B and the scenery in the real space are combined.

By the way, when the experience person faces the direction in which the image projection plane 9 is located and the image projection plane 9 is within the visual field range of the experience person, in the landscape of the real space seen by the experience person, The image B ′ projected on the image projection plane 9 is included. Then, in such a case, step S71
In, the image B ′ is also presented to the experience person.
This image B'is also directly presented to the spectator in step S40, so that the image B'is shared between the experience person and the spectator. Since this image is a part of the image that constitutes the mixed reality presented to the experience person, the experience person and the audience will eventually share a part of the information about the mixed reality that the experience person obtains. . Moreover, the display positions of the image B ′ recognized by both are the same.

In step S80, it is checked whether or not the system is to be terminated. If it is to be terminated, the system is terminated in S90, and if it is not terminated, step S1
The process returns to 0 and the above process is repeated.

As described above, according to the second embodiment,
For the experience person, the scenery and image B of the real space in the direction of the sight line of the experience person
Mixed reality that is composed is presented. Then, when the experience person faces the direction in which the image projection plane 9 is located and the image projection plane 9 falls within the visual field range of the experience person, the scenery of the real space in the direction of the sight line of the experience person, the image B and the image. Mixed reality in which an image composed of B ′ is combined is presented without any trouble.
At the same time, the viewer is presented with the scenery of the real space around the experiencer and the image B ′ at the same position as the position recognized by the experiencer. That is, a part of the image presented to the experience person is also presented to the audience, and the experience person and the audience can share part of the information about the mixed reality that the experience person has, including the position. To be done.

<Other Embodiments> The object of the present invention is to supply a storage medium (or recording medium) recording a program code of software for realizing the functions of the above-described embodiments to a system or apparatus, and It is also achieved by the computer (or CPU or MPU) of the device reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instructions of the program code.
Needless to say, this also includes the case where the above-mentioned processes perform part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion card inserted in the computer or the function expansion unit connected to the computer, based on the instruction of the program code, It goes without saying that a case where the CPU included in the function expansion card or the function expansion unit performs a part or all of the actual processing and the processing realizes the functions of the above-described embodiments is also included.

[0079]

As described above, according to the present invention,
Presents mixed reality to the person who uses the technique of the present invention without any trouble, and at the same time, presents more information about the mixed reality that the experience person has to the audience other than the mixed reality experience person. Then, the information can be shared between the experience person and the audience. Further, this sharing is not limited to information sharing only of visual contents, but it is possible to share information including a position where mixed reality is presented.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a video composition device according to a first embodiment.

FIG. 2 is another block diagram showing a schematic configuration of a video composition device according to the first embodiment.

FIG. 3 is a flowchart illustrating a procedure of processing according to the first embodiment.

FIG. 4 is a diagram illustrating a mechanism in which an experience person and an audience share a part of information about mixed reality.

FIG. 5 is a block diagram showing a schematic configuration of a video composition device according to a second embodiment.

FIG. 6 is another block diagram showing a schematic configuration of a video composition device according to a second embodiment.

FIG. 7 is a flowchart illustrating a procedure of processing according to the second embodiment.

Claims (10)

[Claims] 1. Holding first image data for generating a first image and second image data showing a second image, measuring the position and orientation of an experience person, and depending on the measured position and orientation. A first image is generated from the first image data, a second image corresponding to the second image data is displayed on a second image display unit, and a real image including a display image displayed on the second image display unit is displayed. An information processing method comprising: combining a video and the generated first image and presenting the same to a user. 2. The first image data is data for generating a computer graphic image, and the first image data is
The information processing method according to claim 1, wherein the image is a stereoscopically viewed image. 3. The information processing method according to claim 1, wherein the second image data is moving image data. 4. The information processing method according to claim 1, wherein the first image is a three-dimensionally presented image, and the second image is a two-dimensionally presented image. 5. The information processing method according to claim 1, further comprising the step of inputting an operation from a user and generating the first image and the second image according to the operation. 6. A program for realizing the information processing method according to any one of claims 1 to 5. 7. Holding means for holding the first image data for generating the first image and second image data showing the second image, measuring means for measuring the position and orientation of the experiencer, and the measured value. A first image generation unit that generates a first image from the first image data according to the position and orientation; a second image display unit that displays a second image corresponding to the second image data; Information comprising a real image including a display image displayed on the image display unit and a combined image display unit for combining the first image generated by the first image generation unit and presenting the combined image to the user. Processing equipment. 8. The information processing apparatus according to claim 7, wherein the composite image display means is a head-mounted image display apparatus. 9. The information processing apparatus according to claim 7, wherein the composite image display means is a video see-through system. 10. The information processing apparatus according to claim 7, wherein the composite image display means is an optical see-through system.