JP2016038632A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology to provide an environment for observing a virtual object considering a movable range for an experiencing person to obtain mixed reality.SOLUTION: A system includes: an optical three-dimensional position and attitude measurement device 1300 for measuring the three-dimensional position and attitude of the line of sight of a user; an HMD 1200 that is a head-mounted display device mounted on the head of the user; an image processing apparatus 1000 that creates an image in a mixed reality space; and a display device 1100 that displays the image of the mixed reality space. The system determines a range where the position and attitude of the line of sight can be measured, arranges a virtual object in the range, creates an image of a virtual space including the arranged virtual object according to the position and attitude, and outputs the created image of the virtual space.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for presenting mixed reality.

  In recent years, research on mixed reality (MR) aimed at seamless connection between a real space and a virtual space has been actively conducted. An image display device that presents mixed reality is, for example, a device having the following configuration. That is, an image of a virtual space (for example, a virtual object or character information drawn by computer graphics) generated according to the position and orientation of the imaging device is superimposed on an image of the real space captured by the imaging device such as a video camera. It is a device that displays a drawn image. For such an apparatus, for example, an HMD (head mounted display, head mounted display) can be used.

  As a method for measuring the position and orientation of the imaging device, there is a method using a three-dimensional position and orientation measurement device (for example, an optical three-dimensional position and orientation measurement device VICON manufactured by VICON). This measurement apparatus measures the three-dimensional position and orientation of an index in real space by photographing an index attached to an object whose position and orientation are to be measured from a plurality of cameras. However, the three-dimensional position and orientation of the imaging apparatus can be measured by fixing the index to the imaging apparatus.

  On the other hand, when creating a virtual object, commercially available CAD (Computer Aided Design) or 3DCG software (three-dimensional computer graphics software) is generally used (Non-patent Document 1).

  By combining the above technologies, it is possible to experience a combination of a virtual reality space or a mixed reality space using a virtual object created using CAD or 3DCG software.

Works Corporation Co., Ltd., CG & Video Structure Dictionary, issued December 31, 2003, pp. 98-101

  Experienced persons experiencing mixed reality may approach the virtual object and observe the virtual object in detail, or conversely, move away from the virtual object and view the virtual object from a distance. Is always required to be within the measurable range of the position and orientation of the viewpoint, so the virtual object is preferably arranged at a position that takes into account the measurable range. However, it is difficult for a user who creates a virtual object using CAD or 3DCG software to set the arrangement position in consideration of the measurable range.

  The present invention has been made in view of such a problem, and provides a technique for providing a virtual object observation environment in consideration of a range in which an experiencer can move in order to obtain mixed reality.

  According to one aspect of the present invention, a calculation unit that obtains a measurable range of the position and orientation of a viewpoint, an arrangement unit that arranges a virtual object in the range, and an image of a virtual space that includes the virtual object arranged by the arrangement unit. And generating means for generating the image according to the position and orientation, and output means for outputting an image of the virtual space generated by the generating means.

  According to the configuration of the present invention, it is possible to provide an observation environment for a virtual object in consideration of a range in which an experiencer can move in order to obtain mixed reality.

The figure which shows the structural example of a system. The block diagram which shows the hardware structural example of a computer. 6 is a flowchart for explaining the operation of the image processing apparatus 1000. The figure explaining the process in step S404. The figure explaining the process in step S405. The figure explaining 2nd Embodiment.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
A configuration example of a system that allows a user (experience person) to experience a mixed reality space obtained by synthesizing a virtual space and a real space (presenting a mixed reality) will be described with reference to FIG. As shown in FIG. 1, the system according to the present embodiment includes an optical three-dimensional position / orientation measuring apparatus 1300 for measuring the three-dimensional position / orientation of a user's viewpoint, and a head-mounted display apparatus that is mounted on the user's head. HMD 1200 which is an example, an image processing apparatus 1000 that generates an image of mixed reality space, and a display apparatus 1100 that displays an image of mixed reality space.

  First, the HMD 1200 will be described.

  The imaging devices 1220R and 1220L are for capturing moving images in real space, and are moving images for presenting the right eye of a user wearing the HMD 1200 on the head and moving images for presenting to the left eye, respectively. An image is taken. The images (captured images) of the frames constituting the moving images captured by the image capturing devices 1220R and 1220L are sequentially transmitted to the image processing device 1000.

  The display devices 1210R and 1210L are for displaying the image output from the image processing device 1000, and are positioned in front of the right eye and the left eye of the user wearing the HMD 1200 on the head, respectively. Attached to the HMD 1200.

  The marker 1230 is attached to measure the position and orientation of the HMD 1200, and is an object to be imaged by the optical three-dimensional position and orientation measurement apparatus 1300 described later.

  Note that communication between the HMD 1200 and the image processing apparatus 1000 may be wired communication, wireless communication, or a combination thereof.

  Next, the optical three-dimensional position / orientation measurement apparatus 1300 will be described.

  The measurement imaging devices (optical sensors) 1320A and 1320B both capture the marker 1230 and are fixedly arranged in the real space. The imaging results from the measurement imaging devices 1320A and 1320B are sent to the controller 1310. The controller 1310 controls the operation of the measurement imaging devices 1320A and 1320B, and sends the imaging results from the measurement imaging devices 1320A and 1320B to the image processing apparatus 1000. In FIG. 1, the number of measurement imaging devices is two, but may be three or more. In FIG. 1, the measurement imaging devices 1320A and 1320B and the controller 1310 are separate devices, but the controller 1310 may be incorporated in each of the measurement imaging devices 1320A and 1320B.

  Next, the image processing apparatus 1000 will be described.

  The viewpoint information measurement unit 1010 uses a world coordinate system 1500 (one point in the real space as an origin and three axes orthogonal to each other at the origin based on the imaging results by the measurement imaging devices 1320A and 1320B sent from the controller 1310. The position and orientation of the marker 1230 in the X-axis, Y-axis, and Z-axis coordinate system) are obtained. Since the method for obtaining the position and orientation of the marker 1230 in the world coordinate system 1500 from the imaging results of the measurement imaging devices 1320A and 1320B is a known technique, description thereof will be omitted. In the following description, as the world coordinate system 1500, the floor surface is defined as the XZ plane, and the upward direction with respect to the floor surface is defined as the Y axis. It doesn't matter. Then, the viewpoint information measurement unit 1010 stores the position and orientation of the marker 1230 in the world coordinate system 1500 in the data storage unit 1040.

  The image acquisition unit 1020 stores the image of each frame sent from the HMD 1200 (the imaging devices 1220R and 1220L) in the data storage unit 1040.

  The input unit 1070 is a user interface that is operated by a user of the image processing apparatus 1000 to input various instructions to the image processing apparatus 1000, and includes a mouse, a keyboard, and the like.

  The mode setting unit 1080 is a range in which a virtual object (virtual object 1400 in FIG. 1) can be imaged by the measurement imaging devices 1320A and 1320B (that is, a range in which the position and orientation of the marker 1230 can be measured. Mode (first mode) arranged in a possible area), a mode (second mode) arranged in an arrangement position where the virtual space data stored in the data storage unit 1040 defines the virtual object Mode). This mode is set according to an instruction from the input unit 1070. The mode setting unit 1080 stores the set mode in the data storage unit 1040.

  The virtual object moving unit 1030 operates when the first mode is set by the mode setting unit 1080. The virtual object moving unit 1030 uses the information such as “the position and orientation and the angle of view of each of the measurement imaging devices 1320A and 1320B in the world coordinate system 1500” stored in advance in the data storage unit 1040. An area where the respective visual volumes of 1320A and 1320B overlap and which is closer to the measurement imaging devices 1320A and 1320B than the XZ plane is obtained as the above-described measurable area. Then, the virtual object moving unit 1030 obtains a position where the virtual object is arranged within the measurable area.

  The virtual space generation unit 1050 constructs a virtual object using the virtual space data stored in the data storage unit 1040. The virtual space data includes data for defining a virtual space such as data defining a light source arranged in the virtual space and data defining a virtual object arranged in the virtual space.

  Then, when the second mode is set by the mode setting unit 1080, the virtual space generation unit 1050 places the constructed virtual object at a position defined by the virtual space data for the virtual object.

  On the other hand, when the first mode is set by the mode setting unit 1080, the virtual space generation unit 1050 arranges the constructed virtual object at the arrangement position obtained by the virtual object moving unit 1030.

  Note that the posture in which the virtual object is arranged may be any posture, may be a posture defined by the virtual space data, or may be appropriately changed by the user operating the input unit 1070. It doesn't matter.

  On the other hand, the virtual space generation unit 1050 stores “the position and orientation of the marker 1230 in the world coordinate system 1500” stored in the data storage unit 1040 and the “marker 1230” measured in advance and stored in the data storage unit 1040. By performing a well-known calculation using the “relative position / posture relationship between the image pickup device 1220R” and the image pickup device 1220R, the “position / posture of the image pickup device 1220R in the world coordinate system” is obtained as the position / posture of the right-eye viewpoint. . Similarly, the virtual space generation unit 1050 includes “the position and orientation of the marker 1230 in the world coordinate system 1500” stored in the data storage unit 1040 and “marker 1230 which is measured in advance and stored in the data storage unit 1040. By performing a well-known calculation using the “relative position / posture relationship between the image pickup device 1220L” and the image pickup device 1220L, the “position / posture of the image pickup device 1220L in the world coordinate system” is obtained as the position / posture of the left-eye viewpoint. .

  Then, the virtual space generation unit 1050 obtains an image (right-eye virtual space image) obtained by viewing the virtual space in which the virtual object is arranged as described above from the right-eye viewpoint, and the position and orientation of the virtual space data and the right-eye viewpoint. , To generate. Similarly, the virtual space generation unit 1050 obtains an image (left-eye virtual space image) obtained by viewing the virtual space in which the virtual object is arranged as described above from the left-eye viewpoint, and the position of the virtual space data and the left-eye viewpoint. And using the posture. Note that a series of processing for generating an image of a virtual space that can be seen from a viewpoint having a certain position and orientation is a well-known technique, and thus further detailed description thereof will be omitted.

  The virtual space generation unit 1050 sends each of the right eye virtual space image and the left eye virtual space image to the image generation unit 1060.

  The image generation unit 1060 combines the image captured by the imaging device 1220R (the real space image for the right eye) stored in the data storage unit 1040 and the virtual space image for the right eye sent from the virtual space generation unit 1050. The image is generated as a mixed reality space image for the right eye. Similarly, the image generation unit 1060 includes a captured image (a left-eye real space image) stored in the data storage unit 1040 and a left-eye virtual space image sent from the virtual space generation unit 1050. The synthesized composite image is generated as an image of the mixed reality space for the left eye. Then, the image generation unit 1060 sends the mixed reality space image for the right eye to the display device 1210R of the HMD 1200, and sends the mixed reality space image for the left eye to the display device 1210L of the HMD 1200.

  Accordingly, an image of the mixed reality space for the left eye is presented in front of the left eye of the user wearing the HMD 1200 on his / her head, and an image of the mixed reality space for the right eye is presented in front of the right eye. It will be. Note that the image generation unit 1060 may further output both or one of the mixed reality space image for the right eye and the mixed reality space image for the left eye to the display device 1100.

  Next, the operation of the image processing apparatus 1000 will be described using the flowchart of FIG. Note that the flowchart of FIG. 3 is a flowchart of the process of generating and outputting the mixed reality space image for the right eye for one frame and the mixed reality space image for the left eye for one frame. The image processing apparatus 1000 performs processing according to the flowchart of FIG. 3 for each frame input from the HMD 1200.

<Step S400>
The viewpoint information measurement unit 1010 obtains the position and orientation of the marker 1230 in the world coordinate system 1500 from the imaging results by the measurement imaging devices 1320A and 1320B sent from the controller 1310. Then, the viewpoint information measurement unit 1010 stores the position and orientation of the marker 1230 in the world coordinate system 1500 in the data storage unit 1040.

<Step S401>
The image acquisition unit 1020 stores (acquires) the right-eye real space image sent from the imaging device 1220R and the left-eye real space image sent from the imaging device 1220L in the data storage unit 1040.

<Step S402>
If the mode setting unit 1080 sets the first mode, the process proceeds to step S404 via step S402. If the second mode is set, the process proceeds to step S402. The process proceeds to S403.

<Step S403>
The virtual space generation unit 1050 constructs a virtual object using the virtual space data stored in the data storage unit 1040, and arranges the constructed virtual object at a position defined by the virtual space data for the virtual object. .

<Step S404>
The virtual object moving unit 1030 uses the information such as “the position and orientation and the angle of view of each of the measurement imaging devices 1320A and 1320B in the world coordinate system 1500” stored in advance in the data storage unit 1040. In the region where the respective visual volumes of 1320A and 1320B overlap, the region closer to the measurement imaging devices 1320A and 1320B than the XZ plane is obtained as the measurable region. In FIG. 4, the visual volumes of the measurement imaging devices 1320A and 1320B are indicated by dotted lines, and the region 4000 where the respective visual volumes overlap is indicated by diagonal lines.

<Step S405>
The virtual object moving unit 1030 obtains a position where the virtual object is arranged in the measurable area obtained in step S404. The virtual object may be arranged at any position within the measurable area (within the range), but an example thereof will be described below. Hereinafter, a case where the area 4000 of FIG. 4 is obtained as the measurable area will be described as an example.

  First, a sphere that includes the region 4000 is obtained. Since the region 4000 is defined by the intersection point group of cones representing the visual volumes of the measurement imaging devices 1320A and 1320B, for example, the smallest sphere that includes this intersection point group is obtained. Since the algorithm for the minimum inclusion sphere for obtaining such a sphere is a well-known technique, a detailed description thereof will be omitted. A position obtained by projecting the center position of the sphere onto the XZ plane of the world coordinate system 1500 is obtained as “the position of the bottom surface of the bounding box that includes the virtual object”. In FIG. 5, a sphere 5000 is a sphere including a region 4000, and the center position is indicated by a point 5010. A position where the point 5010 is projected on the XZ plane is indicated by a point 5020. In this case, however, the center position of the bottom surface (quadrangle) of the bounding box that contains the virtual object is matched with the position indicated by the point 5020, whereby the inside of the bounding box having the bottom surface centered on the position indicated by the point 5020. A virtual object will be placed on the screen. Of course, the projection destination of the center position is not limited to the XZ plane, and the center position of the virtual object may be aligned with the center position of the sphere 5000. In this way, the virtual object moving unit 1030 determines the position where the virtual object is to be placed.

<Step S406>
The virtual space generation unit 1050 arranges the virtual object based on the position obtained by the virtual object moving unit 1030 in step S405.

<Step S407>
The virtual space generation unit 1050 obtains the position and orientation of the right-eye viewpoint and the position and orientation of the left-eye viewpoint by the above method. The virtual space generation unit 1050 generates an image (right-eye virtual space image) in which the virtual space in which the virtual object is arranged as described above is viewed from the right-eye viewpoint, and the virtual space is viewed from the left-eye viewpoint. An image (left-eye virtual space image) is generated. Then, the virtual space generation unit 1050 sends each of the right eye virtual space image and the left eye virtual space image to the image generation unit 1060.

<Step S408>
The image generation unit 1060 generates a composite image obtained by combining the right-eye real space image stored in the data storage unit 1040 in step S401 and the right-eye virtual space image generated by the virtual space generation unit 1050 in step S407. It is generated as an image of the mixed reality space. Similarly, the image generation unit 1060 combines a composite image obtained by combining the left-eye real space image stored in the data storage unit 1040 in step S401 and the left-eye virtual space image generated by the virtual space generation unit 1050 in step S407. And generated as an image of the mixed reality space for the left eye.

<Step S409>
The image generation unit 1060 sends the mixed reality space image for the right eye generated in step S408 to the display device 1210R of the HMD 1200, and the mixed reality space image for the left eye generated in step S408 of the HMD 1200. Send to display device 1210L.

  As described above, according to the present embodiment, for example, when determining the arrangement position of the virtual object in the creation stage of the virtual space data, the mixed reality can be obtained without knowing the measurable area in advance. At the presentation stage, the arrangement position can be set to an arrangement position that the user can observe closer.

[Second Embodiment]
In the first embodiment, the case where the number of virtual objects to be arranged is 1 has been described. However, even when there are a plurality of virtual objects to be arranged, each virtual object has appeared in the first embodiment. It can be handled in the same way as a virtual object.

  That is, if the second mode is set, each virtual object is placed at the placement position defined by the virtual space data for the virtual object, and if the first mode is set, The virtual object may be arranged at any position in the measurable area. In that case, it is necessary to arrange so that each virtual object may not overlap.

  For example, as shown in FIG. 6A, the virtual object 6000 is arranged in the same manner as the method described in the first embodiment with reference to FIGS. Next, as shown in FIG. 6B, the vertices constituting the virtual object 6000 are projected on the XZ plane, and a circle 6020 including the projected vertices is converted into, for example, the algorithm of the minimum inner circle described above. Find using. Then, the position of the point 6010 closest to the current position (point 5020) of the virtual object 6000 on the circle 6020 is obtained, and the current position (point 5020) of the virtual object 6000 is moved to the position indicated by the point 6010.

  Various methods are conceivable as a method for determining an arrangement position for arranging a plurality of virtual objects so as not to overlap each other in the measurable region, and is not limited to the above method. Further, when there are virtual objects that may overlap each other, the arrangement positions of other virtual objects excluding such virtual objects are determined so as not to overlap each other.

[Third Embodiment]
In both the first and second embodiments, it is assumed that the virtual object is observed at a closer position. However, there may be cases where it is required to view a virtual object from a distance.

  For example, when observing a virtual object of an elaborate machine or a virtual object of a minute size part, it is required to observe closer. However, a virtual object that imitates a relatively large object such as a building is required to be observed away from a certain distance. Therefore, in order to deal with such a case, for example, the mode setting unit 1080 makes it possible to set the third mode in addition to the first mode and the second mode.

  When the third mode is set, the virtual object moving unit 1030 obtains a sphere that includes the measurable region, for example, in the same manner as in the first mode, and sets a position separated from the center position by a specified distance, It is determined as the placement position of the virtual object. The “specified distance” may be determined in advance and stored in the data storage unit 1040, or may be appropriately changed by the user operating the input unit 1070.

  Further, such a virtual object may be arranged at a position that has a prescribed positional relationship with the center position of the sphere. Further, the specified positional relationship may be changed as appropriate by the user operating the input unit 1070. This applies not only to the positional relationship but also to the posture relationship.

[Fourth Embodiment]
The HMD 1200 used in the first to third embodiments is a video see-through HMD, but an optical see-through HMD may be used. In that case, steps S401 and S408 are removed from the flowchart of FIG. 3, and in step S409, the right-eye virtual space image and the left-eye virtual space image generated by the virtual space generation unit 1050 in step S407 are displayed on the display device 1210R, respectively. Processing to send to the display device 1210L is performed. Moreover, you may make it output the image of mixed reality space with respect to apparatuses, such as a tablet terminal device and a mobile telephone, instead of or in addition to HMD.

[Fifth Embodiment]
In the first to fourth embodiments, an optical three-dimensional position / orientation measurement apparatus is used for measuring the position / orientation of the marker 1230, but other three-dimensional position / orientation measurements such as a magnetic sensor and an ultrasonic sensor are used. The position and orientation of the marker 1230 may be measured using an apparatus. That is, as long as the “range in which the position and orientation of the marker 1230 can be measured” can be obtained, the position and orientation of the marker 1230 may be measured by any method.

  Of course, the position and orientation of the right-eye viewpoint and the position and orientation of the left-eye viewpoint may be obtained directly without using the marker 1230. In this case, the measurable area is a range in which the position and orientation of the viewpoint (imaging devices 1220R and 1220L) can be measured.

[Sixth Embodiment]
Each functional unit included in the image processing apparatus 1000 illustrated in FIG. 1 may be configured by hardware. However, the data storage unit 1040 is configured by a memory, the input unit 1070 is configured by a mouse or a keyboard, and the others. Each of the functional units may be configured by software (computer program). In this case, if the computer has a memory that functions as the data storage unit 1040, a user interface that functions as the input unit 1070, and a processor that executes a computer program corresponding to each of the other functional units, an image The present invention can be applied to the processing apparatus 1000.

  A hardware configuration example of a computer applicable to the image processing apparatus 1000 will be described with reference to the block diagram of FIG.

  As described above, the CPU 2001 performs operations using the computer programs and data stored in the RAM 2002 and the ROM 2003 to control the operation of the entire computer and the image processing apparatus 1000 to which the computer is applied. Execute each process.

  The RAM 2002 stores computer programs and data loaded from the external storage device 2007 and the storage medium drive 2008, and data input from the outside (the controller 1310 and the imaging devices 1220R and 1220L) via the I / F (interface) 2009. Have an area for. Further, the RAM 2002 has a work area used when the CPU 2001 executes various processes. As described above, the RAM 2002 can provide various areas as appropriate.

  The ROM 2003 stores setting data and a boot program for the computer.

  A keyboard 2004 and a mouse 2005 correspond to the input unit 1070 described above, and various instructions can be input to the CPU 2001 by a user of the computer.

  The display unit 2006 is configured by a CRT, a liquid crystal screen, or the like, and can display a processing result by the CPU 2001 using an image, text, or the like.

  The external storage device 2007 is a large-capacity information storage device represented by a hard disk drive device. The external storage device 2007 includes the OS (operating system) and each process described above as performed by each function unit other than the input unit 1070 and the data storage unit 1040 in each function unit of the image processing apparatus 1000 illustrated in FIG. The computer program and data for causing the CPU 2001 to execute are stored. The external storage device 2007 also stores what has been described as known information in the above description. The external storage device 2007 is also used as the data storage unit 1040. Computer programs and data stored in the external storage device 2007 are appropriately loaded into the RAM 2002 under the control of the CPU 2001 and are processed by the CPU 2001.

  The storage medium drive 2008 is a device that reads a computer program and data recorded on a recording medium such as a CD-ROM or a DVD-ROM, and sends them to the RAM 2002 or the external storage device 2007. A part or all of the computer program and data described as being stored in the external storage device 2007 may be recorded on this storage medium.

  The I / F 2009 functions as an interface for connecting the imaging devices 1220R and 1220L, the display devices 1210R and 1210L, and the controller 1310. For example, the I / F 2009 includes an analog video port for connecting the imaging devices 1220R and 1220L, a digital input / output port such as IEEE1394, an Ethernet (registered trademark) port for connecting the display devices 1210R and 1210L, and the like. .

  Each of the above parts is connected to the bus 2010.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

1030: Virtual object moving unit 1050: Virtual space generating unit

Claims (9)

A calculation means for obtaining a measurable range of the position and orientation of the viewpoint;
Arrangement means for arranging a virtual object within the range;
Generating means for generating an image of a virtual space including the virtual object arranged by the arranging means according to the position and orientation;
An image processing apparatus comprising: output means for outputting an image of the virtual space generated by the generating means. The calculating means includes
The image processing apparatus according to claim 1, wherein an area where the visual volumes of a plurality of optical sensors for measuring the position and orientation of the viewpoint overlap is obtained as the range. The arrangement means includes
The image processing apparatus according to claim 1, wherein the bottom surface of the bounding box that includes the virtual object is disposed at a position that is obtained by projecting the center position of the region that includes the range onto the floor surface.   The image processing apparatus according to claim 1, wherein the placement unit further changes the position of the placed virtual object so as not to overlap another virtual object. A calculation means for obtaining a measurable range of the position and orientation of the viewpoint;
Arranging means for arranging the virtual object at a position in a prescribed positional relationship with the range;
Generating means for generating an image of a virtual space including the virtual object arranged by the arranging means according to the position and orientation;
An image processing apparatus comprising: output means for outputting an image of the virtual space generated by the generating means. Furthermore,
Means for acquiring an image of a real space from the viewpoint;
The image processing apparatus according to claim 1, wherein the output unit outputs the virtual space image after combining the virtual space image with the real space image. An image processing method performed by an image processing apparatus,
A calculating step for calculating a range in which the position and orientation of the viewpoint can be measured by the calculation means of the image processing apparatus;
An arrangement step in which an arrangement unit of the image processing apparatus arranges a virtual object in the range;
A generation step in which the generation unit of the image processing apparatus generates an image of a virtual space including the virtual object arranged in the arrangement step according to the position and orientation;
An image processing method comprising: an output step of outputting an image of the virtual space generated in the generation step. An image processing method performed by an image processing apparatus,
A calculating step for calculating a range in which the position and orientation of the viewpoint can be measured by the calculation means of the image processing apparatus;
An arrangement step in which the arrangement means of the image processing apparatus arranges the virtual object at a position in a prescribed positional relationship with the range;
A generation step in which the generation unit of the image processing apparatus generates an image of a virtual space including the virtual object arranged in the arrangement step according to the position and orientation;
An image processing method comprising: an output step of outputting an image of the virtual space generated in the generation step.   A computer program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 6.

Images