JP2016139416A - Information processing system, processing method therefor, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mechanism for enabling a tester to take quick avoidance behavior by visually recognizing a real image in the case that the tester staggers, or the like.SOLUTION: An information processing system for generating a composite image obtained by compositing a real image of a real space imaged by an imaging device with a virtual image of a virtual object includes: speed acquisition means for acquiring speed made when the imaging device is moved; specification means for specifying a virtual object to be non-display within a prescribed range according to a position and a posture of the imaging device when the speed acquired by the speed acquisition means exceeds a prescribed value; and drawing control means for making the virtual object specified by the specification means non-display so as to make a tester visually recognize an image of the real space, and performing drawing control by using a virtual object corresponding to the position and posture of the imaging device and the real image of the real space.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing system that combines and displays a real image and a virtual image in a virtual space, and a processing method and program thereof.

  Conventionally, an image in which a three-dimensional modeled CG (Computer Graphics) is superimposed on an image in real space (actual image) is generated and presented to the observer, so that the viewer can see the CG in real space as CG. There is a technology called mixed reality or augmented reality that can make an object (virtual object) appear as if it exists.

  In a system that presents mixed reality and augmented reality, a stationary display, a mobile device such as a smartphone or a tablet, an optical see-through HMD, a video see-through HMD, or the like is used as a video display device.

  In recent years, a lot of systems and applications that present mixed reality and augmented reality have been announced, and there are many opportunities to see them in general. However, safety measures against falling, sickness, etc. in the general user experience have become issues. Yes.

Especially in a system that uses a video see-through HMD, the HMD body covers the entire field of view, and unlike the optical see-through HMD, it is possible to draw CG in a completely opaque form, so it hits a wall or a desk. When staggered, the user tries to put his hands and feet in a safe place, but during the mixed reality experience, a part of the real world is concealed by CG, and if the user does not remove the HMD, the real world Obstacles can not be confirmed, and the time lag generated there affects safety.
On the other hand, if the CG is hidden or made translucent so that a real object can be seen, there is a problem that the immersive feeling is impaired.

  As a countermeasure against the problem of not knowing what kind of obstacle is in the area hidden by the CG, Patent Document 1 discloses that when a hand approaches a place where there is a real object that can be put on, it is concealed. A technique is described in which the display of a virtual object is changed and presented to the user.

JP 2009-169622 A

  However, in the system described in Patent Document 1, it is possible to identify a place where a hand is placed and change one virtual object where the hand is placed to show some real objects, but emergency evacuation when staggered In operation, since the whole real object cannot be grasped, there is a difficult aspect to apply at the time of emergency evacuation.

  In addition, when showing a real object when placing a hand, if there was no real object in the place where the hand was placed, the judgment at the time of emergency evacuation was delayed, and it was difficult to cope with sudden stagger .

  Furthermore, it is difficult to judge whether it is an urgent hand to touch or a hand to touch, so if you touch it, show the entire real object (real image) If this happens, the sense of immersion will be lost and the benefits of the mixed reality system will be lost.

  Therefore, an object of the present invention is to provide a mechanism that allows an experienced person to take a quick avoidance action by visually recognizing a live-action image when the experienced person wobbles.

  In order to achieve the object of the present invention, an information processing system that generates a composite image obtained by synthesizing a real image captured in a real space and a virtual image of a virtual object, and a speed resulting from the movement of the imaging device And a specifying unit that specifies a virtual object that is hidden in a predetermined range according to the position and orientation of the imaging device when the speed acquired by the speed acquiring unit exceeds a predetermined value. And the virtual object specified by the specifying means is hidden so that the experience person can visually recognize the image of the real space, the virtual object according to the position and orientation of the imaging device, and the real image of the real space, And a drawing control means for performing drawing control using the.

  According to the present invention, when an experienced person staggers, the experienced person can take a quick avoidance action by visually recognizing a live-action image.

It is a figure which shows an example of a mixed reality system configuration and the hardware configuration of each hardware It is a figure which shows an example of a functional block diagram It is a figure which shows an example of an application block diagram It is a flowchart of the detailed process of the mixed reality system in 1st Embodiment. It is an image figure which shows an example of the display of the image which the virtual image which is not transmitted is synthesize | combined It is an image figure which shows an example of the display of the image which the transparent virtual image was synthesize | combined. It is a graph which shows transition of the transmittance | permeability It is a figure which shows an example of the formula which calculates the transmittance | permeability. It is a figure which shows an example of the data which manage the information of a model. It is a flowchart of the detailed process of the mixed reality system in 2nd Embodiment. The figure which shows an example by which the model within predetermined distance is displayed transparently. It is a figure which shows an example of the data which manage the information of each model (3D model as a virtual object) in 2nd Embodiment. It is the figure which showed the concept of the clipping plane. It is a figure which shows an example of the data which manage the information of a model. It is a calculation formula for obtaining the parameters of the view frustum when the front clipping plane located at the distance near is moved by dist. It is a flowchart of the display control at the time of employ | adopting the several non-display control system which shows the detail of 3rd Embodiment. It is a figure which shows an example of the image | video displayed by HMD120 controlled by the mixed reality application in 3rd Embodiment. It is an example of the image | video displayed by HMD120 when an experience person staggers. It is a figure which shows the functional block diagram in this invention.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a system configuration and a hardware configuration of each hardware in the present invention. That is, it is a diagram illustrating an example of an information processing system that generates a composite image obtained by combining a real image captured in a real space captured by an HMD 120 serving as an image capturing apparatus and a virtual image of a virtual object.

  In the present embodiment, the PC 100 (information processing apparatus) and the HMD 120 can communicate with each other. The HMD is a head-mounted display that displays an image obtained by synthesizing an actual video and a three-dimensional (3D) CG model on the PC 100. The HMD 120 is a device that captures real images.

  In FIG. 1, reference numeral 101 denotes a CPU that comprehensively controls each device and controller connected to the system bus 104. Further, the ROM 102 or the auxiliary storage device 113 has a BIOS (Basic Input / Output System) or an operating system program (hereinafter referred to as OS) that is a control program of the CPU 101, a program necessary for realizing a function executed by the PC, and the like. Is remembered.

  Reference numeral 103 denotes a RAM which functions as a main memory, work area, and the like for the CPU 101. The CPU 101 implements various operations by loading a program or the like necessary for execution of processing from the ROM 102 or the auxiliary storage device 113 into the RAM 103 and executing the loaded program.

An input controller 105 controls input from a pointing device such as a keyboard (KB) 110 and a mouse 111. A video controller 106 controls display on a display device such as the display 121 and the display 121 built in the HMD 120. In addition, it outputs to the display 121 using an external output terminal (for example, Digital Visual Interface). The display 121 includes a right eye display and a left eye display.
The display 121 is used by an experienced person, and the display 121 is a liquid crystal display mainly used by an operator.

Reference numeral 107 denotes a general-purpose bus, which is used to capture video from the camera 122 built in the HMD 120. From the camera 122, it inputs using an external input terminal (for example, IEEE1394 terminal). The camera 122 includes a right eye video camera and a left eye video camera.
In the present embodiment. Although the HMD 120 and the PC 100 are connected by wire, the image may be transmitted and received by wireless communication.

  Reference numeral 108 denotes a memory controller, which is installed in an external storage device (hard disk (HD)), flexible disk (FD), or PCMCIA card slot for storing a boot program, various applications, font data, user files, editing files, various data, and the like. Controls access to the auxiliary storage device 113 such as a compact flash (registered trademark) memory connected via the adapter.

  Reference numeral 109 denotes a network controller, which is connected to and communicates with an external device via a network (for example, the network 114 shown in FIG. 1), and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  Various programs to be described later for realizing the present invention are recorded in the auxiliary storage device 113 and are executed by the CPU 101 by being loaded into the RAM 103 as necessary. Further, various model data used at the time of execution of the program are also stored in the auxiliary storage device 113, and detailed description thereof will be described later.

  Next, a functional configuration diagram according to the present invention will be described with reference to FIG. FIG. 2 is a diagram showing an example of a functional configuration diagram according to the present invention.

  The position / orientation measurement unit 201 measures the position and orientation of the HMD 120 in the real world by the experienced person. As a method for measuring the position and orientation, a two-dimensional marker is arranged in the real space, information on the position and orientation and the size of the marker is given to the position and orientation measurement unit 201 in advance, and the two-dimensional marker reflected in the camera 122 is used. The position and orientation of the HMD 120 are determined from the size and distortion on the image. In addition to the method for obtaining the position and orientation described above, there are a method for attaching a position and orientation sensor using magnetism or an infrared marker to the HMD 120, a method for using them in combination, etc., all of which can be realized by known techniques. . The position and orientation measurement is disclosed in Japanese Patent Application Laid-Open No. 2005-83826. That is, the position and orientation can be specified without using the marker of the photographed image.

The acceleration measurement unit 202 has a function of measuring the acceleration during movement of the HMD 120 in the real space and notifying the screen display control unit 203 when the acceleration exceeds a certain threshold. As a method for measuring the acceleration, the acceleration may be obtained by second-order differentiation of the time series data of the three-dimensional position and orientation of the HMD 120 obtained from the position and orientation measurement unit, or an acceleration sensor is directly attached to the HMD 120, A value may be acquired.
The screen display control unit 203 has a role of generating and controlling a video (composite image) to be displayed on the display 121.

  Specifically, the real-image drawing unit 204 is requested to draw a real space image taken by the camera 122. Next, the position / orientation of the HMD 120 at that time is received from the position / orientation measurement unit 201, and the CG image drawing unit 205 is requested to draw a CG (model image) viewed from the position / orientation. Next, the images drawn by the photographed image drawing unit 204 and the CG image drawing unit 205 are superimposed and combined. At this time, it is checked whether there is a notification that the acceleration exceeds the threshold value from the acceleration measuring unit 202. If the threshold value is exceeded, the CG (model image) drawn by the CG image drawing unit 205 is made semi-transparent and superimposed. I do. The transparency at this time is controlled so as to change depending on the time after exceeding the threshold value, and the latter half transparent state in which the CG (model image) fades out lasts for several seconds, and then fades in to the opaque state. This reduces the surprise that the CG suddenly disappeared for the experienced person. Details of this control will be described later.

  A calibration unit 206 is also provided. The calibration unit 206 determines the origin for the virtual space in the real space using the reference marker, and associates the position of the real space with the virtual space. Based on this origin, the position and orientation of the three-dimensional model are stored in advance. As a result, the virtual image of the virtual object in the direction viewed from the HMD 120 is synthesized with the real image in the real space so that the virtual object exists in the real space. By storing the origin and the position and orientation of the three-dimensional model in the graphic engine 302, it is possible to synthesize a virtual image of a virtual object viewed from the HMD 120 with a real image. It is assumed that the calibration by the calibration unit 206 is performed at the beginning of starting the system.

  Next, an application configuration diagram according to the present invention will be described with reference to FIG. FIG. 3 is a diagram showing an example of an application configuration diagram in the present invention.

The PC 100 includes an operating system 301, a graphic engine 302, a mixed reality platform 303 (also referred to as an MR platform), and a mixed reality application 304 (also referred to as an MR application or a viewer application), and is controlled by the CPU 101.
The operating system 301 controls input / output of the HMD 120.

The real image obtained from the camera 122 via the input interface is delivered to the mixed reality platform 303. The composite image drawn by the graphic engine 302 is output to the display 121 via the output interface.

  The graphic engine 302 generates an image to be drawn from the three-dimensional model and synthesizes it with the photographed image. The engine used for drawing may be, for example, a widely used graphic engine such as OpenGL or DirectX, or an independently developed graphic engine. In the present embodiment, OpenGL is used as the graphic library.

  The mixed reality platform 303 detects a marker of a live-action image, specifies the position / posture of the HMD 120 based on the marker, and aligns the real space and the virtual space. Note that the position and orientation and the alignment technique are realized by using JP-A-2002-32784, JP-A-2006-072903, and JP-A-2007-166427, which are disclosed as known techniques. Instead of using the HMD 120, the HMD 120 may be provided with a position sensor, and the position and orientation of the HMD 120 may be specified based on the position measured by triangulation using the position sensor.

  Further, the mixed reality platform 303 calculates the acceleration of the HMD 120 from the marker and determines the transmittance of the three-dimensional model. For calculating the acceleration, the HMD 120 may be provided with an acceleration sensor, and the acceleration may be calculated without using a marker.

  The mixed reality application 304 receives the position, orientation, and transmittance from the mixed reality platform 303 and issues a three-dimensional model drawing command to the graphic engine 302. At this time, a command in which model information, position information, posture information, and transmittance are set is issued using the OpenGL API.

Next, detailed processing of the mixed reality system in the present embodiment will be described with reference to FIG. FIG. 4 is a flowchart of detailed processing of the mixed reality system.
In step S <b> 401, a real image (video) is captured by the camera 122 of the HMD 120.
In step S402, an image captured by the camera 122 of the HMD 120 is transmitted to the PC 100.

In step S403, it is determined whether or not shooting of the HMD 120 has been completed, and imaging and image transmission are repeated until the shooting is completed. That is, when the HMD 120 captures an image, the HMD 120 repeatedly transmits an image to the PC 100.
In step S <b> 404, the mixed reality platform 303 receives an image via the operating system 301.

In step S405, the mixed reality platform 303 substitutes the system time of the PC 100 for the variable “t”. This variable “t” is used in transparency control described later.
In step S406, the position / orientation measurement unit 201 of the mixed reality platform 303 measures the position / orientation of the HMD 120.

  In step S407, acceleration is calculated by the acceleration measuring unit 202 of the mixed reality platform 303. As described above, the acceleration may be obtained by an acceleration sensor.

  In step S408, the mixed reality platform 303 determines that the transmittance control flag managed by the mixed reality platform 303 is True, that is, the state in which the transmittance is controlled. If the transmittance is controlled (True), the process proceeds to step S413. If the transmittance is not controlled (False), the process proceeds to step S409.

  In step S409, the mixed reality platform 303 acquires the acceleration measured in step S407. That is, this is a step showing an example of an acceleration acquisition process for acquiring the acceleration of the imaging apparatus.

  In step S410, a comparison is made with an acceleration threshold value managed by the mixed reality platform 303. If the acceleration is equal to or greater than the threshold value, the process proceeds to step S412. If the acceleration is less than the threshold value, the process proceeds to step S411. The case where the acceleration is greater than or equal to the threshold value is a case where the person wearing the HMD 120 may trip and the person may fall, and the case where the acceleration is less than the threshold value is a normal person This is the case.

  In the present embodiment, the determination is made using “60 m / s ^ 2” as the acceleration threshold value, but this is only an example and the present invention is not limited to this. The graph of FIG. 7 is a graph that represents the acceleration threshold value as “60 m / s ^ 2.”

  In step S411, the mixed reality platform 303 substitutes “0” for the transmittance. Setting the transmittance to “0” means that the CG image is synthesized (superimposed) on the actual image without transmitting it. FIG. 5 shows an example in which the real image and the virtual image are superimposed on each other with the transmittance “0” and displayed on the display 121. The description of FIG. 5 will be described later.

In step S412, since the acceleration exceeds the threshold value, the mixed reality platform 303 sets “True” for the transmittance control flag, and sets the current time of the system of the PC 100 for the variable “t ₀ ”. Thereby, the transmission control is started. That is, it is a step showing an example of a process for setting transparency when the acceleration exceeds a predetermined value.

  In step S413, the mixed reality platform 303 calculates the transmittance at the current time “t”. The transmittance is calculated using the calculation formula of FIG. The calculation formula of FIG. 8 shows an example and is not limited to this. If the transmittance is “0” (immediately after the acceleration exceeds the threshold, that is, in the case of calculation in step S413 after the transmittance control flag is set to “True”), the transmittance is set to a predetermined value. Is controlled to start transmission. Step S413 is a step showing an example of transmission value acquisition for acquiring the transmission value of the virtual image so that the user can visually recognize the image in the real space when the acceleration acquired in the acceleration acquisition process exceeds a predetermined value. .

Further, step S413 is a step showing an example of processing for determining a transmission value higher than the previous transmission value by the calculation formula of FIG. 8 when transmission setting is being performed (transmission control is “True”). In step S418, which will be described later, the virtual image is controlled to be transmitted using a transmission value higher than the previous transmission value.
FIG. 7 is a graph showing the transmittance when the calculation formula of FIG. 8 is used (graph showing the transition of the transmittance).

  Here, FIG. 7 will be described. As described above, “60 m / s ^ 2” is used as the acceleration threshold.

The variable “p” represents the transmittance at time “t”. The variable “t _0” represents the time when the acceleration of the HMD 120 exceeds the threshold value. The constant “a” represents the rising and falling slopes of the transmittance. By changing the transmittance according to the inclination of “a”, it becomes possible to prevent the CG from disappearing suddenly and leading to confusion when the experience person falls.

The constant “d” represents the time for maintaining the maximum transmittance. The constant “pMax” represents the maximum transmittance. Note that if the CG is completely erased, it will cause confusion for the experienced person. Therefore, it is desirable to set the value of “pMax” to less than 100% so that the CG does not completely disappear even during the transmittance control.
If the acceleration of the HMD 120 exceeds the threshold again during “d”, the transmission control is performed with the maximum transmittance of “pMax” for the time “d” again from the time when the acceleration exceeds the threshold. It may be. Further, when the acceleration of the HMD 120 exceeds the threshold value at the time of falling of “a”, the transmittance may be controlled to rise to “pMax” with reference to the current transmittance.

  In step S414, the mixed reality platform 303 determines whether or not the transmittance is “0”. If the transmittance is “0”, the process proceeds to step S415. That is, the case where the transmittance becomes “0” is a case where the transmission is finished. If the transmittance is not “0”, the process proceeds to step S416. That is, the case where the transmittance is not “0” is a case where transmission is in progress.

  In step S415, the mixed reality platform 303 sets the transmittance control flag to “False” and ends the transmission control. That is, when the transmission setting is being performed (transmission control flag “True”) and the transmission (value) returns to a reference value (for example, “0”), the transmission setting is canceled so that the virtual image is not transmitted. It is a step which shows an example.

  In step S <b> 416, the mixed reality platform 303 passes the position information, posture information, and transmittance to the mixed reality application 304. The mixed reality application 304 specifies a model to be combined with a live-action image based on position information and posture information. In the present embodiment, the identified models are 901 and 902. Since a technique for specifying a model to be drawn according to position information and posture information is a known technique, the description thereof is omitted.

FIG. 9 is a diagram illustrating an example of data for managing information of each model (a three-dimensional model as a virtual object). Each model is associated with a three-dimensional model name, position information, and a transparency flag, and is used to specify a model to be drawn or to determine whether or not it is a transparent model. The data shown in FIG. 9 is stored in advance by a user who operates the decrypted reality system.
The data in FIG. 9 is data referred to by the mixed reality application 304. However, the mixed reality platform 303 may refer to the data and the model to be rendered on the mixed reality platform 303 may be determined. It is. Whether it is realized by the mixed reality platform 303 or the mixed reality application 304 may be appropriately changed by design.

In step S417, the mixed reality application 304 determines whether the model to be drawn is a transparent model according to the transparent flag 903. That is, this is a step showing an example of a process of specifying a virtual object that is not displayed in a predetermined range according to the position and orientation of the HMD 120 (imaging device) when the speed exceeds a predetermined value.
The model with the transparency flag “ON” is a model that affects when a person falls and touches the hand, and the model with a transparency flag “OFF” is a model that does not affect when a person falls. By setting the transparent model and the non-transparent model, it is possible to avoid a fall on the mixed reality system without impairing the immersive feeling. Needless to say, it is possible to adopt a configuration in which transmission control is performed for all models.
If the model has the transparency flag “ON”, the process proceeds to step S418. If the model has the transparency flag “OFF”, the process proceeds to step S419.
That is, step S417 is a step showing an example of a transparency determination process for determining whether or not the virtual image is transparent.
Note that a flag is determined for each model to be combined, and the process of step S419 or step S420 is executed for each model.

  In step S418, the mixed reality application 304 uses the API of the graphic engine 302 to send a rendering command including model information, position information, posture information, and transmittance (the transmittance calculated and acquired in step S413) to the graphic engine 302. The graphic engine 302 generates a virtual image to be synthesized from the model and the transmittance. More specifically, the graphic engine 302 determines the position and orientation in the virtual space of the model for which the rendering command has been issued in accordance with the position information and orientation information, and controls the model so as to obtain this position and orientation. Generate an image. A transmittance is set for the generated virtual image.

  The graphic engine 302 manages the 3D model generated by the CAD application so that it can be referred to. Although the three-dimensional model is stored in the auxiliary storage device 113, the three-dimensional model may be managed by a server in a separate housing.

In step S419, since it is not a transparent model, the mixed reality application 304 uses the API of the graphic engine 302 to issue a drawing command including model information, position information, posture information, and transparency (0) to the graphic engine 302. Thus, the graphic engine 302 generates a virtual image to be synthesized from the model and the transmittance.
In other words, when the virtual image that is not transmitted in the transmission determination is not transmitted through step S419, the virtual image is not transmitted. In step S418, the determined transmittance is determined as the virtual image that is transmitted through the transmission determination. 5 is a step showing an example of processing for controlling the transmission of a virtual image according to the steps.

In step S420, it is determined whether the mixed reality application 304 has generated virtual images of all models to be synthesized. Specifically, it is determined whether the mixed reality application 304 has issued a drawing command to the graphic engine 302 for a model for generating a virtual image. Note that. The graphic engine 302 can specify a model to be synthesized based on position information and posture information, generate a virtual image, and draw a virtual image on a real image. A model drawing command may be issued to determine whether the graphic engine 302 has generated a virtual image.
FIG. 6 shows an example in which the virtual image generated in step S418 and step S419 and the photographed image are superimposed and displayed on the display 121. 56 will be described later.

  In step S <b> 421, the mixed reality platform 303 passes position information, posture information, and transmittance to the mixed reality application 304. The mixed reality application 304 specifies a model (CG) to be combined with a live-action image based on position information and posture information. In the present embodiment, the identified models (CG) are 901 and 902. Since a technique for specifying a model to be drawn according to position information and posture information is a known technique, the description thereof is omitted.

  In step S422, since it is not a transparent model, the mixed reality application 304 uses the API of the graphic engine 302 to issue a rendering command including model information, position information, posture information, and transparency (0) to the graphic engine 302. Thus, the graphic engine 302 generates a virtual image to be synthesized from the model and the transmittance. More specifically, the position and orientation of the model in the virtual space are controlled from the position information and orientation information, the transmittance is set for the model whose position and orientation are controlled, and a virtual image in the virtual space is generated. . Note that the position and orientation in the virtual space can be specified from the position and orientation of the HMD 120, and a virtual image of the model that can be seen from this position and orientation is generated.

  In step S423, the graphic engine 302 synthesizes (superimposes) the real image received in step S404 and the virtual image according to the position information and orientation information, and generates a composite image to be displayed on the camera 122 of the HMD 120. That is, step S423 is a step showing an example of a drawing control process for combining the transmitted virtual image, the non-transmitted virtual image, and the real image.

  Steps S418 and S423 are an example of a drawing control process for controlling drawing of a virtual image transmitted by transmittance (value) and a real image in real space with respect to a virtual object determined according to the position and orientation of the imaging apparatus. It is the step shown.

In step S424, the graphic engine 302 transmits the generated composite image to the HMD 120 via the operating system 301. That is, it is a step showing an example of output processing for outputting the synthesized image to be displayed on the HMD 120 as the imaging device.
In step S425, the HMD 120 receives the composite image from the PC 100.
In step S426, the received composite image is displayed on the display 121 of the HMD 120.

FIG. 5 shows an example in which a virtual image is generated with the transmittance “0” and displayed on the display 121. FIG. 6 shows an example in which the acceleration exceeds the threshold value and the virtual image is controlled to be transmitted and displayed on the display 121.
The model A of 501 in FIG. 5 synthesizes and displays a virtual image on the live-action image with the transmittance “0”, and the model B of 502 synthesizes the virtual image with the real-image image with the transmittance of “0”. it's shown.
A model A of 601 in FIG. 6 is a model to be transmitted (transmission flag ON), and the virtual image generated in accordance with the transmittance calculated in step S413 and the photographed image are combined and displayed, and a model B of 602 is displayed. FIG. 5 shows an example of a model that does not allow transmission (transmission flag OFF) and is displayed on the transmittance “0” display 121. The description of FIG. 5 will be described later.

[Second Embodiment]
Next, a second embodiment will be described. In the second embodiment, the close model is hidden in accordance with the acceleration.

First, a case assumed in the present embodiment will be described.
In the mixed reality application 304, when a method of visualizing a real object by transmitting CG as in the first embodiment is used, the CG transmission processing may affect the entire field of view. This is because, in mixed reality, the real image and many virtual images are superimposed and displayed, so images of various virtual objects far from the experience are transparently displayed, and the image being experienced changes greatly. This is because it leads to confusion of the experience.
Thus, in addition to the transmission processing of the first embodiment, in order to avoid danger more, it is desirable to minimize the influence on the field of view.
Specifically, it is sufficient that a real space of several meters around the experience person can be visually recognized, and it is conceivable to hide the CG model existing within the range.

In the present embodiment, a CG model to be hidden or controlled for transmission is determined based on the position of the experience person's HMD 120, the position of the CG model, and acceleration, and a necessary object is determined.
Details of the second embodiment will be described below. Note that description of the same parts as in the first embodiment is omitted.

  Detailed processing of the mixed reality system in the second embodiment will be described with reference to FIG. FIG. 10 is a flowchart of detailed processing of the mixed reality system in the second embodiment.

  In step S1001, it is determined whether the position of the model specified in step S416 is within a predetermined distance from the HMD 120. Specifically, a straight line distance is calculated from the position information (spatial coordinates) of the specified model and the position of the HMD 120, and it is determined whether the calculated distance is within the distance of the transmission range 1204 in FIG. Assume that 1201 is described in a setting file or the like held in the PC 100. Note that the calculation method for calculating the distance between the HMD 120 and the model is a known calculation method, and thus the description thereof is omitted.

  If it is determined in step S1001 that the model is within the predetermined distance, the process proceeds to step S418. If it is determined that the model is outside the predetermined distance, the process proceeds to step S419. As a result, models within a predetermined distance are displayed transparently, and models outside the predetermined distance are normally displayed without being transparently displayed.

FIG. 11 shows an example in which a model within a predetermined distance is transparently displayed. Referring to FIG. 11, before the acceleration exceeds the threshold, model A1001, model B1002, and model C1003 are superimposed and displayed on the real image. When the acceleration exceeds the threshold value, they are superimposed and displayed as model A1004, model B1005, and model C1006.
The model C1006 is displayed without transmission control because it is outside the predetermined distance, and the model A is displayed with transmission control being performed within the predetermined distance. The model B 1005 is displayed without being controlled for transparency because the transparency flag 1200 is OFF.
Needless to say, the distance within the predetermined distance may be within or less than the distance.

  FIG. 12 is a diagram illustrating an example of data for managing information of each model (a three-dimensional model as a virtual object) in the second embodiment.

  According to the second embodiment, a near CG model is transmitted, and a distant CG model (a CG model that does not affect the user when staggered) is normally displayed, thereby greatly changing the image being experienced. It is possible to avoid danger by the experience person without any trouble.

[Third Embodiment]
Next, a third embodiment will be described. In the third embodiment, a close model is hidden by clipping according to acceleration.
The third embodiment is a mechanism that is more effective when the virtual object is large in addition to the case where the virtual object is small as in the second embodiment.

In the mixed reality application 304, there may be a case where a huge object such as a building, large furniture, or plant piping is arranged as a virtual object and covers the periphery of the experience person. When these objects are single CG data, the entire virtual object becomes transparent or non-displayed, which affects the entire field of view.
For example, in the case of a model such as a building, member information is lost in the process of converting from original CAD data to a model format that can be displayed in a mixed reality application, and is often handled as a single huge object. . Therefore, there is a case where it is difficult to control only the members near the experience person to hide.

  Therefore, in the third embodiment, a mechanism for avoiding the danger by performing non-display control using control based on the operation of the clipping plane as a method for controlling the display state of the model will be described.

First, the clipping plane will be described with reference to FIG.
FIG. 13 is a diagram showing the concept of the clipping plane. Specifically, two planes that are perpendicular to the viewing direction of the camera 122 are defined, and a plane that is closer to the camera 122 (near) is a front clipping plane and a plane that is farther (far) is rearwardly clipped. Called a plane. Only the object located in the space surrounded by the two planes and the plane cut by the four sides (top, bottom, left, right) of the screen is drawn, and the object located outside is cut out and excluded from the drawing target. To display. The space surrounded by the six planes described above is called a viewing frustum.
In the drawing, when a graphic engine such as OpenGL or DirectX is used, the virtual space to be drawn is limited to a finite space due to restrictions on the calculation algorithm.
In OpenGL, the shape of the visual cone is determined by six parameters, near, far, top, bottom, left, and right.

The clipping plane technology can be realized, for example, by using the technology described in “Principle of 3D Computer Animation 2nd Edition P77-P79 (Toppan Co., Ltd./M. Oroke / Copyright)”. is there.
In this embodiment, using the clipping plane technology, while the staggered experiencer takes a risk avoidance action, the distance of the front clipping plane is kept away from the camera to hide the CG near the experiencer.

  The calculation formula of FIG. 14 is an example of a calculation formula for obtaining the parameters of the view frustum when the front clipping plane located at the distance near in FIG. 13 is moved by dist.

Here, in order to alleviate a sudden change in the scenery in the situation where the experience person is staggering, the near clip surface is stepped back in stages by animation, and after a certain time has passed, it is made to return to the original clip surface by animation. Here, if the retreat rate of the near clip plane with time is “p”, the parameters defining the viewing frustum are calculated by the calculation formula of FIG.
For variables such as “p”, “a”, “t ₀ ”, and “d”, the same value is used because non-display control is performed by clipping at the transmission timing of the CG model to be transmitted. Arbitrary values can be used as values of variables such as “a” and “d”.
Further, “dMax” is a variable for representing the maximum distance for moving the near clipping plane, and it is possible to hide the range of the maximum distance “near + dMax” as viewed from the experience person. Since the near clipping plane cannot be set behind the far clipping plane, a positive number satisfying dMax <far-near is arbitrarily set as dMax.
In the present embodiment, near is normally set to 10 cm and is a position where drawing of the CG model is stopped.
Further, when near is determined by the calculated near ′, what has been displayed until then is not displayed (the range of near ′ is not displayed), and a real image can be seen.
The calculation formula of FIG. 14 is a formula for shifting the clipping plane backward while maintaining the angle of view.

  FIG. 15 is a diagram illustrating an example of data for managing model information in the present embodiment. Instead of the transparency flag of FIG. 9 of the first embodiment and FIG. 12 of the second embodiment, a non-display control method is defined. As attributes of the non-display control method 1504, transparency, none, and clipping can be defined.

  In the first embodiment or the second embodiment, the transparency flag 903 has a binary value (ON) / non-off (OFF), which controls to hide, but in this embodiment, the non-display control method. In addition to the transmission (first non-display) described in the first embodiment, three values of clipping (second non-display) and no transmission (normal display) are taken. In addition to these three values, a new display control method may be defined.

  FIG. 16 is a flowchart of display control when a plurality of non-display control methods are employed, showing details of the third embodiment.

In step S1601, the mixed reality platform 303 substitutes the system time of the PC 100 for the variable “t”. This variable “t” is used in the non-display control described later. Note that step S1601 is a process corresponding to step S405 in FIG.
In step S1602, the mixed reality platform 303 receives an image via the operating system 301. This process corresponds to step S404 in FIG.

  In step S1603, the position / orientation measurement unit 201 of the mixed reality platform 303 measures the position / orientation of the HMD 120. As a result, the position / posture of the HMD 120 is determined, and the CG model to be drawn is determined according to the position / posture of the HMD 120. That is, this corresponds to a process of specifying a virtual object that is hidden (transmitted or clipped) in a predetermined range according to the position and orientation of the HMD 120 (imaging device) when the speed exceeds a predetermined value. This process corresponds to step S406 and step S416 in FIG.

  In step S1604, acceleration is calculated by the acceleration measuring unit 202 of the mixed reality platform 303. The acceleration may be obtained by an acceleration sensor. This process corresponds to step S407 in FIG.

  Thereafter, a loop is performed from step S1605 to S1622 by the number of registered virtual objects (for the three-dimensional model in FIG. 15). Note that the virtual object looping here is performed on the virtual object drawn from the position and orientation of the HMD 120.

  In step S1606, the mixed reality platform 303 determines whether or not the non-display control flag managed by the mixed reality platform 303 is True, that is, the non-display control state. If non-display is controlled (True), the process proceeds to step S1610. If non-display is not controlled (False), the process proceeds to step S1607.

  In step S1607, a comparison is made with an acceleration threshold value managed by the mixed reality platform 303. If the acceleration is equal to or greater than the threshold value, the process proceeds to step S1609. If the acceleration is less than the threshold value, the process proceeds to step S1608. The case where the acceleration is greater than or equal to the threshold value is a case where the person wearing the HMD 120 may trip and the person may fall, and the case where the acceleration is less than the threshold value is a normal person This is the case.

  In step S1608, the non-display control is not performed, and the CG model of the virtual object is drawn in a normal state. The acceleration threshold is configured to use “60 m / s ^ 2” as in the first embodiment and the second embodiment, but is merely an example and is not limited thereto.

  In step S1609, “True” is set as the non-display control flag, and the current time of the system of the PC 100 is set as the variable “t0”. Thereby, the transmission control is started. That is, it is a step showing an example of a process for setting transparency when the acceleration exceeds a predetermined value.

  Steps S1610 to S1614 are processes for performing non-display by transmission. Hereinafter, a process of performing non-display by transmission will be described, but the transmission control of the first embodiment and the second embodiment is the same.

  In step S1610, referring to FIG. 15, the process branches according to the non-display control method 1504 corresponding to the model currently being processed. In the present embodiment, the process moves to step S1611 in the case of the transmission method, step S1615 in the case of the clipping method, and step S1621 in the case of the normal display without non-display control.

  In step S <b> 1611, the mixed reality platform 303 calculates the transmittance at the current time “t”. The transmittance is calculated using the calculation formula of FIG. The calculation formula of FIG. 8 shows an example and is not limited to this. Further, when the transmittance is “0” (immediately after the acceleration exceeds the threshold, that is, in the case of calculation in step S1611 after the transmittance control flag is set to “True”), the transmittance is set to a predetermined value. Is controlled to start transmission.

  In step S1612, the mixed reality platform 303 determines whether or not the transmittance is “0”. If the transmittance is “0”, the process proceeds to step S1613. That is, the case where the transmittance becomes “0” is a case where the transmission is finished. If the transmittance is not “0”, the process proceeds to step S1614. That is, the case where the transmittance is not “0” is a case where transmission is in progress.

  In step S1613, the mixed reality platform 303 sets the transmittance control flag to “False” and ends the transmission control. That is, when the transmission setting is being performed (transmission control flag “True”) and the transmission (value) returns to a reference value (for example, “0”), the transmission setting is canceled so that the virtual image is not transmitted. It is a step which shows an example.

  In step S1614, the CG model of the virtual object is drawn by reflecting the transmittance calculated in step S1611. The process of drawing through the CG model is controlled by the mixed reality application 304 issuing a drawing command to the graphic engine.

Steps S1615 to S1620 are processes for performing non-display by clipping.
In step S1615, information (near, far, top, bottom, left, right) shown in FIG. 13 in the mixed reality platform 303 is saved in a temporary area (for example, RAM). This is to limit the effect of changing the clipping plane to the model currently being processed.

In step S1616, a distance (near ') to the front clipping plane is obtained based on the calculation formula of FIG.
In step S1617, the mixed reality platform 303 determines whether “p” in the calculation method of FIG. 14 is “0”. If p is "0", the process proceeds to step S1618. That is, the case where p becomes “0” is the case where the non-display control by clipping is finished. If p is not “0”, the process proceeds to step S1614. That is, the case where p is not “0” is a case where non-display control by clipping is being performed.

  In step S1618, the mixed reality platform 303 sets the non-display control flag to “False” and ends the non-display control. That is, when clipping is being set (non-display control flag “True”) and p returns to a reference value (eg, “0”), the clipping plane is returned to the normal state so that the virtual image is not clipped. It is a step showing an example.

  In step S1619, the CG model of the virtual object is drawn by reflecting the information (near ', far', top ', bottom', left ', right') obtained in step S1616. This drawing is realized by outputting a drawing command to the graphic engine 302.

  In step S1620, the information on the visual cone retreated in step S1615 is reflected on the mixed reality platform 303 to restore the original visual cone. As a result, the next model is displayed in the normal clipping plane.

  In step S1621, non-display processing such as transparency and clipping is not performed, and CG is drawn as usual. As a result, a model that is a virtual object located far away or at a high place where there is no possibility of putting a hand or a foot when the experience person stumbles is not subjected to the non-display process.

  In step S 1623, the virtual object drawn so far is displayed on the display 121 via the graphic engine 302.

  FIG. 17 is a diagram illustrating an example of an image displayed on the HMD 120 controlled by the mixed reality application according to the third embodiment. A virtual object teapot 1501, a window 1502, and a long desk 1503 are arranged in a room 1701 in the real space. This is a normal display state when the experience person is not staggered (acceleration is less than or equal to a predetermined value).

  FIG. 18 is an example of an image displayed on the HMD 120 when the experience person staggers in the mixed reality application. As described above with reference to the flowchart of FIG. 16, the teapot 1501 is displayed in a transparent manner (step S1614), the long desk 1503 is displayed with a section cut off by the front clipping plane 1801 (step S1619), and the window 1502 is affected. And displayed as usual (step S1621).

  According to the third embodiment, it is possible to avoid danger by the experience person without largely changing the image being experienced by hiding the position near the CG model.

  Also, by defining a non-display control method for each CG model, it can be made transparent for each model or part of the image can be hidden by clipping, thereby facilitating drawing control at the time of danger avoidance appropriate to the model. It becomes possible.

  According to the first to third embodiments, the non-display process in the present invention is a process including transparent display and non-display.

  In the first to third embodiments, the case where the experience person staggers has been described. However, the present invention is also applicable when the experience person runs. For example, when a distance of a predetermined range or more is moved within a predetermined time (when a predetermined moving speed is reached), the non-display control is performed. That is, the acceleration includes the moving speed.

  Since running with the HMD 120 leads to dangerous behavior, controlling the display so that danger can be avoided by hiding nearby CG models (including transmission) is effective when performing avoidance behavior. is there.

  Next, a functional block diagram of the present invention corresponding to the first to third embodiments will be described with reference to FIG.

  The speed acquisition unit 1901 is a functional unit that acquires a speed due to the movement of the imaging apparatus.

  The specifying unit 1902 is a functional unit that specifies a virtual object that is not displayed in a predetermined range in accordance with the position and orientation of the imaging device when the speed exceeds a predetermined value.

The drawing control unit 1903 hides the virtual image for the virtual object specified by the specifying unit 1902 and determines the virtual image determined according to the position and orientation of the imaging device so that the user can visually recognize the image in the real space. This is a functional unit that performs drawing control using a virtual image of an object and a real image of the real space. Further, the non-display process of the drawing control is a transparent process.
The drawing control unit 1903 is a functional unit that transmits the virtual image according to the transmission value determined according to the speed acquired by the speed acquisition unit 1901.
In addition, in a case where the drawing control unit 1903 is a virtual object that is not allowed to be transmitted by the determination unit 1904, the drawing control unit 1903 is determined to be a virtual object that is transmitted by the determination unit 1904 without transmitting the virtual image corresponding to the virtual object. It is a functional unit that controls transmission of a virtual image corresponding to a virtual object according to a transmission value.

Further, when the drawing control unit 1903 is set to be transparent by the transparency setting unit 1905,
This is a functional unit that controls the transmission of a virtual image using a transmission value higher than the previous transmission value.
The drawing control unit 1903 is a functional unit that combines the transmitted virtual image, the non-transmitted virtual image, and the real image.
The drawing control unit 1903 is a functional unit that controls drawing of a virtual object to be hidden according to the clipping plane determined by the clipping plane determination unit 1906.
In addition, the drawing control unit 1903 is a functional unit that controls drawing of a virtual image using a clipping plane that is retracted by a predetermined range when the determination unit 1904 determines that the virtual object is not displayed by clipping.

The determination unit 1904 is a functional unit that determines whether the drawing control unit 1903 is a virtual object to be transmitted.
The determination unit 1904 is a functional unit that determines whether or not the virtual object is hidden by clipping.

  The transmission setting unit 1905 is a functional unit that performs transmission setting when the speed acquired by the speed acquisition unit 1901 exceeds a predetermined value. Further, the transparency setting is canceled so that the virtual image is not transmitted when the transparency setting is being performed and the transparency value returns to the reference value.

The clipping plane determination unit 1906 is a functional unit that determines a clipping plane that has been retracted by a predetermined range.
The output unit 1907 is a functional unit that outputs the image synthesized by the drawing control unit 1903 to be displayed on the imaging apparatus.

  As described above, according to the present embodiment, in an emergency such as when the experience person staggers, the experience person can take a quick avoidance action by visually recognizing the live-action image. Further, there is no need to perform special processing on the model data to be displayed, and a mechanism for non-display control can be easily introduced.

  In particular, in the mixed reality system, HMD is used to synthesize not only a small virtual object in a live-action image but also a large part of the real-action image with a virtual image and display it in HMD. Is difficult. Therefore, when the experience person staggers and stumbles to avoid a fall, the user may suddenly touch the virtual object. Therefore, the avoidance behavior of the experience person is facilitated by transmitting / hiding only the image of the virtual object and recognizing the reality more. In addition, it is slow even if it hides at the timing of putting a hand, it is necessary to hide quickly when the acceleration reaches a predetermined value, and when the acceleration is acquired and exceeds the predetermined value, It is technically significant to hide nearby CG models.

  In addition, the mixed reality system synthesizes a live-action image and a virtual image and makes the virtual object appear to be real in the actual location. This causes confusion to the experience, especially when falling. Therefore, when the acceleration exceeds a predetermined value, it is possible to quickly take the fall avoidance action of the experience person while avoiding confusion of the experience person by transmitting the virtual image at a constant speed.

  It should be noted that the configuration and contents of the various data described above are not limited to this, and it goes without saying that the various data and configurations are configured according to the application and purpose.

  Although one embodiment has been described above, the present invention can take an embodiment as, for example, a system, apparatus, method, program, or recording medium, and specifically includes a plurality of devices. The present invention may be applied to a system including a single device.

  The program according to the present invention is a program that allows a computer to execute the processing methods of the flowcharts shown in FIGS. 4, 10, and 16, and the storage medium according to the present invention uses the processing methods of FIGS. A computer executable program is stored. Note that the program in the present invention may be a program for each processing method of each apparatus in FIGS. 4, 10, and 16.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by executing the reading.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk, solid state drive, or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  Further, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention.
In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

100 PC
120 HMD
121 Display 122 Camera 201 Position / Orientation Measurement Unit 202 Acceleration Measurement Unit 203 Screen Display Control Unit 204 Real Image Drawing Unit 205 CG Image Drawing Unit 206 Calibration Unit

Therefore, an object of the present invention is to provide a mechanism that allows an experienced person to take a quick avoidance action by drawing control of a virtual object using a display control method set for the virtual object according to the acceleration of the imaging apparatus .

To achieve the object of the present invention, a display information processing system capable of synthesizing image using a real image of the imaged real space in the imaging device and a virtual object, and acquires the acceleration from the imaging device and acceleration acquisition means, with respect to the virtual object, a storage means for storing display control scheme for one of the plurality of display control method, and displays the synthesized with the photographed image according to the position and orientation of the front Stories imaging apparatus A specifying means for specifying a virtual object; a reference means for referring to a display control method of the virtual object specified by the specifying means; and when the acceleration acquired by the acceleration acquiring means exceeds a predetermined value, the reference means wherein by controlling the drawing of the identified virtual object by the specific means in accordance with the referenced display control method, especially in that it comprises an output means for outputting the synthesized image using the photographed images of the real space To.

According to the present invention, the experience person can take a quick avoidance action by controlling the drawing of the virtual object using the display control method set for the virtual object according to the acceleration of the imaging apparatus .

Claims (11)

An information processing system for generating a composite image obtained by combining a real image of a real space captured by an imaging device and a virtual image of a virtual object,
Speed acquisition means for acquiring a speed due to movement of the imaging device;
A specifying means for specifying a virtual object that is not displayed in a predetermined range according to the position and orientation of the imaging device when the speed acquired by the speed acquisition means exceeds a predetermined value;
The virtual object specified by the specifying unit is hidden so that the experience person can visually recognize the image of the real space, and the virtual object corresponding to the position and orientation of the imaging device and the real image of the real space are used. An information processing system comprising drawing control means for controlling drawing. The non-display of the drawing control is transparent,
The information processing system according to claim 1, wherein the drawing control transmits a virtual image corresponding to the virtual object according to a transmission value determined according to a speed acquired by the speed acquisition unit. Determining means for determining whether or not the drawing control means is a virtual object to be transmitted;
In the case where the drawing control means is a virtual object that is not allowed to be transmitted by the determination means, the transmission control is performed when the virtual object corresponding to the virtual object is determined to be a transparent object that is transmitted by the determination means without being transmitted. The information processing system according to claim 2, wherein the virtual image corresponding to the virtual object is controlled to be transmitted according to the value. When the speed acquired by the speed acquisition means exceeds a predetermined value, further comprising a transparency setting means for setting transparency,
4. The transmission control unit according to claim 2, wherein when the transmission setting unit sets transmission, the drawing control unit performs transmission control of the virtual image using a transmission value higher than the previous transmission value. 5. Information processing system.   5. The information according to claim 4, wherein the transmission setting unit cancels the transmission setting so that the virtual image is not transmitted when the transmission setting is performed and the transmission value returns to a reference value. Processing system.   The information processing system according to claim 1, wherein the drawing control unit synthesizes a transmitted virtual image, a non-transmitted virtual image, and a real image. Clipping plane determining means for determining a clipping plane that has been retracted by a predetermined range is further provided,
7. The drawing control unit according to claim 1, wherein the drawing control unit performs drawing control based on the clipping plane determined by the clipping plane determination unit so that the virtual object is not displayed. The information processing system described. The determination means determines whether or not the virtual object is hidden by clipping,
The drawing control means controls the drawing of the virtual image using a clipping plane that is retracted by a predetermined range when the determination means determines that the virtual object is not displayed by clipping. Item 4. The information processing system according to Item 3.   9. The information processing system according to claim 1, further comprising output means for outputting an image synthesized by the drawing control means to be displayed on the imaging device. A processing method of an information processing system that generates a composite image obtained by combining a real image captured in an imaging device and a virtual object virtual image,
A speed acquisition step for acquiring a speed due to movement of the imaging device;
A specifying step of specifying a virtual object that is not displayed in a predetermined range according to the position and orientation of the imaging device when the speed acquired in the speed acquisition step exceeds a predetermined value;
The virtual object specified in the specifying step is hidden so that the user can visually recognize the image in the real space, and the virtual object according to the position and orientation of the imaging device and the real image in the real space are used. And a drawing control step for controlling drawing. A program for an information processing system that generates a composite image obtained by combining a real image captured in an imaging device and a virtual image of a virtual object,
Speed acquisition means for acquiring a speed due to movement of the imaging apparatus in the information processing system;
A specifying means for specifying a virtual object that is not displayed in a predetermined range according to the position and orientation of the imaging device when the speed acquired by the speed acquisition means exceeds a predetermined value;
The virtual object specified by the specifying unit is hidden so that the experience person can visually recognize the image of the real space, and the virtual object corresponding to the position and orientation of the imaging device and the real image of the real space are used. A program that functions as a drawing control means for controlling drawing.

Images