JP2004126869A - Position/attitude setting device, conversion parameter setting method and composite reality feeling system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a re-calculation operation for a conversion parameter of a coordinate system, when variation occurs to a real space object, arranged in a composite real space and to an environment in a composite reality feeling device which composites a real space video image and a virtual space video image. <P>SOLUTION: A composite reality feeling system generates a composite real image by merging the real space video image, which can be viewed in the position of the view point of a viewer or in a viewer's line-of-sight direction, and the virtual space video image together, and displays the composite real image on an HMD 110. In this case, when a conversion parameter, which converts the coordinate system of a position sensor 113 for the eyes to the global coordinate system of a composite real space, is set, a real object arrangement device 215 is arranged according to an arrangement mark 210, and the HND 110 is arranged at a prescribed position in the real object arrangement device. A real index 220, which the real object arrangement device 215 has, is displayed as a virtual index on the display screen of the HND 110. In adjusting the attitude of the HMD 110, the conversion parameter is set by making the position of the virtual index coincide with that of the real index in the composite real image displayed. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention is a mixed reality device that generates a mixed reality space image by synthesizing a real space image captured from a video camera or the like and a virtual space image in a viewpoint position and a line-of-sight direction measured by a position and orientation measurement device. This is related to initial alignment.
[0002]
[Prior art]
In the mixed reality system, it is necessary to acquire the three-dimensional position and orientation of the real space object by some means in order to fuse the real space and the virtual space. One of the general methods for acquiring a three-dimensional position and orientation is to use a position and orientation measurement device typified by FASTRAK (trademark) manufactured by Polhemus, USA.
[0003]
When such a position and orientation measurement device is used, the viewpoint position and line of sight of the observer and the position and orientation of the real object are obtained as values in the position and orientation measurement device coordinate system. Therefore, it is necessary to convert the obtained coordinate values into a coordinate system of a mixed reality space, that is, a world coordinate system. In other words, a transformation parameter from the position and orientation measurement device coordinate system to the world coordinate system is required. As a means for obtaining this, a method using a real index and a virtual index is known. The specific procedure will be described below.
[0004]
First, a real index is set in advance on a real object installed in a real space used as a mixed reality space. Then, the positions of these real indices in the world coordinate system are measured, and these values are given to the virtual indices. That is, the positions of the real index and the virtual index are set to be the same. Subsequently, the reference position and orientation are set so that these virtual indices are displayed on the display screen. Then, the virtual indices at the reference position and orientation are displayed on the display device, and the position and orientation of the display device are adjusted and fixed so that the virtual indices match the real indices. The output of the position and orientation measurement device at this point is recorded. Conversion parameters are obtained from the values set or measured in the above series of operations.
[0005]
[Problems to be solved by the invention]
The real space object and the real space environment used as the mixed reality space need to be changed according to the content and the scenario of the mixed reality space presented to the observer. In some cases, the real space object on which the reality index is set may be replaced with another real space object, or the location of the real space object may be changed. In such a case, it is necessary to recalculate the conversion parameters.However, in the conventional method, the change of the setting position of the real index, the measurement of the setting position of the real index, the change of the position of the virtual index, and the recalculation of the reference posture are required. You will need it.
[0006]
The present invention has been made in view of the above problems, and has been made to reduce the recalculation of coordinate system transformation parameters when there is a change in a real space object or environment placed in a mixed reality space. Aim.
[0007]
[Means for Solving the Problems]
A position and orientation setting device according to the present invention for achieving the above object,
In a mixed reality system that generates and displays a mixed reality image that combines a real space image that can be observed at the observer's viewpoint position and line of sight and a virtual space image, the coordinate system of the position and orientation measurement means is set to the mixed reality space. An instrument for setting conversion parameters for converting to a world coordinate system,
An installation unit for installing a display unit for displaying the mixed reality video,
And an index disposed on the main body of the device and used when acquiring the conversion parameter.
[0008]
Further, to achieve the above object, the conversion parameter setting method according to the present invention,
A coordinate system of measuring means for measuring the position and orientation of a real object in a mixed reality system that generates and displays a mixed reality image that combines a real space image and a virtual space image that can be observed at the observer's viewpoint position and gaze direction. Is a method of setting transformation parameters for transforming into a world coordinate system of mixed reality space,
Placing the real object at a predetermined position on a setting device installed at a predetermined place,
A virtual image representing a part of the setting tool is displayed at a predetermined position on a display device, and a real space image observed at the position and orientation of the real object is displayed on the display device.
The conversion parameter is set by adjusting the posture of the real object so that a part of the setting tool based on the virtual image matches a part of the setting tool based on the real space image.
[0009]
Furthermore, the mixed reality system according to the present invention for achieving the above object,
A mixed reality system that generates a mixed reality image that combines a real space image and a virtual space image that can be observed at the observer's viewpoint position and line-of-sight direction, and displays the mixed reality image on a display device,
Specifying means for specifying a setting mode for setting a conversion parameter for converting a coordinate system of measuring means for measuring the position and orientation of the real object into a world coordinate system of mixed reality space;
A setting tool that can be installed at a predetermined location, and the setting tool can be disposed at a predetermined position so that the posture of the real object can be changed,
When the setting mode is specified by the specifying unit, a virtual image corresponding to a part of the setting device is displayed at a predetermined position on the display device, and a real space image observed in a position and orientation of the real object is displayed. Display control means for displaying on a display device;
When a predetermined notification is given from the user, on the display of the display device, it is considered that a part of the setting tool based on the virtual image matches a part of the setting tool based on the real space image, and the Calculating means for calculating the conversion parameter based on the measurement result of the measuring means.
[0010]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
[0011]
<First embodiment>
FIG. 1 is a diagram illustrating an overall configuration of a mixed reality system that is operated after acquiring a coordinate system conversion parameter obtained using the real object installation tool according to the first embodiment.
[0012]
In FIG. 1, an observer 100 wears a head-mounted display (HMD) 110 on the head and a glove 120 on a hand.
[0013]
As shown in FIG. 2, the HMD 110 includes a video camera 111, an LCD 112, a position and orientation measuring device receiver (eye position sensor) 113, and optical prisms 114 and 115. The video camera 111 captures a real space image of the observer's viewpoint position and line of sight guided by the optical prism 115. The eye position sensor 113 measures the observer's viewpoint position and line-of-sight direction. The LCD 112 displays a mixed reality space image, and the image is guided to the observer's pupil by the optical prism 114.
[0014]
The glove 120 incorporates a hand position sensor 121 and a speaker (not shown). The hand position sensor 121 measures the position and direction of the observer's hand. Further, a sound is output from the speaker in accordance with an event occurring at the position of the hand. For example, it is a sound generated when the virtual space object is touched or hit with a hand, or a sound generated when the state of the virtual space object displayed in synchronization with the position of the hand changes.
[0015]
130 is a position and orientation measurement device transmitter (position sensor transmitter), and 131 is a position and orientation measurement device main body (position sensor main body). The eye position sensor 113, the hand position sensor 121, and the position sensor transmitter 130 are connected to the position sensor main body 131. The magnetism is transmitted from the position sensor transmitter 130, and the eye position sensor 113 and the hand position sensor 121 receive the magnetism. The position sensor body 131 calculates the position and direction of each position sensor from the reception intensity output from each position sensor. Here, as the position and orientation measurement device, FASTRAK (trademark) manufactured by Polhemus, USA, or the like can be used.
[0016]
A processing device 140 generates a mixed reality space image for one observer and displays the mixed reality image on the HMD 110. For example, it includes a personal computer, a video capture card, a video card having a CG drawing function, and a sound card. The HMD 110, the speaker 122, and the position sensor main body 131 are connected to the processing device 140.
[0017]
In FIG. 1, there is only one observer. However, a plurality of observers are prepared at the same time by preparing a plurality of the same device configurations as in FIG. It is also possible to share. However, since it departs from the gist of the present invention, the description thereof is omitted here.
[0018]
Reference numeral 160 denotes a real space area used as a mixed reality space by the present system. The present system synthesizes a virtual space image only in this area. Fusion of virtual space images is not performed for the real space outside this area.
[0019]
Reference numeral 170 denotes an example of a real object installed in the mixed reality space. The present system generates a rational mixed reality video by judging the contact between the real object and the virtual object and the context of the virtual object viewed from the viewpoint. For example, when the virtual object is located behind these real objects when viewed from the viewpoint, the viewer is provided with an image in which the whole or a part of the virtual object appears to be covered by the real object. In order to perform such processing, the present system holds the shape information and installation positions of these real objects.
[0020]
FIGS. 3 to 5 are flowcharts illustrating a procedure in which a program operating in the processing device 140 generates a mixed reality space image. The program has two modes, a "normal mode" in which the observer can experience the mixed reality space, and an "initial setting mode" in which correction parameters (part of the conversion parameters) are acquired.
[0021]
First, the processing device 140 starts the main program. The main program first sets the operation mode to the "normal mode" (step S300), and starts the position and orientation thread, the video capture thread, and the video generation thread. (Steps S301 to S303) Here, a thread is a series of programs that are processed in parallel in the processing device 140. Therefore, information such as the operation mode is shared by the main program and all threads.
[0022]
After starting each thread, the main program enters an event wait state and continues to wait until an event arrives (step S304). Here, the event is an input from a user interface, information notification from another thread, a hardware interrupt, and the like. Specifically, an operation mode is changed (step S305), an initial parameter is changed (step S306), and a correction parameter is obtained. (Step S320), virtual index position change (Step S330), end event (Step S307), and the like.
[0023]
When these events have arrived, the corresponding processes (steps S310, S311, S321, S331, S308) are performed. Although a specific description of these processes is omitted, it is assumed that all system instructions such as inputting and changing values can be performed by a general user interface such as a window system.
[0024]
In the present embodiment, the conversion parameter is a generic term for the combination of the initial parameter and the correction parameter. These will be described later.
[0025]
The video capture thread started in step S302 captures the video captured by the video camera 111 as digital data, and continuously outputs the digital data to the main thread. This operation is repeatedly performed unless there is an end notification.
[0026]
The processing of the position and orientation thread started in step S301 will be described with reference to FIG. First, a measured value is obtained from the position sensor main body 131 (step S401), and processing is performed according to the operation mode (step S402). Here, if the operation mode is the “normal mode”, the position and orientation of the measurement target object in the world coordinate system are calculated from the measured values and the conversion parameters, and output to the main thread (steps S403 and S404). On the other hand, if the operation mode is the “initial setting mode”, a correction parameter is calculated from the measured value and the reference posture, and the reference position and posture are output to the main thread (steps S406 and S407). The above operation is repeatedly performed until the end notification is received (step S405).
[0027]
Next, the processing of the video generation thread started in step S303 will be described with reference to FIG.
[0028]
First, upon receiving an output from the position and orientation thread, the position and orientation of the measurement target object are updated (steps S501 and S510). Subsequently, the time of the virtual space is updated to update the state of the virtual space (the type, position, and state of the virtual space object) (step S502). At this time, if there is a virtual space object whose position and orientation changes in synchronization with the position and orientation of the real space object, the state is updated accordingly. For example, when it is shown that the glove of the virtual space object is always worn on the hand, the position and orientation of the glove are updated in this step. In addition, a process for judging the anteroposterior relationship of the virtual object viewed from the viewpoint and making it appear that part or the whole of the virtual object is covered by the real object is also performed here.
[0029]
Next, the relationship between the position and orientation of the real space object (such as the position and orientation of the hand) and the position and orientation of the virtual space object is examined. If it is determined that a predefined event has occurred, the virtual space The state is updated (step S503). For example, when a virtual space object is touched with a hand, the virtual space object explodes.
[0030]
Subsequently, a real space image at the viewpoint position and the line of sight of the observer obtained from the video camera 111 is captured (steps S504 and S520). If the operation mode is the “normal mode” (step S530), the virtual space image obtained from the viewpoint position and the line-of-sight direction of the observer acquired in step S501 is changed according to the state of the virtual space updated in steps S502 and S503. (Step S505). On the other hand, if the operation mode is the “initial setting mode”, the virtual index is drawn at a preset position (step S531).
[0031]
Finally, the virtual space image generated in step S505 or S531 and the real space image captured in step S504 are combined and output to the LCD 112 of the HMD 110 (step S506). The above operation is repeated until a termination instruction is issued (step S507).
[0032]
In the foregoing, the devices constituting the mixed reality system of the present embodiment and the operation of generating a mixed reality space image have been described. Next, the principle of calculating the position and orientation in step S403 and acquiring the correction parameters in step S406 will be described in detail.
[0033]
FIG. 6 is a diagram showing the arrangement of each coordinate system. G is a world coordinate system, T is a transmitter coordinate system of the position and orientation sensor, S is a receiver coordinate system of the position and orientation sensor, and E is a viewpoint coordinate system. S ′ and E ′ are a sensor coordinate system and a viewpoint coordinate system when the viewpoint position and the line-of-sight direction are fixed at the reference position and orientation, respectively. Also, P is a point (viewpoint) from which a position is to be obtained, and P 'is a reference position.
[0034]
Here the position of the point P as viewed from the coordinate system A to be expressed as ^{A P.} Also between positions ^{B P} of the point P as viewed from the position ^{A P} and the coordinate system B of a point P as viewed from the coordinate system A is a rotation matrix _{B A} ^R describes the orientation of the coordinate system A and the coordinate system B use ^it, it is known that can be expressed as _{^{a P = B a R B P}} .
[0035]
From the above, the position ^{G P} and orientation _{E G} ^R on the world coordinate system of the point P to be determined,
_{^{G P = G P'- E 'G}} R E' P '+ T G R (T P Eorg - T P' Eorg) + E G R E P
_{^{E G R = E 'G R}} S' T R -1 S T R
Can be written as
[0036]
_Here, if the installation position and orientation sensor transmitter such that ^{T G R = I, G P} ' is the reference position, _E' ^G R is the reference ^{posture, E 'P', E} P is to the measurement point from the sensor ^offset, T _{P EORG} position ^{measurements, T} P _'Eorg position measurement at the reference position and _orientation, ^{E G} R attitude calculated _value, ^{S T} R attitude measurements, _S' ^T R ^-1 is the reference position It is the inverse matrix of the posture measurement values at the posture, and all can be represented by a known matrix. That is, the calculation can be performed without depending on the coordinate system of the position and orientation transmitter.
[0037]
Note that the transformation parameter as referred in the present embodiment, the position measuring values ^T P _Erog, is a generic term of the matrix other than the orientation measurement value _{S T} ^R. The conversion parameters are divided into correction parameters and initial parameters.
[0038]
The correction parameters are two of the position measurement value ^T P ′ _Eorg at the reference position and orientation and the inverse matrix _{S ′} ^T R ⁻¹ of the orientation measurement values at the reference position and orientation. The correction parameters are calculated in step S406, and are stored in the system in the processing in steps S320 and S321.
[0039]
The initial parameter reference position ^G P ', the reference posture _E' ^G R, offset ^E from the sensor to the measurement point ^'P', which is one third ^E P. As the initial parameter, the value input in S311 or the value held by the processing device is used.
[0040]
The position and orientation calculation and conversion parameters have been described above. Next, a procedure for setting a conversion parameter will be described.
[0041]
First, a real index is set in advance on a real object installed in a real space used as a mixed reality space. Then, the positions of these real indices in the world coordinate system are measured, and these values are given to the virtual indices (step S331). The number of virtual indices is not particularly limited, and may be any number. Whenever the position of the real index is changed, the position of the real index in the world coordinate system must be re-measured and changed in step S331.
[0042]
Subsequently, the reference position and orientation are set so that these virtual indices are displayed on the display screen. That is, the initial parameters are input (steps S306 and S311). Subsequently, the operation mode is changed to the "initial setting mode" (step S310). In the “initial setting mode”, the position and orientation measurement values of the HMD 110 are fixed at the reference position and orientation (step S407). The virtual index 190 is drawn and displayed (steps S530 and 531).
[0043]
Subsequently, the installation tool is arranged, and the HMD 110 is installed at a predetermined position of the installation tool. Then, the position and orientation of the display device are adjusted and fixed so that the virtual index and the real index appear to match. Finally, the correction parameter at this point (inverse matrix of the position measurement value at the reference position and orientation and the posture measurement value at the reference position and orientation) is obtained (steps S320 and S321). With the above operations, the conversion parameters are obtained.
[0044]
First, a procedure for acquiring a correction parameter according to a general method will be described.
[0045]
Since the basic procedure may be performed as described above, here, a portion for setting the HMD 100 at the reference position and orientation will be described in detail.
[0046]
FIG. 7 is a diagram showing a state where the HMD 110 is installed at the reference position and orientation in a general manner. To fix the HMD 110 at the reference position and orientation, a real object installation tool 200 is used. The real object installation device 200 must be arranged so that the HMD 110 as a real object can be installed at a reference position. 210 is an installation mark for that purpose. The installation mark 210 is attached in advance to a predetermined position in the world coordinate system so that the HMD 110 can be installed at the reference position by placing the real object installation tool 200 on the installation mark 210.
[0047]
To install the HMD 110 at the reference position, the HMD 110 may be fixed at a predetermined position of the real object installation device 200 in this state. The real object installation tool 200 and the HMD 110 are marked in advance, and the HMD 110 is placed on the real object installation tool 200 so that the marks overlap each other, so that the HMD 110 is designed in advance so that it can be installed at the reference position. Have been. In order to be more accurate, it is desirable to create and use a dedicated mounting device.
[0048]
To install the HMD 110 in the reference posture, the virtual index and the real index may be matched on the display screen in the “initial setting mode”. 8 and 9 show the display state of the HMD 110 when the virtual index 190 is displayed. The display position of the virtual index is a position observed when the HMD 110 is fixed at a predetermined position and orientation using the real object installation tool 200, and does not change during the display in the initial setting mode. When the virtual index 190 and the real index 180 do not match on the display screen as shown in FIG. 8, the HMD 110 is rotated so as not to deviate from the reference position, and the virtual index and the real index match as shown in FIG. To do. When the virtual index and the real index match on the display screen, the HMD 110 is set in the reference posture.
[0049]
Next, a procedure for acquiring the correction parameters of the HMD 110 according to the present embodiment will be described. Since the basic procedure is the same as that of the conventional method, here, the instrument to be installed at the reference position and orientation will be described in detail.
[0050]
FIG. 10 is a diagram illustrating a state in which the HMD 110 according to the present embodiment acquires correction parameters. The correction parameters are acquired in the same procedure as the method described above with reference to FIGS. 7 to 9. However, in the configuration illustrated in FIG. 10, the real index 220 is set on the real object setting tool 215.
[0051]
The display state of the HMD 110 when the virtual index 190 is displayed is shown in FIGS. Similar to the method described with reference to FIGS. 7 to 9, as shown in FIGS. 11 and 12, the real index 220 (provided on the real object setting device 215) and the virtual index 190 match. The HMD 110 is rotated (the posture of the HMD 110 is changed). Note that the real object installation tool 215 is provided with a portion on which the user wearing the HMD 110 places a chin, and by placing the chin on this portion, the HMD can be installed at the reference position. The HMD body may be left as it is. However, when the user does not wear the HMD, an observation image displayed on the HMD is displayed on an external monitor, and positioning is performed using the observation image.
[0052]
Although not particularly described, it is added that the location and the size of the reality indicator 220 are made to be recognizable when present on the display device so that the existence and overall shape thereof can be recognized. Further, a light-emitting element or the like may be used as the real index 220 to further facilitate recognition on the display screen.
[0053]
The correction parameter acquisition procedure, the installation tool, and the method of use in the conventional method and the method of the present embodiment have been described above.
[0054]
As described above, in the general index setting method, when the real object is changed, the real index is reinstalled, the position of the real index is measured, the position of the virtual index is changed based on the measurement result, and the virtual index is displayed on the HMD 110. The reference position and orientation must be recalculated to be within the range.
[0055]
On the other hand, if the installation tool 215 according to the above-described embodiment is used, the real object that is to be a real index is held by the installation tool that is installed only when setting the conversion parameter. Even when the environment changes, the relative position and orientation between the reference position and orientation and the virtual index are constant. Therefore, it is not necessary to change the setting position of the real index, measure the setting position of the real index, change the position of the virtual index, and recalculate the reference posture.
[0056]
In the case of the index setting method using a general method, in the case of a mixed reality system in which a plurality of observers face each other and stay at an almost constant place and share a mixed reality space, an initial position is set at each observer's position. Therefore, it is necessary to prepare a reality index for each observer. Therefore, as the number of observers increases, the management of the index increases. On the other hand, if the installation tool of the present embodiment is used, the time and effort for managing the reality index will be greatly reduced. Also, the position calculation of the real index and the calculation of the initial parameters can be recalculated by a simple offset calculation.
[0057]
Further, it is possible to remove a reality index that should not originally exist in the mixed reality space when the mixed reality space system is operated, and to provide a more realistic mixed reality space image.
[0058]
<Second embodiment>
In the first embodiment, the position / posture adjustment is performed by setting the real index 220 on the real object setting device 215 and matching the virtual index on the observation image. In the second embodiment, position / posture adjustment is performed using a part of the real object installation tool 215.
[0059]
FIGS. 13 and 14 show display screens in the “initial setting mode” according to the second embodiment. In the second embodiment, the installation tool itself is used as a real index. Correspondingly, the virtual index is a point and a small piece on the display screen in the first embodiment, but is represented by the three-dimensional computer graphics wire-frame model 230 of the real index in the second embodiment. I have. In the second embodiment, the wire frame and the actual installation tool are matched, but the basic configuration and operation are the same as in the first embodiment.
[0060]
When matching the virtual index and the real index on the display screen, if the index is a plurality of small pieces, it is easy to concentrate on matching one index and shift another index. In the second embodiment, since the displacement can be recognized on the entire screen, the alignment can be performed more easily.
[0061]
According to the second embodiment, an installation tool for fixing a real object to be measured for position and orientation including a display device to a predetermined position and orientation-reference position and orientation is caused to hold a real object to be a reality index. , The relative position and orientation between the reference position and orientation and the virtual index are kept fixed. With this, if the reference position and orientation are set so that the reality index is within the display screen only once when the installation equipment is created, the installation position of the real index even if the real space object used as the mixed reality space and the environment change. There is no need to change the position, measure the installation position of the real index, change the position of the virtual index, and recalculate the reference attitude. Also, when installing and using the installation equipment in another place in the world coordinate system, it is not necessary to measure the position of the real index installed on the surface of the real object having a complicated shape, and simple offset calculation Allows the reference posture to be recalculated.
[0062]
As described above, according to the first embodiment, the relative position and orientation between the reference position and orientation and the virtual index are constant even if the real space object used as the mixed reality space and the environment change. The effects of changing the setting position of the index, measuring the setting position of the real index, changing the position of the virtual index, and reducing the recalculation of the reference posture can be obtained. Further, in the second embodiment, in addition to the effects of the first embodiment, an effect that the real index and the virtual index can be more easily matched on the screen can be obtained.
[0063]
【The invention's effect】
As described above, according to the present invention, it is possible to reduce the recalculation of coordinate system conversion parameters when there is a change in a real space object or environment placed in a mixed reality space.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an appearance of a mixed reality system according to an embodiment.
FIG. 2 is a diagram illustrating a configuration of an HMD according to the embodiment.
FIG. 3 is a flowchart showing processing by a mixed reality space image generation program (main program).
FIG. 4 is a flowchart showing processing by a position and orientation thread.
FIG. 5 is a flowchart showing processing by a video generation thread.
FIG. 6 is a diagram illustrating the arrangement of each coordinate system.
FIG. 7 is a diagram for describing conversion parameter acquisition by a general method.
FIG. 8 is a diagram showing an HMD display state when a conversion parameter is obtained by a general method.
FIG. 9 is a diagram illustrating a display state of an HMD at a reference position and orientation when a conversion parameter is obtained by a general method.
FIG. 10 is a diagram for describing conversion parameter acquisition in the first embodiment.
FIG. 11 is a diagram illustrating an HMD display state at the time of acquiring a conversion parameter according to the first embodiment.
FIG. 12 is a diagram illustrating an HMD display state in a reference position and orientation at the time of acquiring a conversion parameter according to the first embodiment.
FIG. 13 is a diagram illustrating an HMD display state at the time of acquiring a conversion parameter according to the second embodiment.
FIG. 14 is a diagram illustrating an HMD display state in a reference position and orientation at the time of acquiring a conversion parameter according to the second embodiment.

Claims (8)

In a mixed reality system that generates and displays a mixed reality image that combines a real space image that can be observed at the observer's viewpoint position and line of sight and a virtual space image, the coordinate system of the position and orientation measurement means is set to the mixed reality space. An instrument for setting conversion parameters for converting to a world coordinate system,
An installation unit for installing a display unit for displaying the mixed reality video,
A position and orientation setting device, comprising: an index, which is arranged on a main body of the device and is used when acquiring the conversion parameter. The position and orientation setting device according to claim 1, wherein the index includes a light emitting element. A coordinate system of measuring means for measuring the position and orientation of a real object in a mixed reality system that generates and displays a mixed reality image that combines a real space image and a virtual space image that can be observed at the observer's viewpoint position and gaze direction. Is a method of setting transformation parameters for transforming into a world coordinate system of mixed reality space,
Placing the real object at a predetermined position on a setting device installed at a predetermined place,
A virtual image representing a part of the setting tool is displayed at a predetermined position on a display device, and a real space image observed at the position and orientation of the real object is displayed on the display device.
Setting the conversion parameter by adjusting a posture of the real object to match a part of the setting tool based on the virtual image with a part of the setting tool based on the real space image. Method. 4. The conversion parameter setting method according to claim 3, wherein the setting device is installed at the predetermined location only when the conversion parameter is set. The conversion parameter setting method according to claim 3, wherein a part of the setting device is an index used for setting a conversion parameter. The conversion parameter setting method according to claim 3, wherein a part of the setting tool is a predetermined part of the setting tool. The method according to claim 6, wherein the virtual image representing a part of the setting tool displayed on the display device is a wire-frame model corresponding to the part of the setting tool. A mixed reality system that generates a mixed reality image that combines a real space image and a virtual space image that can be observed at the observer's viewpoint position and line-of-sight direction, and displays the mixed reality image on a display device,
Specifying means for specifying a setting mode for setting a conversion parameter for converting a coordinate system of measuring means for measuring the position and orientation of the real object into a world coordinate system of mixed reality space;
A setting tool that can be installed at a predetermined location, and the setting tool can be disposed at a predetermined position so that the posture of the real object can be changed,
When the setting mode is specified by the specifying unit, a virtual image corresponding to a part of the setting device is displayed at a predetermined position on the display device, and a real space image observed in a position and orientation of the real object is displayed. Display control means for displaying on a display device;
When a predetermined notification is given from the user, on the display of the display device, it is considered that a part of the setting tool based on the virtual image matches a part of the setting tool based on the real space image, and the Calculating means for calculating the conversion parameter based on a measurement result of the measuring means.

Images