JP2009287933A - Motion tracker device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion tracker device for measuring the motion of a measuring object such as a helmet having a head-worn display device relative to a reference object such as a flight vehicle, even when the measuring object points to any direction relative to the reference object. <P>SOLUTION: This motion tracker device 1 includes: an optical marker 7 attached to the measuring object 10; a camera device 2 attached to the reference object 30; and a relative information calculation section 33 for calculating relative information. The motion tracker device 1 also includes: a reflector 5 that is attached to the reference object 30 and reflects light; and a determination result information creating section 34 for creating the determination result information of determination whether the optical marker 7 is directly photographed in a first image 15 and a second image 16 or is photographed via the reflector 5. An optical marker position information calculation section 32 calculates the optical marker position information, based on the determination result information, the first image 15, and the second image 16. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

The present invention relates to an optical motion tracker device (hereinafter also referred to as an “MT device”), for example, a current position and a current angle of a helmet with a head-mounted display device used in a game machine, a vehicle, etc. This is used for a head motion tracker device (hereinafter also referred to as “HMT device”) or the like that measures a wearer's head current position and current angle in a machine, a vehicle, or the like.

Techniques for accurately measuring the current position and current angle of an object (measurement target) that changes every moment are used in various fields. For example, in a game machine, an image is displayed by using a helmet with a head-mounted display device in order to realize virtual reality (VR). At this time, it is necessary to change the image in accordance with the current position and the current angle of the head mounted display-equipped helmet. Therefore, an HMT device is used to measure the current position and current angle of the head-mounted helmet with a display device (measurement object).
Also, in the rescue operation by the rescue flying boat, in order not to lose sight of the found rescue target, the aiming image displayed on the visor of the helmet with a head-mounted display device, the rescue target visually recognized through the visor, The position of the locked rescue target is calculated by locking when responding. At this time, in order to calculate the position of the rescue target, in addition to the latitude, longitude, altitude and attitude of the flying object (rescue flying boat), the current position and angle of the pilot's head relative to the flying object are measured. Yes.

As an HMT device used for such a helmet with a head-mounted display device, an apparatus that optically measures the current position and current angle of the helmet with a head-mounted display device is disclosed. FIG. 2 is a perspective view of an example of a helmet with a head-mounted display device, and FIG. 8 is a perspective view showing a positional relationship between the helmet with a head-mounted display device and the camera device shown in FIG.
Specifically, on the outer peripheral surface of the helmet 10 with a head-mounted display device, LEDs (light emitting diodes) 7a to 7k, which are light emitters, are attached to a plurality of locations as the optical marker group 7 so as to be separated from each other The positions of the LEDs 7a to 7k on the head mounted helmet with a display device (X′Y′Z ′ coordinate system) are stored in advance in the HMT device. The LEDs 7a to 7k are simultaneously photographed by the camera device 2 (first camera 2a, second camera 2b) fixed to the upper part of the seat 30a of the flying object (reference object), so-called triangulation. Based on the above principle, the positional relationship between the current at least three LEDs and the camera device 2 is measured. Thus, the current position of the head-mounted display-equipped helmet 10 with respect to the flying object is determined by specifying three positions (positions of three LEDs) fixed on the head-mounted display-equipped helmet 10. And the current angle is calculated.
By the way, in order to measure the positional relationship between the three LEDs and the camera device 2 using the optical HMT device as described above, it is necessary to identify the three LEDs. Therefore, LEDs 7a to 7k that can be individually identified are attached to the helmet 10 with a head-mounted display device. For example, as shown in FIG. 2, LEDs that emit infrared light having different wavelengths A, B, and C are attached to the outer peripheral surface of the helmet 10 with a head-mounted display device.

In addition, when identifying the plurality of LEDs attached to the head-mounted display-equipped helmet, the present applicant does not give identification information to the LEDs or turn on the LEDs one by one. However, the thing which can identify each LED was disclosed (for example, refer patent document 1).
Specifically, first, since the shooting direction of the first camera 2a and the shooting direction of the second camera 2b are determined, the first image shot by the first camera 2a by prediction based on epipolar geometry. And a common set of LED images between the second image captured by the second camera 2b.
Then, after associating each group of the plurality of LED images projected in the first image and the plurality of LED images projected in the second image, they are projected in the first image. The direction angle α from the first camera 2a is calculated from the position of the LED image being displayed, and the direction angle β from the second camera 2b is calculated from the position of the LED image displayed in the second image. Furthermore, by using the distance d between the first camera 2a and the second camera 2b, the position of the LED with respect to the camera device 2 (XYZ coordinate system) is calculated by a so-called triangulation method (see FIG. 4). . Note that it is not yet possible to recognize which LED calculated at this time corresponds to the LED attached to the head-mounted helmet with a display device.
Therefore, by storing the position of each LED at time t1 (immediately before) and setting each expected movement range that is a spherical shape of a certain size centered on the stored position of each LED, time t2 (currently ) In the expected movement range is identified as the same LED corresponding to the expected movement range set at time t1.
Thereby, the current position and the current angle of the helmet with a head mounted display device with respect to the camera device 2 (XYZ coordinate system) are calculated.
JP 2006-284442 A

By the way, in order to specify the current position and the current angle of the head mounted helmet 10 with the display device with respect to the camera device 2 with the HMT device as described above, the triangulation principle is used, so the image is taken by the first camera 2a. At least three identical LEDs had to exist in the imaging region where the imaging region to be imaged and the imaging region imaged by the second camera 2b overlapped (see FIG. 8). Therefore, as shown in FIG. 2, LEDs 7 a to 7 k are attached to the entire outer peripheral surface of the helmet 10 with a head-mounted display device.

However, even if the LEDs 7a to 7k are attached to the entire outer peripheral surface of the helmet 10 with a head-mounted display device, the same three LEDs cannot be photographed by the first camera 2a and the second camera 2b. There was a thing. For example, as shown in FIG. 9, in the second camera 2b, when a pilot wearing a helmet 10 with a head-mounted display device moves toward the right side and looks at the right side, only less than three LEDs are photographed. I could not. That is, the current position and current angle of the helmet 10 with a head mounted display device with respect to the camera device 2 could not be calculated.
Therefore, the present invention measures the movement of the measurement object relative to the reference object even when the measurement object such as the head-mounted display-equipped helmet is directed in any direction with respect to the reference object such as the flying object. An object of the present invention is to provide a motion tracker device that can perform the above-described operation.

The motion tracker device of the present invention, which has been made to solve the above-mentioned problems, is attached to a measurement object, and is provided with three or more optical markers that emit light and attached to a reference object, and images the optical marker. And a second camera that creates a second image by photographing an optical marker from a different direction from the first camera at the same time as the first camera photographs. An optical marker position information calculation unit that calculates optical marker position information that is a current position of the optical marker with respect to the camera device, and based on the optical marker position information, a current position and a current position of the measurement object with respect to the reference object A motion tracker device comprising a relative information calculation unit for calculating relative information including an angle, wherein the reference object A determination result for determining whether the optical marker is directly photographed in the reflection plate attached and reflecting the light, and the first image and the second image, or photographed through the reflection plate. A determination result information creation unit that creates information, and the optical marker position information calculation unit calculates the optical marker position information based on the determination result information, the first image, and the second image. Yes.

Here, examples of the “light emitted from the optical marker” include infrared light, ultraviolet light, and visible light.
According to the motion tracker device of the present invention, a plurality of optical markers are attached to the measurement object so as to be separated from each other, and the positions of the plurality of optical markers on the measurement object are stored in advance. In addition, the camera device and the reflector are attached to the reference object, and the relative positional relationship between the camera device and the reflector on the reference object is stored in advance.
Then, the camera device photographs at least three of the plurality of optical markers, and measures the positional relationship between the at least three optical markers and the camera device according to the so-called triangulation principle. . At this time, for example, in the camera device installed on the upper part of the seat of the flying object (reference object), when the wearer wearing the helmet (measurement object) is looking in front, Take three optical markers directly with two cameras. On the other hand, when a wearer wearing a helmet is looking at the right side while moving his body to the right side, the first camera directly shoots three optical markers, and the second camera directly shoots three optical markers. Although not possible, the three optical markers are photographed through the reflector.
Note that it is necessary to determine whether the optical marker is directly photographed in the first image and the second image or photographed via the reflector, but in the motion tracker device of the present invention, the determination is made. The result information creation unit makes a determination based on, for example, the light intensity of each optical marker image in the first image and the second image, the positional relationship of each optical marker image, and the like.
Accordingly, the optical marker position information calculation unit specifies the positions of the three points attached to the measurement target (positions of the three optical markers) based on the determination result information, the first image, and the second image. Thus, the current position and the current angle of the measurement object with respect to the reference object are calculated.

As described above, according to the motion tracker device of the present invention, even if the measurement object such as the helmet with a head-mounted display device is directed in any direction with respect to the reference object such as the flying object, the reference object The movement of the object to be measured can be measured.

(Means and effects for solving other problems)
Further, in the above invention, an optical marker lighting control unit for lighting at least three optical markers selected from the plurality of optical markers is provided, and each optical marker is selected from light of at least three types of wavelengths. Light of one type of wavelength set in advance may be emitted, and the optical marker lighting control unit may perform control so that the optical markers that are simultaneously turned on emit light of different types of wavelengths.
According to the present invention, the three optical markers can be identified by the wavelength difference.

Further, in the above invention, the determination result information creation unit is an optical marker image taken directly based on a light intensity of each optical marker image in the first image and the second image, or a reflection plate The determination result information may be created by determining whether the image is an optical marker image photographed via the.
In the above invention, is the determination result information creation unit an optical marker image group directly captured based on a positional relationship of the optical marker images in the first image and the second image? Alternatively, the determination result information may be created by determining whether the optical marker image group is photographed through a reflecting plate.
In the above invention, the determination result information creation unit creates a coordinate system in each of the first image and the second image, and changes from the first optical marker image to the second optical marker image. By calculating the sign of the outer product of the vector of the first vector and the vector from the first optical marker image to the third optical marker image, or a group of directly photographed optical marker images or photographed via a reflector It may be determined whether the optical marker image group has been selected.

In the above invention, each optical marker emits light of the same type of wavelength, and the determination result information creation unit is based on the light intensity of each optical marker image in the first image and the second image. The optical marker position information calculating unit creates the determination result information by determining whether the optical marker image is a direct image or an optical marker image captured through a reflector. Based on the determination result information, the optical marker position information may be calculated by creating an epipolar line of each optical marker image photographed by the first camera in the second image.
According to the present invention, it is possible to identify each optical marker without giving identification information to each optical marker or without lighting each one in turn.

In the above invention, the reference object is a moving body on which a wearer rides, and the measurement object is a hemispherical helmet mounted on the head of the wearer. Also good.
Furthermore, in the above invention, the optical marker may be a light emitting diode.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments described below, and it goes without saying that various aspects are included without departing from the spirit of the present invention.

FIG. 1 is a diagram showing an example of a schematic configuration of an HMT device according to an embodiment of the present invention, and FIG. 2 is a perspective view of an example of a helmet with a head-mounted display device (measurement object) shown in FIG. FIG. 3 and 5 are perspective views showing the positional relationship between the helmet with a head-mounted display device shown in FIG. 2 and the camera device, and FIGS. 4 and 6 show the head-mounted type shown in FIG. It is a top view which shows the positional relationship of a helmet with a display apparatus and a camera apparatus.
In the present embodiment, the current position and the current angle of the head of the pilot 3 who rides on the vehicle (reference object) 30 with respect to the vehicle 30 are calculated. That is, the HMT device 1 has a helmet coordinate system (X′Y ′) set in the helmet 10 with a head mounted display device worn by the pilot 3 with respect to the relative coordinate system (XYZ coordinate system) set in the vehicle 30. Relative information including the current position (X, Y, Z) and the current angle (Θ, Φ, Ψ) of the Z ′ coordinate system) is calculated. The relative coordinate system (XYZ coordinate system) is stored in advance in the control unit 20 with the camera device 2 described later as a reference, while the helmet coordinate system (X′Y′Z ′ coordinate system) is an LED described later. Pre-stored in the control unit 20 with the group 7 as a reference. The angle (Θ) is the angle in the roll direction (rotation with respect to the X axis), the angle (Φ) is the angle in the elevation direction (rotation with respect to the Y axis), and the angle (Ψ) is in the azimuth direction ( Angle of rotation with respect to the Z-axis).

The HMT device 1 includes a head mounted helmet 10 with a display device mounted on the head of a pilot 3, a camera device 2 (2 a, 2 b) attached to an upper portion of a seat 30 a of the vehicle body 30, and the vehicle body 30. The reflecting plate 5 (5a, 5b) that reflects infrared light attached to the front surface of the head and a control unit 20 configured by a computer.
The mounted body 30 is a cockpit of a flying body on which the pilot 3 is boarded, and includes a seat 30a on which the pilot 3 is seated.

The helmet 10 with a head-mounted display device includes a display (not shown), a visor 8 that guides the eyes of the pilot 3 by reflecting image display light emitted from the display, and a hemispherical helmet 9. And an LED group (optical marker group) 7 that functions as an index when measuring the current position (X, Y, Z) and the current angle (Θ, Φ, Ψ) of the head. In addition, the pilot 3 wearing the helmet 10 with a head-mounted display device can visually recognize the display image by the display and the front actual thing of the visor 8.
As shown in FIG. 2, the LED group 7 has a plurality of LEDs 7 a to 7 k attached to the entire outer peripheral surface of the helmet 9.
Here, the helmet coordinate system (X′Y′Z ′ coordinate system) will be described. Although the origin position O ′ and the X′Y′Z ′ axis direction in the helmet coordinate system (X′Y′Z ′ coordinate system) can be arbitrarily determined, in this embodiment, as shown in FIG. And the forward direction is defined as the X′-axis direction, the forward direction is defined as the Y′-axis direction, and the vertical direction is defined as the X′-axis direction and the Y′-axis direction as the Z′-axis direction. These are stored in advance in a helmet coordinate system storage area 42 described later. Further, the positions (x ′, y ′, z ′) of the plurality of LEDs 7 a to 7 k on the helmet coordinate system (X′Y′Z ′ coordinate system) are also stored in the helmet coordinate system storage area 42 in advance. .
The LEDs 7a, 7d, 7g, and 7j emit infrared light having a wavelength A, the LEDs 7b, 7e, 7h, and 7k emit infrared light having a wavelength B, and the LEDs 7c, 7f, and 7i have wavelengths C. Infrared light is emitted. That is, each LED emits infrared light having one preset wavelength from among three types of infrared light having wavelengths A, B, and C. Accordingly, when the optical marker lighting control unit 31 described later lights three LEDs selected from the plurality of LEDs 7a to 7k, the three LEDs to be turned on simultaneously have different types of wavelengths A and B. , C can be controlled to emit infrared light.

The camera device 2 is composed of a first camera 2a and a second camera 2b. The photographing direction is directed to the helmet 10 with a head-mounted display device, and stereoscopic viewing of the helmet 10 with a head-mounted display device is possible. It is installed in the upper part of the seat 30a of the vehicle body 30 so as to be separated by a possible constant distance d. Then, the first image 15 is created by the first camera 2a, and the second image 16 is created by the second camera 2b.
Accordingly, as shown in FIG. 4, when the LED 7c is directly photographed by the first camera 2a, the direction angle α from the first camera 2a is calculated from the position of the LED image displayed in the first image 15. When the second camera 2b directly captures the LED 7c, the direction angle β from the second camera 2b is calculated from the position of the LED image displayed in the second image 16, and the first camera 2a By using the distance d between the second camera 2b and the so-called triangulation method, the position of the LED 7c relative to the camera device 2 can be calculated.
At this time, a relative coordinate system (XYZ coordinate system), which is a coordinate system fixed to the camera device 2, is used to express the position of the LED 7c in spatial coordinates.
Here, the relative coordinate system (XYZ coordinate system) will be described. Although the origin position O and the XYZ axis directions in the relative coordinate system (XYZ coordinate system) can be arbitrarily determined, in this embodiment, as shown in FIGS. 4 and 6, the first camera 2 a to the second camera 2 b A direction is defined as a Y-axis direction, a direction perpendicular to the Y-axis direction, perpendicular to the ceiling, an upward direction as a Z-axis direction, a direction perpendicular to the Y-axis direction and the Z-axis direction, and a forward direction as an X-axis direction. Is stored in advance in the relative coordinate system storage area 41 so as to be defined as the midpoint between the first camera 2a and the second camera 2b.
Therefore, in the present embodiment, such a relative coordinate system (XYZ coordinate system) is set in the camera device 2 in advance, so that the current position of the LED 7c is a position (x in the relative coordinate system (XYZ coordinate system)). , Y, z). Since the current positions (x, y, z) of the three LEDs are specified in this way, the positions of the plurality of LEDs 7a to 7k on the helmet coordinate system (X'Y'Z 'coordinate system). By referring to the initial data of (x ′, y ′, z ′), the current position (X, Y, Z) and the current angle (Θ, Φ, Ψ) of the helmet 10 with a head mounted display device can be obtained. The current position and the current angle of the helmet coordinate system (X′Y′Z ′ coordinate system) relative to the relative coordinate system (XYZ coordinate system) are represented.

The reflection plate 5 includes a flat plate-shaped first reflection plate 5 a and a flat plate-shaped second reflection plate 5 b, and reflects the infrared light emitted from the LED group 7. Examples of the reflecting plate include those formed of a dielectric multilayer film that reflects infrared light and transmits visible light.
The first reflecting plate 5a is installed on the front surface of the vehicle 30 so that the reflecting surface thereof is directed to the first camera 2a and is photographed by the first camera 2a. The second reflecting plate 5b is installed on the front surface of the vehicle 30 so that the reflecting surface thereof is directed to the second camera 2b and is photographed by the second camera 2b.
The light intensity (luminance) of infrared light emitted from the LED group 7 is weaker when guided to the camera device 2 via the reflector 5 than when guided directly to the camera device 2. It is supposed to be. Thereby, the determination result information creation unit 34 to be described later directly captures the LED by the light intensity of the LED image displayed in the first image 15 and the second image 16, or passes through the reflector 5. It is possible to determine whether the image was taken. Further, since it is determined whether the LED is directly photographed or photographed via the reflector 5, the LED 7a is directly captured by the first camera 2a as shown in FIGS. When it is determined that the image has been taken, the direction angle α from the first camera 2a is calculated from the position of the LED image displayed in the first image 15, and the LED 7a passes through the second reflector 5b in the second camera 2b. Is determined, the direction angle γ from the second camera 2b and the direction angle δ from the second reflector 5b are calculated from the position of the LED image displayed in the second image 16, By using the distance d between the one camera 2a and the second camera 2b, the position of the LED 7a relative to the camera device 2 can be calculated. That is, the position (x, y, z) of the LED 7a on the relative coordinate system (XYZ coordinate system) can be calculated.

The control unit 20 is configured by a computer including a CPU 21, a memory 40, and the like, and performs various controls and arithmetic processes. The processing executed by the CPU 21 will be described separately for each functional block. An optical marker lighting control unit 31 that lights the LEDs 7a to 7k, an optical marker position information calculation unit 32, a relative information calculation unit 33, and determination result information creation A section 34 and a video display section 35.
The memory 40 is formed with an area for storing various data necessary for the control unit 20 to execute processing, and the relative coordinate system (XYZ coordinate system) and the position (x, y) of the reflecting plate 5 are formed. , Z), a relative coordinate system storage area 41, a helmet coordinate system (X′Y′Z ′ coordinate system), and a helmet coordinate system that stores the positions (x ′, y ′, z ′) of the LEDs 7a to 7k. And a storage area 42.

The optical marker lighting control unit 31 performs control for lighting three LEDs selected from the plurality of LEDs 7a to 7k. When three LEDs selected from the plurality of LEDs 7a to 7k are turned on, the three LEDs that are turned on simultaneously emit infrared light of different types of wavelengths A, B, and C. To control. For example, as shown in FIG. 2, infrared light having a wavelength A is emitted from the LED 7c, infrared light having a wavelength B is emitted from the LED 7d, and infrared light having a wavelength C is emitted from the LED 7e. Accordingly, it is possible to identify which LED image the LED image displayed in the first image 15 and the second image 16 is.

Based on the light intensity of each LED image in the first image 15 and the second image 16, the determination result information creation unit 34 is an LED image captured directly or an LED image captured via the reflector 5. The determination result information for determining whether the LED is directly photographed in the first image 15 and the second image 16 or photographed via the reflection plate 5 is created. Take control.
Specifically, a threshold value _Ath of light intensity (luminance) for determining whether the LED image is a directly captured LED image or an LED image captured via the reflector 5 is stored in advance. When the light intensity (luminance) A of one LED image in the first image 15 is _{equal to} or greater than the light intensity threshold _Ath , it is determined that the LED image is directly captured, while the light intensity A of the LED image is Is less than the light intensity threshold _Ath , it is determined that the LED image is taken through the first reflector 5a. Similarly, the light intensity A of another LED image in the first image 15 and the light intensity A of the LED image in the second image 16 are also determined.

Based on the determination result information and the first image 15 and the second image 16, the optical marker position information calculation unit 32 uses the current position (x, y, z) of the LED on the relative coordinate system (XYZ coordinate system). Control to calculate certain optical marker position information is performed.
For example, as shown in FIG. 4, when it is determined that the LED 7c has been directly photographed by the first camera 2a, the position from the first camera 2a depends on the position of the LED image of the wavelength C displayed in the first image 15. When the direction angle α is calculated and it is determined that the LED 7c is directly photographed by the second camera 2b, the direction angle from the second camera 2b is determined by the position of the LED image of the wavelength C displayed in the second image 16. By calculating β and using the distance d between the first camera 2a and the second camera 2b, the current position (x, y, z) of the LED 7c on the relative coordinate system (XYZ coordinate system) is calculated. .
Further, as shown in FIG. 6, when it is determined that the LED 7a is directly photographed by the first camera 2a, the position from the first camera 2a depends on the position of the LED image of the wavelength A displayed in the first image 15. The direction angle α is calculated, and when the second camera 2b determines that the LED 7a has been photographed through the second reflector 5b, the second camera 2b determines whether the LED image of the wavelength A projected in the second image 16 By calculating the direction angle γ from the second camera 2b and the direction angle δ from the second reflector 5b, and using the distance d between the first camera 2a and the second camera 2b, the relative coordinate system (XYZ coordinates) The current position (x, y, z) of the LED 7a on the system) is calculated.
In this way, the current position (x, y, z) of the three LEDs is calculated, and the optical marker position information is calculated.

Based on the optical marker position information, the relative information calculation unit 33 determines the current position (X, Y, Z) and the current angle (Θ, Φ) of the helmet 10 with a head-mounted display device relative to the relative coordinate system (XYZ coordinate system). , Ψ) is controlled to calculate relative information.
Specifically, since the current positions (x, y, z) of the three LEDs are specified, the positions of the plurality of LEDs 7a to 7k on the helmet coordinate system (X′Y′Z ′ coordinate system) ( The current position (X, Y, Z) and current angle (Θ, Φ, Ψ) of the helmet 10 with a head mounted display device are calculated by referring to the initial data as x ′, y ′, z ′). To do.
The video display unit 35 performs control for emitting video display light from the display based on the relative information. Thereby, the pilot 3 visually recognizes the display image by the display.

Next, the helmet coordinate system (X′Y) set in the helmet 10 with a head mounted display device worn by the pilot 3 with respect to the relative coordinate system (XYZ coordinate system) set in the vehicle 30 by the HMT device 1. A measurement operation for measuring the current position (X, Y, Z) and the current angle (Θ, Φ, ψ) of the “Z” coordinate system will be described. FIG. 7 is a flowchart for explaining the measurement operation by the HMT apparatus 1.
First, in the process of step S101, the optical marker lighting control unit 31 lights three LEDs selected from the plurality of LEDs 7a to 7k.
Next, in the process of step S <b> 102, the first camera 2 a and the second camera 2 b photograph the LEDs simultaneously to create the first image 15 and the second image 16.

Next, in the process of step S103, based on the light intensity of each LED image in the first image 15 and the second image 16, it is an LED image taken directly or an LED taken through the reflector 5 Determination result information is created by determining whether the image is an image.
Next, in the processing of step S104, the optical marker position information calculation unit 32 presents the current LED on the relative coordinate system (XYZ coordinate system) based on the determination result information and the first image 15 and the second image 16. Optical marker position information which is the position (x, y, z) is calculated.
Next, in the process of step S105, the relative information calculation unit 33, based on the optical marker position information, the current position (X, Y,...) Of the helmet 10 with head mounted display device with respect to the relative coordinate system (XYZ coordinate system). Z) and relative information including the current angle (Θ, Φ, ψ) is calculated.

Next, in the process of step S106, the video display unit 35 emits video display light from the display based on the relative information.
Next, in step S107, it is determined whether or not to end the measurement. If it is determined not to end the measurement, the process returns to step S101. That is, the processes in steps S101 to S106 are repeated until it is determined to end the measurement.
When it is determined that the measurement is to be finished, this flowchart is finished.

As described above, according to the HMT device 1, even if the helmet 10 with a head-mounted display device is directed in all directions with respect to the vehicle 30, the helmet 10 with a head-mounted display device with respect to the vehicle 30. Movement can be measured.

(Other embodiments)
(1) In the HMT device 1 described above, the determination result information creation unit 34 is an LED image taken directly based on the light intensity of each LED image in the first image 15 and the second image 16, or Although it was set as the structure which determines whether it is an LED image image | photographed via the reflecting plate 5, a coordinate system is created in each image of a 1st image and a 2nd image, and it is 2nd from a 1st LED image. By calculating the sign of the outer product of the vector to the LED image and the vector from the first LED image to the third LED image, it is a group of LED images taken directly, or taken through a reflector It is good also as a structure which determines whether it is the LED image group which carried out. At this time, the code in the outer product when the LED image group is directly photographed and the code in the outer product when the LED image group is photographed through the reflector are stored in advance and compared with the stored code. Thus, it may be determined whether the LED image group is a directly photographed LED image group or an LED image group photographed via a reflector.
(2) In the HMT device 1 described above, each LED is configured to emit infrared light having one preset wavelength from among three types of infrared light having wavelengths A, B, and C. Each LED may be configured to emit the same type of infrared light. At this time, the optical marker position information calculation unit calculates the optical marker position information by creating an epipolar line of each LED image photographed by the first camera in the second image based on the determination result information. You can do it.

The HMT device of the present invention is used, for example, as a device for measuring the current position and the current angle of a helmet with a head-mounted display device used in a game machine or a vehicle.

It is a figure which shows an example of schematic structure of the HMT apparatus which is one Embodiment of this invention. It is a perspective view of an example of a helmet with a head mounting type display device. It is a perspective view which shows the positional relationship of the helmet with a head mounting | wearing type display apparatus shown in FIG. 2, and a camera apparatus. It is a top view which shows the positional relationship of the helmet with a head mounted display apparatus shown in FIG. 2, and a camera apparatus. It is a perspective view which shows the positional relationship of the helmet with a head mounting | wearing type display apparatus shown in FIG. 2, and a camera apparatus. It is a top view which shows the positional relationship of the helmet with a head mounted display apparatus shown in FIG. 2, and a camera apparatus. It is a flowchart for demonstrating the measurement operation | movement by a HMT apparatus. It is a perspective view which shows the positional relationship of the helmet with a head mounting | wearing type display apparatus shown in FIG. 2, and a camera apparatus. It is a perspective view which shows the positional relationship of the helmet with a head mounting | wearing type display apparatus shown in FIG. 2, and a camera apparatus.

Explanation of symbols

1 Head Motion Tracker Device 2 Camera Device 3 Pilot 5 Light Source 7 LED (Optical Marker)
10 Helmet with head mounted display (measurement object)
15 First image 16 Second image 30 Carriage (reference object)
32 Optical marker position information calculation unit 33 Relative information calculation unit 34 Determination result information generation unit

Claims (8)

Three or more optical markers attached to the measurement object and emitting light; and
A first camera attached to a reference object and creating a first image by photographing the optical marker; and by photographing the optical marker from a different direction from the first camera at the same time as the first camera photographs. A camera device having a second camera for creating a second image;
An optical marker position information calculating unit that calculates optical marker position information that is a current position of the optical marker with respect to the camera device;
A motion tracker device comprising: a relative information calculation unit that calculates relative information including a current position and a current angle of a measurement object with respect to the reference object based on the optical marker position information;
A reflector attached to the reference object and reflecting the light;
A determination result information creating unit for creating determination result information for determining whether the optical marker is directly photographed in the first image and the second image or photographed via a reflector;
The optical marker position information calculation unit calculates the optical marker position information based on the determination result information, the first image, and the second image. An optical marker lighting control unit for lighting at least three optical markers selected from the plurality of optical markers;
Each optical marker emits light of one kind of preset wavelength from light of at least three kinds of wavelengths,
The motion tracker device according to claim 1, wherein the optical marker lighting control unit controls the optical markers that are simultaneously turned on to emit light of different types of wavelengths. The determination result information creation unit is an optical marker image taken directly based on the light intensity of each optical marker image in the first image and the second image, or an optical marker taken via a reflector The motion tracker device according to claim 2, wherein the determination result information is created by determining whether the image is an image. The determination result information creation unit is a group of optical marker images taken directly based on the positional relationship of the optical marker images in the first image and the second image, or via a reflector. The motion tracker device according to claim 2, wherein the determination result information is created by determining whether the optical marker image group is a photographed image. The determination result information creation unit creates a coordinate system in each of the first image and the second image, a vector from the first optical marker image to the second optical marker image, and a first By calculating the sign of the outer product of the vector from the optical marker image to the third optical marker image, it is an optical marker image group directly photographed, or an optical marker image group photographed through a reflector. The motion tracker device according to claim 4, wherein the motion tracker device is determined. Each optical marker emits light of the same type of wavelength,
The determination result information creation unit is an optical marker image taken directly based on the light intensity of each optical marker image in the first image and the second image, or an optical marker taken via a reflector The determination result information is created by determining whether the image is an image,
The optical marker position information calculation unit calculates the optical marker position information by creating an epipolar line of each optical marker image photographed by the first camera in a second image based on the determination result information. The motion tracker device according to claim 2, wherein: The reference object is a moving body on which a wearer rides, and the measurement object is a hemispherical helmet to be worn on the head of the wearer. The motion tracker device according to any one of the above. The motion tracker device according to claim 1, wherein the optical marker is a light emitting diode.

Images