JP2011015831A - Head motion tracker - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a head motion tracker capable of calculating the present position and present angle of the head part of an occupant relative to the inside of a cockpit in a wide range when performing measuring without constraining a flying body or the like for a long period of time before performing the measuring.SOLUTION: The head motion tracker 1 includes: a mounting body 6 mounted on the head part of the occupant P; a camera 2 for acquiring image data; and a control part 20 for calculating head part information which is the present position and/or present angle of the head part of the occupant P relative to the inside of the cockpit 30 where the occupant P is boarding on the basis of the image data. In this case, a storage part 40 for storing the ambient environment information of the inside of the cockpit 30 is provided. The camera 2 is attached to the mounting body 6, the image data are acquired by imaging the inside of the cockpit 30, and the control part 20 calculates the heat part information on the basis of the image data and the ambient environment information.

Description

  The present invention relates to a head motion tracker (HMT) device having a function of calculating head information that is a current position and / or current angle of a passenger's head. The present invention is used for, for example, an HMT device that detects a current position and a current angle of a helmet with a head-mounted display device used in a game machine or a vehicle.

  A technique for accurately measuring the current position and current angle of an object that changes from moment to moment is used in various fields. For example, in a game machine, in order to realize virtual reality (VR), an image is displayed by using a helmet with a head-mounted display device. At this time, it is necessary to change the image in accordance with the current position and the current angle of the head mounted display-equipped helmet. Therefore, in order to measure the current position and current angle of the helmet with a head-mounted display device, an HMT device is used.

An example of such an HMT device is a magnetic HMT device (see, for example, Patent Document 1). FIG. 5 shows a system configuration example using a magnetic HMT apparatus.
The HMT device 61 includes a sensor 63 attached to a head mounted helmet 60 with a display device mounted on the head of the pilot P, a control unit 64 configured by a computer, and a source 62.
The cockpit interior 30 is a cockpit of a flying body on which the pilot P is boarded, and includes a seat 30a on which the pilot P is seated, a control device group 30b, a window 30c, and a source 62.
The source 62 generates an alternating magnetic field, and a magnetic field unique to each position is generated at each point in the space of the cockpit 30.
The helmet 60 with a head-mounted display device includes a helmet (mounted body) mounted on the head of the pilot P, a display, and image display light emitted from the display, thereby reflecting the eyes of the pilot P. It has a guiding visor and a sensor 63 for detecting magnetic data. The pilot P wearing the head-mounted display-equipped helmet 60 can visually recognize the display image by the display and the front outside of the visor.

  The control unit 64 controls the source 62 and the sensor 63, and has a signal processing unit 66 having a function of calculating the current position and current angle of the sensor 63 based on the magnetic field data detected by the sensor 63, and a memory 65. Prepare. Also, by setting the coordinate system (XYZ coordinate system) of the cockpit interior 30 centered on the position of the source 62 and obtaining the current position and current angle of the sensor 63 based on the magnetic field data detected by the sensor 63, The current position (X, Y, Z) and current angle (Θ, Φ, Ψ) of the pilot P head are calculated.

In addition, as an HMT device different from the magnetic HMT device, there is an optical HMT device (see, for example, Patent Document 2). FIG. 6 is a system configuration example using an optical HMT apparatus. In addition, the same code | symbol is attached | subjected about the thing similar to the HMT apparatus 61. FIG.
The HMT device 51 includes light emitting diodes (LEDs) 57a to 57c attached to the helmet 50 with a head mounted display device attached to the head of the pilot P, a control unit 54 configured by a computer, and a camera device 52. Composed.
The cockpit interior 30 is a cockpit of a flying body on which the pilot P is boarded, and includes a seat 30a on which the pilot P is seated, a control device group 30b, a window 30c, and a camera device 52 that acquires image data. Prepare.
The camera device 52 includes a CCD camera or a CMOS camera, and includes a first camera 52a and a second camera (not shown). The first camera 2a and the second camera are fixed in the driver's cab 30 so as to have different shooting directions and at a certain distance allowing stereoscopic viewing. The shape of the imaging region γ of the first camera 52a is a cone having the camera 52a as a vertex as shown in FIG. The shape of the shooting area γ of the second camera is the same as the shape of the shooting area γ of the first camera 52a.
The helmet 50 with a head-mounted display device includes a helmet (mounted body) worn on the pilot P's head, a display, and the image display light emitted from the display to reflect the eyes of the pilot P. It has a visor for guiding and three LEDs 57a to 57c attached to the outer peripheral surface of the helmet.

  The control unit 54 controls the camera device 52 and the LEDs 57a to 57c, and also has a signal processing unit 56 having a function of calculating the current positions of the LEDs 57a to 57c based on image data detected by the camera device 52, a memory 55, Is provided. In addition, a coordinate system (XYZ coordinate system) is set in advance in the cockpit interior 30, and by obtaining the current positions of the LEDs 57 a to 57 c with respect to the camera device 52 based on image data detected by the camera device 52, The current position (X, Y, Z) and current angle (Θ, Φ, Ψ) of the pilot P head with respect to the coordinate system (XYZ coordinate system) are calculated.

JP 2002-081904 A Japanese Patent Application No. 2007-155370

  By the way, in the HMT device 61, since there is a conductive material such as a metal around the source 62 and the sensor 63 in the cockpit 30, the magnetostriction in the magnetic field is caused by the influence of the eddy current generated in the conductive material by the magnetic field. Occurs. Therefore, it is necessary to acquire magnetic field data in the cockpit 30 before using it for actual measurement and use it for correction, which is very laborious and time consuming. That is, there is a problem that the flying object is restrained for a long time in order to acquire magnetic field data (correction data).

On the other hand, in order to perform the measurement with the HMT device 51, since the principle of triangulation is used, an imaging region where the imaging region γ captured by the first camera 52a and the imaging region γ captured by the second camera overlap. All three LEDs 57a-57c had to be present.
However, in the cockpit interior 30, there is a limit to the position where the first camera 52a and the second camera can be installed. As a result, the shooting area γ shot by the first camera 52a and the second camera are shot. The imaging area where the imaging area γ overlaps cannot be set in a wide range and is limited.

Therefore, all of the three LEDs 57a to 57c may not exist in a shooting area where the shooting area γ shot by the first camera 52a overlaps the shooting area γ shot by the second camera. That is, all of the three LEDs 57a to 57c may not be photographed in the image data photographed by the first camera 52a or the image data photographed by the second camera. The current position (X, Y, Z) and the current angle (Θ, Φ, Ψ) may not be calculated.
Accordingly, the present invention provides a head motion that can calculate the current position and current angle of the passenger's head relative to the interior of the cockpit over a wide range without restraining the flying object for a long time to obtain correction data. An object is to provide a tracker device.

  The head motion tracker device of the present invention, which has been made to solve the above-described problems, includes a mounting body mounted on the head of a passenger, a camera device that acquires image data, and the passenger based on the image data. A head motion tracker device comprising: a control unit that calculates head information that is a current position and / or current angle of a passenger's head with respect to the inside of the cockpit where the passenger is boarding, A storage unit for storing; the camera device is attached to the mounting body and acquires image data by photographing the inside of the cockpit; the control unit is based on the image data and ambient environment information Thus, the head information is calculated.

Here, the “ambient environment information” refers to data such as size / shape / color information, mounting position information, and the like of a control device group and windows in the cockpit. Such ambient environment information is stored in the storage unit using information on various dimensions, colors, and positions of the control device group, CAD data, and the like before measurement.
According to the HMT device of the present invention, it is only necessary to store the ambient environment information inside the cockpit in the storage unit before performing the measurement, and it is not necessary to restrain the flying object or the like for a long time.
On the other hand, the camera device is attached to a mounting body and acquires image data when performing measurement. At this time, the camera device captures ambient environment information somewhere inside the cockpit.
Thereby, a control part calculates head information by comparing the image data acquired with the camera apparatus, and the surrounding environment information memorize | stored in the memory | storage part.

  As described above, according to the HMT device of the present invention, the current position of the passenger's head relative to the inside of the cockpit is The current angle can be calculated over a wide range.

(Means and effects for solving other problems)
In the above invention, the camera device may be three or more cameras.

Furthermore, in the above invention, the ambient environment information may be a plurality of optical markers positioned and attached to the inside of the cockpit.
Here, examples of the “optical marker” include a light emitter (lamp, LED, etc.), a reflector, and a phosphor.
According to the HMT apparatus of the present invention, a plurality of optical markers are positioned and attached to the inside of the cockpit before measurement. At this time, since the optical marker is small and easy to attach, the position where it can be installed is not limited, and can be attached anywhere in the cab. And a memory | storage part memorize | stores the surrounding environment information which shows the position of the inside of the cab where the some optical marker was attached.
Thus, when performing the measurement, the control unit calculates head information by comparing the image data created by the camera device with the ambient environment information stored in the storage unit.

It is a schematic block diagram which shows the HMT apparatus which is one Embodiment of this invention. It is a side view of the helmet with a head mounted display device shown in FIG. It is a top view of the helmet with a head-mounted display device shown in FIG. It is a schematic block diagram which shows the HMT apparatus which is other one Embodiment of this invention. It is a system configuration example using a magnetic HMT device. It is a system configuration example using an optical HMT device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments described below, and it goes without saying that various aspects are included without departing from the spirit of the present invention.

FIG. 1 is a schematic configuration diagram showing an HMT apparatus according to an embodiment of the present invention. 2 is a side view of the helmet with a head-mounted display device shown in FIG. 1, and FIG. 3 is a plan view of the helmet with a head-mounted display device shown in FIG.
In the present embodiment, the current position and the current angle of the head of the pilot P with respect to the cockpit 30 are calculated. That is, the HMT device 1 has a helmet coordinate system (X′Y ′) set in the helmet 10 with a head mounted display device worn by the pilot P with respect to the coordinate system (XYZ coordinate system) set in the cockpit 30. The head information including the current position (X, Y, Z) and the current angle (Θ, Φ, Ψ) of the Z ′ coordinate system is calculated.
Although details will be described later, the coordinate system (XYZ coordinate system) and the helmet coordinate system (X′Y′Z ′ coordinate system) are stored in the memory (storage unit) 40 in advance.

The HMT device 1 includes a camera device 2 attached to a head mounted display-equipped helmet 10 attached to the head of a pilot 3 and a control unit 20 configured by a computer.
The cockpit interior 30 is a cockpit of a flying body on which the pilot 3 is boarded, and includes a seat 30a for the pilot 3 to sit on, a control device group 30b, a window 30c, and the like.
A coordinate system (XYZ coordinate system) is preset in the cab 30 and stored in the memory 40 in advance. The coordinate system (XYZ coordinate system) can arbitrarily determine the origin and the direction of each coordinate axis. In this embodiment, as shown in FIG. 1, the origin is the position of the seat 30a, and the forward direction is the X-axis direction. And stored in advance in the coordinate system storage area 42 of the memory 40 so as to define the vertical direction in the forward direction as the Y-axis direction and the vertical direction in the X-axis direction and the Y-axis direction as the Z-axis direction. .

  Furthermore, the position (surrounding environment information) of the control device group 30b and the window part 30c with respect to the coordinate system (XYZ coordinate system) is also stored in the coordinate system storage area 42 in advance. Here, the “ambient environment information” refers to data such as size / shape / color information and mounting position information of the control device group 30b and the window portion 30c in the cab 30. Such ambient environment information is stored in the coordinate system storage area 42 using information on various dimensions, colors, and positions of the control device group 30b, CAD data, and the like before measurement. That is, ambient environment information can be stored without restraining the flying object or the like for a long time.

The helmet 10 with a head mounted display device guides the pilot P's eyes by reflecting the helmet 6 mounted on the pilot P's head, the display 7, and the image display light emitted from the display 7. It has a visor 8 and a camera device 2 that acquires image data.
The camera device 2 is constituted by a CCD camera or a CMOS camera, and has a plurality of cameras 2a to 2g. As shown in FIGS. 2 and 3, the plurality of cameras 2 a to 2 g are attached over the entire outer peripheral surface of the helmet 6. Further, the plurality of cameras 2 a to 2 g are attached to the outer peripheral surface of the helmet 6 so that the shooting directions are different so that the image of the entire periphery can be acquired.
By the way, the shape of the imaging region of the cameras 2a to 2g is a cone having the cameras 2a to 2g as apexes as shown in FIG. At this time, the size, shape, and degree of distortion of the shooting area that can be shot by the lenses used in the cameras 2a to 2g are determined. Therefore, the direction that requires accuracy is reduced by using a small viewing angle, and the other direction is handled by using a wide-angle lens (e.g., the cameras 2a to 2c are low distortion lenses having a small viewing angle α, the cameras 2d and 2e are A wide-angle lens with a large viewing angle β). As will be described later, the image data acquisition / correction unit 28 corrects image data acquired by the cameras 2a to 2g.
Thereby, if the image data acquired with the some cameras 2a-2g are couple | bonded, the whole periphery of the cockpit interior 30 can be image | photographed.

Further, a helmet coordinate system (X′Y′Z ′ coordinate system) is set in advance in the helmet 10 with a head-mounted display device, and is stored in the memory 40 in advance. The helmet coordinate system (X'Y'Z 'coordinate system) can arbitrarily determine the origin and the direction of each coordinate axis. In this embodiment, as shown in FIG. 2, the origin is the position of the camera 2a, and the camera The direction from the position 2a to the position of the camera 2b is the X′-axis direction, the direction perpendicular to the X′-axis direction is the Y′-axis direction, and the direction perpendicular to the X′-axis direction and the Y′-axis direction is the Z′-axis direction. It is stored in advance in the helmet coordinate system storage area 41 of the memory 40 so as to be defined.
Further, the positions and photographing directions of the plurality of cameras 2 a to 2 g with respect to the helmet coordinate system (X′Y′Z ′ coordinate system) are also stored in the helmet coordinate system storage area 41 in advance.

The control unit 20 includes a computer including a signal processing unit 21, a memory 40, and the like, and performs various controls and arithmetic processes. The processing executed by the signal processing unit 21 will be described separately for each functional block. An image data acquisition / correction unit 28 that acquires image data and corrects distortion, and an all-around image that creates all-around image data from these images A data generation unit 27, a data comparison unit 26, and a head information calculation unit 25 that calculates head information are included.
In addition, the memory 40 has an area for storing various data necessary for the control unit 20 to execute processing, and stores a helmet coordinate system (X′Y′Z ′ coordinate system) and the like. It has a coordinate system storage area 41, a coordinate system storage area 42 for storing a coordinate system (XYZ coordinate system) and the like, and an image data storage area 43 for storing image data.

The image data acquisition / correction unit 28 acquires the image data created by the camera device 2 and corrects the optical distortion of the image data.
The omnidirectional image data generation unit 27 creates omnidirectional image data based on the positions and photographing directions of the cameras 2a to 2g stored in the helmet coordinate system storage area 41 and the image data corrected for optical distortion. Then, control for storing the image data in the image data storage area 43 is performed.

The data comparison unit 26 determines the helmet coordinates relative to the coordinate system (XYZ coordinate system) based on the all-around image data stored in the image data storage area 43 and the ambient environment information stored in the coordinate system storage area 42. Control is performed to calculate the current position (Xh, Yh, Zh) and current angle (Θh, Φh, Ψh) of the system (X′Y′Z ′ coordinate system).
The head information calculation unit 25 sets the current position (Xh, Yh, Zh) and current angle (Θh, Φh, Ψh) of the helmet coordinate system (X′Y′Z ′ coordinate system) relative to the coordinate system (XYZ coordinate system). Based on this, control is performed to calculate head information including the current position (X, Y, Z) and current angle (Θ, Φ, Ψ) of the pilot P head relative to the cockpit 30.
Then, the display control unit 24 displays the indicator based on the head information including the current position (X, Y, Z) and the current angle (Θ, Φ, Ψ) of the pilot P head received from the control unit 20. 7 generates and controls a video signal emitted from. Thereby, the pilot P visually recognizes the display image by the display device 7.

  As described above, according to the HMT device 1, the current position of the head of the pilot P with respect to the inside of the cockpit 30 (X , Y, Z) and the current angle (Θ, Φ, ψ) can be calculated over a wide range.

(Other embodiments)
(1) The above-described HMT device 1 is configured to use data such as size / shape / color information and mounting position information of the control device group 30b and the window 30c in the cockpit 30 as ambient environment information. A configuration may be adopted in which a plurality of LEDs or markers 70 are attached to the cockpit 30 (see FIG. 4).
(2) In the HMT device 1 described above, the camera lenses are of different types, but they may all be composed of the same type of camera lens.

  The HMT device of the present invention is used for measuring the current position and the current angle of a head-mounted display-equipped helmet used in, for example, game machines and vehicles.

1 Head motion tracker device 2 Camera device 6 Helmet (wearing body)
DESCRIPTION OF SYMBOLS 10 Helmet with head mounted display device 20 Control unit 30 Inside cockpit 40 Memory (storage unit)
P Pilot (passenger)

Claims (3)

A body to be worn on the head of the passenger;
A camera device for acquiring image data;
A head motion tracker device comprising, based on the image data, a control unit that calculates head information that is a current position and / or a current angle of a passenger's head with respect to a cockpit where the passenger is boarded;
A storage unit for storing information on the surrounding environment of the cockpit;
The camera device is attached to the mounting body, acquires image data of the cockpit,
The head motion tracker device, wherein the control unit calculates the head information based on the image data and ambient environment information.   The head motion tracker device according to claim 1, wherein the camera device is three or more cameras. The head motion tracker device according to claim 1, wherein the ambient environment information is a plurality of optical markers positioned and attached to the cockpit.

Images