JP2022017894A - Head motion tracker device and head motion tracker device for aircraft - Google Patents

Abstract

To provide a head motion tracker device easy to apply to even a conventional travel body, and a head motion tracker device for an aircraft.SOLUTION: A head motion tracker device 10 comprises: an image acquisition part 2 which is mounted to a head mounting part 1 and acquires an image of a direction along the orientation of the head of an observer 11; and a control part 4 which on the basis of the image acquired by the image acquisition part 2, performs control to acquire the position and angle of the head of the observer 11.SELECTED DRAWING: Figure 9

Description

The present invention relates to a head motion tracker device and an aircraft head motion tracker device, and more particularly to a head motion tracker device having a head mounting portion mounted on the observer's head and an aircraft head motion tracker device.

Conventionally, a head motion tracker device and an aircraft head motion tracker device having a head-mounted portion mounted on the observer's head are known (see, for example, Patent Document 1).

The above-mentioned Patent Document 1 discloses a head motion tracker device including a helmet (head mounting portion) mounted on the head of a pilot (observer). This head motion tracker device includes the helmet, a camera device installed in a boarding body (airframe body), and a control unit.

The helmet of Patent Document 1 is provided with a plurality of head-mounted body light emitting portions that emit infrared light having different wavelengths from each other. The control unit is configured to perform control to acquire the position and angle of the pilot's head based on the position information of each of the plurality of head-mounted body light emitting units captured by the camera device. In the head motion tracker device, the camera device is arranged at a position where a plurality of head-mounted body light emitting units can be reliably photographed in order to acquire the position and angle of the pilot's head by the control unit.

Japanese Unexamined Patent Publication No. 2009-109319

However, in the head motion tracker device of Patent Document 1, since it is necessary to install the camera device in the aircraft body at a position where a plurality of head-mounted body light emitting portions can be reliably photographed, the camera device is mounted on the aircraft body. It is necessary to secure the installation space for the (image acquisition unit). Therefore, in the head motion tracker device of Patent Document 1, it may be difficult to secure the installation space in the existing boarding body. Therefore, the head motion tracker device is an existing boarding body (moving body). Is also desired to be easily applicable.

The present invention has been made to solve the above-mentioned problems, and one object of the present invention is a head motion tracker device and an aircraft that can be easily applied even to an existing mobile body. It is to provide a head motion tracker device for.

In order to achieve the above object, the head motion tracker device according to the first aspect of the present invention is attached to a head-mounted portion mounted on the observer's head and a head-mounted portion, and is attached to the observer's head. An image acquisition unit that acquires an image in a direction along the direction of the image, and a control unit configured to control the position and angle of the observer's head based on the image acquired by the image acquisition unit. And.

In the head motion tracker device according to the first aspect of the present invention, the image acquisition unit attached to the head mounting unit is provided as described above. Further, the head motion tracker device is provided with a control unit configured to control the position and angle of the observer's head based on the image acquired by the image acquisition unit. Thereby, the position and angle of the observer's head can be acquired by the control unit based on the image of the image acquisition unit attached to the head mounting unit instead of the moving body. That is, the position and angle of the observer's head can be acquired by the control unit without installing the image acquisition unit on the moving body. As a result, it is not necessary to secure an installation space for the image acquisition unit in the moving body, so that the head motion tracker device can be easily applied even to an existing moving body.

In order to achieve the above object, the aircraft head motion tracker device according to the second aspect of the present invention includes a head motion tracker device mounted on a pilot as an observer inside the aircraft, and the head motion tracker device is a head motion tracker device. , The head-mounted part attached to the pilot's head, the image acquisition part attached to the head-mounted part and acquiring an image in the direction along the direction of the pilot's head, and the image acquired by the image acquisition part. Includes a control unit configured to control the acquisition of the position and angle of the pilot's head based on.

The aircraft head motion tracker device according to the second aspect of the present invention is provided with an image acquisition unit attached to the head mounting portion as described above. Further, the head motion tracker device is provided with a control unit configured to control the position and angle of the observer's head based on the image acquired by the image acquisition unit. As a result, the position and angle of the observer's head can be acquired by the control unit based on the image of the image acquisition unit attached to the head mounting unit instead of the moving body. That is, the position and angle of the observer's head can be acquired by the control unit without installing the image acquisition unit on the moving body. As a result, since it is not necessary to secure an installation space for the image acquisition unit on the moving body, it is possible to obtain an aircraft head motion tracker device to which the head motion tracker device can be easily applied even to an existing moving body. Can be done.

The head motion tracker device in the first aspect is attached to the head-mounted portion mounted on the observer's head and the head-mounted portion, and acquires an image in a direction along the direction of the observer's head. An image acquisition unit is provided, and a control unit configured to control the acquisition of the position and angle of the observer's head based on the image acquired by the image acquisition unit. Thereby, the position and angle of the observer's head can be acquired by the control unit based on the image of the image acquisition unit attached to the head mounting unit instead of the moving body. That is, the position and angle of the observer's head can be acquired by the control unit without installing the image acquisition unit on the moving body. As a result, it is not necessary to secure an installation space for the image acquisition unit in the moving body, so that the head motion tracker device can be easily applied even to an existing moving body.

The aircraft head motion tracker device in the second aspect is provided with a head motion tracker device mounted on the pilot as an observer inside the aircraft, and the head motion tracker device is a head mounted on the pilot's head. The position of the pilot's head based on the mounting part, the image acquisition part that is attached to the head mounting part and acquires an image in the direction along the direction of the pilot's head, and the image acquired by the image acquisition part. And a control unit configured to control the acquisition of the angle. As a result, the position and angle of the observer's head can be acquired by the control unit based on the image of the image acquisition unit attached to the head mounting unit instead of the moving body. That is, the position and angle of the observer's head can be acquired by the control unit without installing the image acquisition unit on the moving body. As a result, since it is not necessary to secure an installation space for the image acquisition unit on the moving body, it is possible to obtain an aircraft head motion tracker device to which the head motion tracker device can be easily applied even to an existing moving body. Can be done.

It is a schematic diagram which showed the state which the pilot in the aircraft wears the head motion tracker device by 1st Embodiment. It is a block diagram which showed the structure of the head motion tracker apparatus by 1st Embodiment. It is a schematic diagram which showed the state which photographed the front of the pilot by the image acquisition part of the head motion tracker apparatus by 1st Embodiment, and the image before movement (base image). It is a schematic diagram which showed the front frame map (base map) created by the image acquisition part of the head motion tracker apparatus by 1st Embodiment. It is a schematic diagram which showed the state which imaged the lower side than the front of the pilot by the image acquisition part of the head motion tracker apparatus by 1st Embodiment, the image before movement and the image after movement. It is a schematic diagram which showed the current frame map created by the image acquisition part of the head motion tracker apparatus by 1st Embodiment. It is a schematic diagram which showed the post-movement environment map created in the head motion tracker apparatus by 1st Embodiment. It is a schematic diagram which showed an example of the movement of the head of the pilot which attached the head motion tracker device by 1st Embodiment. It is a schematic diagram which showed the method of estimating the position and the angle of the head of the pilot in the control part of the head motion tracker apparatus by 1st Embodiment. It is a flowchart which showed the position and angle acquisition process of the head of a pilot by 1st Embodiment. It is a schematic diagram which showed the state which the pilot in the aircraft wears the head motion tracker device by 2nd Embodiment. It is a block diagram which showed the structure of the mark part of the head motion tracker apparatus by 2nd Embodiment. It is a block diagram which showed the structure of the head motion tracker apparatus by 2nd Embodiment. It is a schematic diagram which showed the state which the control part side communication part of the head motion tracker apparatus by 2nd Embodiment, and the mark part side communication part communicate with each other. It is a schematic diagram which showed the example which reduced the position and the angle of the head of the pilot to be created in the estimation of the position and the angle of the head of a pilot in the control part of the head motion tracker apparatus by 2nd Embodiment. It is a block diagram which showed the structure of the head motion tracker apparatus by 3rd Embodiment. It is a schematic diagram which showed the head motion tracker device and the mark part by 3rd Embodiment. It is a schematic diagram which showed the head motion tracker apparatus by 1st modification of 2nd Embodiment. It is a block diagram which showed the structure of the head motion tracker apparatus by the 2nd modification of 3rd Embodiment.

Hereinafter, embodiments embodying the present invention will be described with reference to the drawings.

[First Embodiment]
The configuration of the aircraft head motion tracker device 10 (hereinafter, simply referred to as the head motion tracker device 10) according to the first embodiment will be described with reference to FIGS. 1 to 9. As shown in FIG. 1, the head motion tracker device 10 is provided on a head-mounted display mounted on the pilot 11 in the aircraft 100. Specifically, the head motion tracker device 10 has a function of measuring the position and angle of the head of the pilot 11 in the head-mounted display. The aircraft head motion tracker device 10 is an example of a "head motion tracker device" within the scope of the claims. Further, the aircraft 100 is an example of a "moving body" within the scope of claims. Pilot 11 is an example of an "observer" in the claims.

Specifically, as shown in FIGS. 1 and 2, the head motion tracker device 10 includes a head mounting unit 1, an image acquisition unit 2, a marker unit 3, and a control unit 4.

The head mounting portion 1 is a member mounted on the head of the pilot 11, such as a helmet, a headset, and a belt. The image acquisition unit 2 is attached to the head mounting unit 1 and is configured to acquire an image in a direction along the direction of the head of the pilot 11 (hereinafter referred to as the A direction). The image acquisition unit 2 includes a plurality (two) cameras 21 arranged side by side. Specifically, the two cameras 21 are area sensor cameras. The two cameras 21 are sensitive to ultraviolet light and near infrared light. The two cameras 21 are arranged at a predetermined distance from each other. Each of the two cameras 21 is configured to shoot in the same A direction. The same lens is attached to the two cameras 21. Specifically, the two cameras 21 are equipped with the same wide-angle lens or fisheye lens.

The mark portion 3 is arranged around the pilot 11 and is acquired as an image by the image acquisition portion 2. That is, the mark portion 3 has a function as a marker. A plurality (three) of the mark portions 3 are arranged on the aircraft 100. The mark portion 3 is arranged in the cockpit of the aircraft 100. In addition, the mark portion 3 may be arranged in 1, 2 or 4 or more.

The mark portion 3 includes a mounting portion 31 and a light emitting portion 32. The attachment portion 31 is a base for attaching the mark portion 3 to the moving body. The light emitting portion 32 is mounted on the surface of the mounting portion 31 on the pilot 11 side. The light emitting unit 32 is composed of an LED (Light Emitting Diode) capable of irradiating ultraviolet light or near-infrared light, a VCSEL (Vertical Cavity Surface Emitting Laser), and the like. Further, the light emitting unit 32 is attached with a lens for irradiating light at a wide angle.

The mark portion 3 is configured to be supplied with electric power by non-contact power supply, solar power generation, or the like. As described above, the wiring connection for supplying electric power to the mark portion 3 is unnecessary.

Since it is necessary to accurately measure the position and angle of the head of the pilot 11, a plurality (two) of the mark portions 3 are arranged on the front side of the pilot 11. As a result, when the pilot 11 faces the front, the image acquisition unit 2 acquires a plurality (two) of the mark portions 3.

The control unit 4 is composed of a computer that performs various controls and arithmetic processing. The control unit 4 includes an image processing unit 41, a CPU (Central Processing Unit) unit 42, and a storage unit 43.

The image processing unit 41 extracts feature points 12 in the image based on the image acquired by the image acquisition unit 2, and creates an ambient environment map 5 as a three-dimensional map based on the image by machine learning. Has a function to perform. The ambient environment map 5 created by the image processing unit 41 is stored in the storage unit 43. Here, the feature point 12 in the image is the shape of the skeleton in the aircraft 100, the center of gravity of a part such as a switch and a display arranged in the aircraft 100, which is acquired from the difference in contrast in the acquired image. Show a point. Further, as the feature point 12 of the image, the mark portion 3 is also acquired as the feature point 12.

In the ambient environment map 5, in the relative coordinate system (XYZ coordinate system), the axis extending in the direction in which the two cameras 21 are lined up is the X axis (see FIG. 4), and the axis extending in the optical axis direction of the two cameras 21 is the Y axis. (See FIG. 4), and the axis extending in the direction orthogonal to the X-axis and the Y-axis is the Z-axis (see FIG. 4). The origin of the relative coordinate system is the midpoint of the two cameras 21.

Here, since the two cameras 21 are area sensor cameras, the image processing unit 41 can acquire information on the XZ plane in the surrounding environment map 5. Further, the image acquisition unit 2 can acquire information in the Y-axis direction in the surrounding environment map 5 by a stereo method based on the parallax of the two cameras 21.

The surrounding environment map 5 has a base map 51 (see FIG. 4), a current frame map 52 (see FIG. 6), and a front frame map 53 (see FIG. 4). The base map 51 and the front frame map 53 are examples of the "second surrounding environment map" in the claims. The current frame map 52 is an example of the "third ambient environment map" in the claims.

The base map 51 is a reference map in the surrounding environment map 5. The base map 51 may be, for example, an ambient environment map 5 based on the base image 61 taken when the pilot 11 faces directly in front of the A direction. The base map 51 is registered in the storage unit 43 in advance.

The front frame map 53 is a current frame map 52 stored in the storage unit 43 before the present time. The front frame map 53 may be, for example, a three-dimensional map based on the pre-movement image 62 taken when the pilot 11 faces directly in front of the A direction. The pre-movement image 62 is an example of the “second image” in the claims.

The current frame map 52 is, for example, an image obtained by synthesizing a pre-movement image 62 taken when the pilot 11 faces directly in front and a post-movement image 63 taken when the head of the pilot 11 moves from the front. It may be a three-dimensional map based on. The current frame map 52 is added according to the image acquired by the image acquisition unit 2. The moved image 63 is an example of the "first image" in the claims.

The CPU unit 42 has a function of executing the comparison process 42a and the position angle detection process 42b. The comparison process 42a is a process of comparing the surrounding environment map 5 converted by the image processing unit 41. The position / angle detection process 42b is a process of detecting the current position and angle of the head of the pilot 11 based on the surrounding environment map 5 created by the image processing unit 41. The position / angle detection process 42b will be described later with reference to an example.

The storage unit 43 is a storage medium having a RAM (Random Access Memory) and a ROM (Read Only Memory). Here, the previous frame map 53 is stored in the RAM. Further, the base map 51 is stored in the ROM.

(Position and angle detection processing)
The control unit 4 of the first embodiment is configured to control the acquisition of the position and angle of the head of the pilot 11 based on the image acquired by the image acquisition unit 2. The position and angle of the head of the pilot 11 are shown as the position and angle of the midpoint of the two cameras 21.

Specifically, the control unit 4 acquires the position and angle of the head of the pilot 11 based on the portion of the image acquired by the image acquisition unit 2 other than the window in which the outside world can be visually recognized in the aircraft 100. It is configured to provide control. That is, the control unit 4 has the head of the pilot 11 based on the shape of the skeleton in the aircraft 100 and the feature points 12 such as the switch and the display arranged in the aircraft 100 in the image acquired by the image acquisition unit 2. It is configured to control the acquisition of the position and angle of the unit.

The position angle detection process 42b to which the particle filter is applied will be described as an example with reference to FIGS. 3 to 9. In the following description, an example in which the base map 51 and the front frame map 53 are the same is shown, but the base map 51 and the front frame map 53 may be different. When the base map 51 and the front frame map 53 are different, the position angle detection process 42b may be performed using the front frame map 53.

As shown in FIGS. 3 and 4, the control unit 4 uses machine learning to perform machine learning on a three-dimensional ambient environment map 5 around the pilot 11 based on an image when the pilot 11 faces directly in front (image 62 before movement). It is configured to control the creation of (previous frame map 53). That is, the control unit 4 acquires the distance in the direction along the direction of the head of the pilot 11 based on the parallax of the two cameras 21, and based on the acquired distance, the three-dimensional ambient environment map 5 ( It is configured to control the creation of the front frame map 53). At this time, the control unit 4 is configured to control acquisition of a plurality of feature points 12 based on the difference in contrast in the pre-movement image 62. The control unit 4 is configured to control acquisition of a plurality of marker units 3 based on the difference in contrast in the pre-movement image 62.

Here, the positions and angles of the head of the pilot 11 are indicated as (X1, Y1, Z1) and (RL1, EL1, AZ1), respectively. RL1 is an angle in the roll direction (rotation with respect to the X axis). EL1 is an angle in the elevation direction (rotation with respect to the Y axis). AZ1 is an angle in the azimuth direction (rotation with respect to the Z axis).

Next, FIGS. 5 and 6 show a case where the head of the pilot 11 is moved downward from the direction directly in front of the pilot 11. In this case, the control unit 4 uses machine learning to create a three-dimensional ambient environment around the pilot 11 based on the pre-movement image 62 and the image in the A direction after the head of the pilot 11 has moved (post-movement image 63). It is configured to control the creation of the map 5 (current frame map 52). That is, the control unit 4 acquires the distance in the direction along the direction of the head of the pilot 11 based on the parallax of the two cameras 21, and based on the acquired distance, the three-dimensional ambient environment map 5 ( It is configured to control the creation of the current frame map 52). At this time, the control unit 4 is configured to control acquisition of a plurality of feature points 12 based on the difference in contrast in the image 63 after movement. The control unit 4 is configured to control acquisition of a plurality of marker units 3 based on the difference in contrast in the image 63 after movement.

The current frame map 52 is created based on an image in which the same portion (common portion) of the pre-movement image 62 and the post-movement image 63 is combined, and a portion of the post-movement image 63 different from the pre-movement image 62 is added. It is a series of three-dimensional maps. Here, the current frame map 52 becomes the front frame map 53 the next time the post-movement image 63 is acquired. Then, the post-movement image 63 is similarly combined and added to the front frame map 53. In this way, the current frame map 52 is used to create the surrounding environment map 5 after the current frame.

Here, the positions and angles of the head of the pilot 11 after being moved downward from the direction directly in front are shown as (X2, Y2, Z2) and (RL2, EL2, AZ2), respectively.

Next, as shown in FIG. 7, the control unit 4 is configured to control to acquire the post-movement environment map 54 based on the comparison between the front frame map 53 and the current frame map 52. That is, the control unit 4 is configured to control the acquisition of the portion corresponding to the moved image 63 from the current frame map 52. The post-movement environment map 54 is an example of the "first surrounding environment map" in the claims.

As described above, the post-movement environment map 54 is a map composed of the post-movement image 63 acquired by the image acquisition unit 2 in the current frame (current time). The current frame map 52 is a map based on an image obtained by synthesizing the pre-movement image 62 and the post-movement image 63. The front frame map 53 is a map composed of the pre-movement image 62 of the frame (previous time point) before the post-movement image 63.

Then, as shown in FIGS. 7 and 8, the control unit 4 is the head of the pilot 11 capable of acquiring all of the plurality of feature points 12 and the mark portions 3 as the feature points 12 included in the post-movement environment map 54. It is configured to control the position and angle to be estimated as the current position and angle of the head of the pilot 11.

In detail, as shown in FIG. 9, the control unit 4 is configured to perform control to estimate the position and angle of the head of the pilot 11 of the current frame based on the particle filter.

That is, the control unit 4 creates the positions and angles of the heads of the plurality of pilots 11 having different head positions and angles from each other based on the positions and angles of the heads of the pilots 11 in the frames before the current frame. It is configured to control the operation. The control unit 4 is configured to randomly move the positions and angles of the heads of the plurality of pilots 11 having different head positions and angles from each other.

The control unit 4 can acquire a map correlated with the post-movement environment map 54 from the current frame map 52 from the positions and angles of the heads of the plurality of pilots 11 randomly moved. It is configured to control the selection of the position and angle of. Specifically, the control unit 4 includes all of the plurality of feature points 12 and the marker portions 3 as the feature points 12 included in the post-movement environment map 54 from the positions and angles of the heads of the plurality of pilots 11 created. It is configured to control the selection of the position and angle of the head of the pilot 11 that can be acquired.

Then, the control unit 4 of the plurality of pilots 11 having different head positions and angles from each other based on the positions and angles of the heads of the pilots 11 (marked with a circle in FIG. 9) capable of acquiring a correlated map. It is configured to control the position and angle of the head. At this time, the control unit 4 is configured to control to delete the position and angle (marked with x in FIG. 9) of the head of the pilot 11 having no correlation.

As described above, the control unit 4 controls to create a three-dimensional front frame map 53 around the pilot 11 based on the pre-movement image 62 in the A direction (direction along the direction of the head of the pilot 11). Is configured to do. The control unit 4 controls to create a three-dimensional current frame map 52 around the pilot 11 based on the pre-movement image 62 and the post-movement image 63 in the A direction (direction along the direction of the head of the pilot 11). Is configured to do. The control unit 4 is configured to control the acquisition of the position and angle of the head of the pilot 11 based on the front frame map 53 and the current frame map 52.

Here, the control unit 4 acquires the post-movement environment map 54 based on the comparison between the front frame map 53 and the current frame map 52, and controls to estimate the position and angle of the head of the pilot 11. It is configured as follows. That is, the control unit 4 is configured to control the acquisition of the post-movement environment map 54 from the current frame map 52.

The control unit 4 acquires the position and angle of the head of the pilot 11 based on the mark portion 3 in the three-dimensional post-movement environment map 54 based on the acquired post-movement image 63, and a plurality of control units 4. Control to acquire the position and angle of the head of the pilot 11 based on the plurality of feature points 12 in the three-dimensional post-movement environment map 54 based on the acquired post-movement image 63. It is configured. That is, the control unit 4 is configured to perform control to estimate the position and angle of the head of the pilot 11 based on the plurality of feature points 12 and the mark unit 3 included in the post-movement environment map 54.

(Head position and angle acquisition process)
Hereinafter, the position and angle acquisition process of the head will be described with reference to FIGS. 3 to 10. The head position and angle acquisition process is a process of acquiring the position and angle of the head of the pilot 11 by using the camera 21 mounted on the head of the pilot 11.

In step S1, the control unit 4 acquires the base map 51 as the front frame map 53 from the storage unit 43 (see FIGS. 3 and 4).

In step S2, the control unit 4 acquires the current frame map 52 based on the image in the direction in which the pilot 11 is facing (see FIGS. 5 and 6). In step S3, the control unit 4 acquires the post-movement environment map 54 based on the comparison between the front frame map 53 and the current frame map 52 (see FIG. 7). In step S4, the control unit 4 estimates the position and angle of the observer's head capable of acquiring all of the plurality of feature points 12 and the mark portions 3 included in the post-movement environment map 54 in the current frame map 52. (See FIG. 9).

In step S5, in the control unit 4, the current frame map 52 is stored in the storage unit 43 as the front frame map 53. After step S5, the process returns to step S3 and is repeated.

(Effect of the first embodiment)
In the first embodiment, the following effects can be obtained.

In the first embodiment, as described above, the head motion tracker device 10 is provided with the image acquisition unit 2 attached to the head mounting unit 1. Further, the head motion tracker device 10 is provided with a control unit 4 configured to control the acquisition of the position and angle of the head of the pilot 11 based on the image acquired by the image acquisition unit 2. As a result, the position and angle of the head of the pilot 11 can be acquired by the control unit 4 based on the image of the image acquisition unit 2 attached to the head mounting unit 1 instead of the aircraft 100. That is, the position and angle of the head of the pilot 11 can be acquired by the control unit 4 without installing the image acquisition unit 2 on the aircraft 100. As a result, since it is not necessary to secure an installation space for the image acquisition unit 2 in the aircraft 100, the head motion tracker device 10 can be easily applied even to the existing aircraft 100.

Further, in the first embodiment, as described above, the control unit 4 creates a three-dimensional ambient environment map 5 around the pilot 11 based on the image in the direction along the direction of the head of the pilot 11. It is configured to control and acquire the position and angle of the head of the pilot 11 based on the surrounding environment map 5. As a result, the position and angle of the head of the pilot 11 can be acquired by the calculation of the control unit 4 with respect to the surrounding environment map 5 on the virtual space. The position and angle of the head of the pilot 11 can be reliably acquired also in the environment map 5.

Further, in the first embodiment, as described above, the surrounding environment map 5 is provided with the post-movement environment map 54 composed of the post-movement image 63 acquired by the image acquisition unit 2 in the current frame (current time). The control unit 4 is configured to perform control to estimate the position and angle of the head of the pilot 11 based on the plurality of feature points 12 included in the post-movement environment map 54. As a result, since the position and angle of the head of the pilot 11 are estimated with reference to the plurality of feature points 12, the position and angle of the head of the pilot 11 can be estimated more accurately.

Further, in the first embodiment, as described above, the surrounding environment map 5 is provided with the front frame map 53 composed of the pre-movement image 62 before the post-movement image 63. The control unit 4 is configured to perform control to estimate the position and angle of the head of the pilot 11 based on at least a plurality of feature points 12 and the front frame map 53 included in the post-movement environment map 54. As a result, the front frame map 53 composed of the pre-movement image 62 is also used as compared with the case where the position and angle of the head of the pilot 11 are estimated based only on the plurality of feature points 12 of the post-movement environment map 54. This makes it possible to more accurately grasp changes in the position and angle of the head of the pilot 11. As a result, the accuracy of estimating the position and angle of the head of the pilot 11 can be improved.

Further, in the first embodiment, as described above, the surrounding environment map 5 is provided with the current frame map 52 based on the image obtained by synthesizing the pre-movement image 62 and the post-movement image 63. The control unit 4 is configured to acquire the post-movement environment map 54 based on the comparison between the front frame map 53 and the current frame map 52, and to control to estimate the position and angle of the head of the pilot 11. do. As a result, the post-movement environment map is compared with the case where the post-movement environment map 54 is acquired based only on the post-movement image 63 by the current frame map 52 based on the image obtained by synthesizing the post-movement image 63 with the pre-movement image 62. The amount of information contained in 54 can be increased. As a result, since the more precise post-movement environment map 54 can be created, the accuracy of estimating the position and angle of the head of the pilot 11 can be further improved.

Further, in the first embodiment, as described above, the position and angle of the head of the pilot 11 capable of acquiring all the plurality of feature points 12 included in the post-movement environment map 54 by the control unit 4 are set to the current frame ( It is configured to control the estimation as the position and angle of the head of the pilot 11 (currently). As a result, the position and angle of the head of the pilot 11 in the current frame (current time) can be acquired depending on whether or not all of the plurality of feature points 12 included in the post-movement environment map 54 can be acquired, which is simple. The position and angle of the head of the pilot 11 can be estimated by the method.

Further, in the first embodiment, as described above, the control units 4 are headed to each other based on the position and angle of the head of the pilot 11 of the front frame (the time point before the current frame (current time)). After creating the positions and angles of the heads of the plurality of pilots 11 having different positions and angles and changing the positions and angles of the heads of the created plurality of pilots 11, a plurality of images included in the post-movement environment map 54. It is configured to control the position and angle of the head of the pilot 11 that can acquire all the feature points 12. As a result, by creating a plurality of positions and angles of the head of the pilot 11, the position and angle of the head of the pilot 11 in the current frame (current time) can be estimated from many variations, so that a more appropriate pilot can be used. The position and angle of the head of 11 can be selected.

Further, in the first embodiment, as described above, the control unit 4 is configured to control the acquisition of a plurality of feature points 12 based on the difference in contrast in the image 63 after movement. As a result, since a plurality of feature points 12 can be acquired by a simple method, it is possible to suppress an increase in the processing load on the control unit 4.

Further, in the first embodiment, as described above, the head motion tracker device 10 is provided with a marker portion 3 arranged around the pilot 11 and acquired as an image by the image acquisition unit 2. The control unit 4 is configured to control the acquisition of the position and angle of the head of the pilot 11 based on the marker unit 3 in the three-dimensional ambient environment map 5 based on the image acquired by the marker unit 3. .. As a result, the mark portion 3 can be used as the feature point 12 in the image acquired by the image acquisition unit 2, so that the image acquisition unit 2 can acquire the feature point 12 more reliably.

Further, in the first embodiment, as described above, the image acquisition unit 2 is provided with two cameras 21 arranged side by side. The control unit 4 acquires the distance in the direction along the direction of the head of the pilot 11 based on the parallax of the two cameras 21, and the three-dimensional ambient environment based on the distance acquired by the two cameras 21. It is configured to control the creation of the map 5. As a result, it is possible to accurately acquire the distance in the direction along the direction of the head of the pilot 11 as compared with the case of acquiring the distance in the direction along the direction of the head of the pilot 11 by one camera 21. Therefore, it is possible to create a three-dimensional ambient environment map 5 more accurately.

Further, in the first embodiment, as described above, the aircraft head motion tracker device is provided with the head motion tracker device 10 mounted on the pilot 11 as an observer inside the aircraft 100. The head motion tracker device 10 is provided with an image acquisition unit 2 attached to the head mounting unit 1. Further, the head motion tracker device 10 is provided with a control unit 4 configured to control the acquisition of the position and angle of the head of the pilot 11 based on the image acquired by the image acquisition unit 2. As a result, the position and angle of the head of the pilot 11 can be acquired by the control unit 4 based on the image of the image acquisition unit 2 attached to the head mounting unit 1 instead of the aircraft 100. That is, the position and angle of the head of the pilot 11 can be acquired by the control unit 4 without installing the image acquisition unit 2 on the aircraft 100. As a result, since it is not necessary to secure an installation space for the image acquisition unit 2 in the aircraft 100, the aircraft head motion tracker device 10 can be easily applied even to the existing aircraft 100. Can be obtained.

[Second Embodiment]
Next, the head motion tracker device 210 of the second embodiment will be described with reference to FIGS. 11 to 14. Specifically, unlike the head motion tracker device 10 of the first embodiment, which includes the mark portion 3 that only emits light, in the head motion tracker device 210 of the second embodiment, the mark portion 203 uses the mark portion side communication unit 234. I have. In the second embodiment, the same components as those in the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

As shown in FIGS. 11 and 12, the head motion tracker device 210 includes a head mounting unit 1, an image acquisition unit 2, a marker unit 203, and a control unit 204 (see FIG. 13).

The mark portion 203 is arranged around the pilot 11 and is acquired as an image by the image acquisition unit 2. The mark portion 203 has both a function as a marker and a communication function. A plurality (three) of the mark portions 203 are arranged on the moving body. In addition, the mark portion 203 may be arranged in 1, 2 or 4 or more.

Since the plurality of marking portions 203 can continuously acquire a good current frame map 52, at least one marking portion 203 is provided by the image acquisition unit 2 regardless of the direction in which the pilot 11 is oriented. It is located in a position where it can be photographed. As a result, the control unit 204 can avoid a decrease in the estimation accuracy of the position and angle of the head of the pilot 11. In addition, it is possible to reliably avoid a state in which the mark portion 203 is not detected by the control unit 204.

The mark unit 203 includes a mounting unit 31, a light emitting unit 203a, a light receiving unit 203b, and a control unit 203c.

The light emitting unit 203a is used for optical communication with the control unit 204 together with a marker function. The light emitting unit 203a is composed of an LED capable of irradiating ultraviolet light or near infrared light, a VCSEL, and the like. Further, the light emitting unit 203a is attached with a lens for irradiating light at a wide angle. The light receiving unit 203b is composed of a light receiving element such as a photodiode capable of receiving ultraviolet light or near infrared light. A lens for enabling light to be received at a wide angle is attached to the light receiving unit 203b.

The control unit 203c has a CPU unit 231, a storage unit 232, a sensor unit 233, and a mark unit side communication unit 234.

The CPU unit 231 has a function of causing the sensor data processing 231a to execute a predetermined process. The sensor data processing 231a performs a process of converting the measured value from the sensor unit 233 into the measured value information that can be transmitted to the control unit 204. The sensor data processing 231a performs a process of converting the data from the control unit 204 into data recognizable by the control unit 203c.

The storage unit 232 is a storage medium having a RAM and a ROM. The storage unit 232 stores the identification number 232a of the mark unit 203 and the aircraft coordinate position 232b. The aircraft coordinate position 232b indicates the coordinate position of the mark portion 203 with respect to the reference position of the aircraft of the aircraft 100 carrying the pilot 11. The machine body coordinate position 232b is input to the storage unit 232 after being measured when the mark unit 203 is installed on the machine body. The aircraft coordinate position 232b is an example of the "position of the mark portion" in the claims. Further, the aircraft 100 is an example of a "moving body" within the scope of claims.

The sensor unit 233 has an acceleration sensor 233a, a gyro sensor 233b, a tilt sensor 233c, and an orientation sensor 233d. The acceleration sensor 233a measures the acceleration of the airframe. The gyro sensor 233b measures the angular velocity of the airframe. The tilt sensor 233c measures the tilt of the airframe. The azimuth sensor 233d measures the azimuth of the aircraft. The gyro sensor 233b is an example of the "mark portion side gyro sensor" in the claims.

The mark unit side communication unit 234 is configured to be communicable with the control unit side communication unit 204f (see FIG. 13), which will be described later. The marker unit side communication unit 234 has a wireless communication function using light and radio waves. The mark unit side communication unit 234 performs wireless communication with the control unit side communication unit 204f by radio waves when adjusting / setting the machine coordinate position 232b of the storage unit 204b of the control unit 204. Further, the mark unit side communication unit 234 performs wireless communication with the control unit side communication unit 204f by light during normal use other than adjustment / setting. The mark unit side communication unit 234 transmits the information of the identification number 232a of the mark unit 203, the information of the machine coordinate position 232b of the mark unit 203, and the measurement value information of the sensor unit 233 to the control unit side communication unit 204f by light. It is configured in.

Here, the mark unit side communication unit 234 is configured to control the light emitting unit 203a to emit light. That is, the mark unit side communication unit 234 can superimpose light emission as a marker of the light emitting unit 203a and light emission of the light emitting unit 203a as wireless communication by light with the control unit side communication unit 204f. The mark unit side communication unit 234 receives the light emitted from the light emitting unit 204c on the control unit 204 side by the light receiving unit 203b.

As shown in FIGS. 13 and 14, the control unit 204 is composed of a computer that performs various controls and arithmetic processing. The control unit 204 includes an image processing unit 41, a CPU unit 204a, a storage unit 204b, a light emitting unit 204c, a light receiving unit 204d, a sensor unit 204e, and a control unit side communication unit 204f.

The CPU unit 204a has a function of causing the comparison process 42a, the position angle detection process 42b, and the sensor data process 240 to execute predetermined processes. The sensor data processing 240 performs a process of converting the measured value from the sensor unit 204e into the measured value information recognizable by the control unit 204. The sensor data processing 240 also performs a process of passing information transmitted from the mark portion 203. The sensor data processing 240 performs a process of converting the data transmitted from the control unit 204 into data that can be transmitted to the control unit 203c.

The storage unit 204b is a storage medium having a RAM and a ROM. The storage unit 204b stores the front frame map 53, the base map 51, and the aircraft coordinate position 232b.

The light emitting unit 204c is composed of an LED capable of irradiating ultraviolet light or near infrared light, a VCSEL, and the like. The optical axis of the light emitting unit 204c is arranged in the same direction as the optical axis of the camera 21. A lens for enabling light to be irradiated at a wide angle is attached to the light emitting unit 204c. The light emitting unit 204c can both emit light as illumination light of the light emitting unit 204c and emit light from the marker unit side communication unit 234 and the light emitting unit 204c as wireless communication by light by the control unit side communication unit 204f. Is.

The light receiving unit 204d is composed of a light receiving element such as a photodiode capable of receiving ultraviolet light or near infrared light. A lens for enabling light to be received at a wide angle is attached to the light receiving unit 204d.

The sensor unit 204e has an acceleration sensor 241, a gyro sensor 242, a tilt sensor 243, and an azimuth sensor 244. The accelerometer 241 measures the acceleration of the head of the pilot 11. The gyro sensor 242 measures the angular velocity of the head of the pilot 11. The tilt sensor 243 measures the tilt of the head of the pilot 11. The orientation sensor 244 measures the orientation of the head of the pilot 11. The gyro sensor 242 is an example of the "device-side gyro sensor" in the claims.

The control unit side communication unit 204f is configured to be able to communicate with the mark unit side communication unit 234. The control unit side communication unit 204f has a wireless communication function using light and radio waves. The control unit side communication unit 204f performs wireless communication with the mark unit side communication unit 234 by radio waves when adjusting and setting the machine coordinate position 232b of the storage unit 204b of the control unit 204. Further, the control unit side communication unit 204f performs wireless communication with the mark unit side communication unit 234 by light during normal use other than adjustment / setting. The control side communication unit is configured to receive the information of the identification number 232a of the mark unit 203, the information of the machine coordinate position 232b of the mark unit 203, and the measured value information of the sensor unit 233 from the mark unit side communication unit 234 by light. Has been done. The control unit side communication unit 204f is configured to cause the light emitting unit 204c to emit light as illumination light.

(Position and angle detection processing)
As shown in FIGS. 13 and 14. The control unit 204 of the second embodiment is configured to control the acquisition of the position and angle of the head of the pilot 11 based on the image acquired by the image acquisition unit 2. The position and angle of the head of the pilot 11 are shown as the position and angle of the midpoint of the two cameras 21.

When a sufficient number of feature points 12 cannot be extracted from the moved image 63 acquired by the image acquisition unit 2, the control unit 204 uses the measured value information of the sensor unit 233 and the measured value information of the sensor unit 204e. Based on this, it is configured to control the acquisition of the position and angle of the head of the pilot 11. Further, the control unit 204 is configured to perform the same control based on the front frame map 53 and the current frame map 52 even when the angle of the observer's head cannot be acquired within a predetermined time. At this time, the control unit 204 is configured to control the light of the light emitting unit 204c to transmit a signal requesting the measurement value information of the sensor unit 204e to the mark unit 203 acquired by the image acquisition unit 2. ..

Specifically, the control unit 204 of the pilot 11's head measured by the gyro sensor 242 when the angle of the pilot 11's head cannot be acquired within a predetermined time based on the front frame map 53 and the current frame map 52. It is configured to control the acquisition of the angle of the head of the pilot 11 based on the difference between the angular velocity and the angular velocity of the aircraft 100 measured by the gyro sensor 233b. That is, the information on the angular velocity of the aircraft 100 measured by the gyro sensor 233b is used to offset the information on the angular velocity of the aircraft 100 included in the angular velocity of the head measured by the gyro sensor 242. As a result, the control unit 204 can acquire the information on the angular velocity of the aircraft 100 by optical communication, so that the control unit 204 does not need to use a dedicated cable to acquire the information on the angular velocity of the aircraft 100.

Further, in detail, when the angle of the head of the pilot 11 cannot be acquired within a predetermined time based on the front frame map 53 and the current frame map 52, the control unit 204 measures the head of the pilot 11 by the acceleration sensor 241. It is configured to control the acquisition of the position of the head of the pilot 11 based on the difference between the acceleration of the unit and the acceleration of the aircraft 100 measured by the acceleration sensor 233a. That is, the information on the acceleration of the aircraft 100 measured by the acceleration sensor 233a is used to offset the information on the acceleration of the aircraft 100 included in the acceleration of the head measured by the acceleration sensor 241. As a result, the control unit 204 can acquire the acceleration information of the aircraft 100 by optical communication, so that the control unit 204 does not need to use a dedicated cable to acquire the acceleration information of the aircraft 100.

In addition to optical communication, the pilot 11 may input the attitude, azimuth, and the like of the aircraft 100 as needed.

The control unit 204 compares the angle of the head of the pilot 11 acquired by the gyro sensor 233b and the gyro sensor 242 with the angle of the head of the pilot 11 estimated based on the front frame map 53 and the current frame map 52. It is configured to control the operation. Further, the control unit 204 has a position of the head of the pilot 11 acquired by the acceleration sensor 233a and the acceleration sensor 241 and a position of the head of the pilot 11 estimated based on the front frame map 53 and the current frame map 52. It is configured to control the comparison. When the angle and position of the head of the pilot 11 match in both of the above two comparison results, the control unit 204 synthesizes the front frame map 53 and the base map 51, or combines the base map 51 with the base map 51. It is configured to control updating to the previous frame map 53.

The control unit 204 has a post-movement environment map in order to prevent the front frame map 53 and the current frame map 52 from being similar to each other and to prevent the accuracy of the position and angle of the head of the pilot 11 from deteriorating due to disturbance or the like. It is configured to estimate the position and angle of the head of the pilot 11 by the aircraft coordinate position 232b of the mark portion 203 instead of the coordinate position of the feature point 12 on the 54. Specifically, when the mark unit 203 is acquired by the image acquisition unit 2, the control unit 204 pilots based on the machine coordinate position 232b of the mark unit 203 acquired from the storage unit 232 via the marker unit side communication unit 234. It is configured to control the acquisition of the position and angle of the head of the eleven. This makes it possible to reset (reset) the position and angle of the head of the pilot 11 each time the image acquisition unit 2 acquires the mark unit 203. Here, when the position and angle of the head of the pilot 11 are reset, the control unit 204 may perform control to correct the coordinate position of the front frame map 53 based on the aircraft coordinate position 232b of the mark unit 203. preferable. The aircraft coordinate position 232b is an example of the "position of the mark portion" in the claims.

As shown in FIG. 15, the control unit 204 has heads of a plurality of pilots 11 having different head positions and angles from each other based on the positions and angles of the heads of the pilots 11 in the frames prior to the current frame. When creating a position and an angle, it is configured to control the position and angle of the heads of the plurality of pilots 11 to be created. That is, the control unit 204 is configured to perform control for limiting the positions and angles of the heads of the plurality of pilots 11 to be created based on the measured value information of the tilt sensor 243 and the azimuth sensor 244. As a result, the positions and angles of the heads of the plurality of pilots 11 to be referred to and compared are reduced, so that the processing of the control unit 204 can be reduced. The other configurations of the second embodiment are the same as the configurations of the first embodiment.

(Effect of the second embodiment)
In the second embodiment, the following effects can be obtained.

In the second embodiment, as described above, the head motion tracker device 210 is provided with the image acquisition unit 2 attached to the head mounting unit 1. Further, the head motion tracker device 210 is provided with a control unit 204 configured to control the acquisition of the position and angle of the head of the pilot 11 based on the image acquired by the image acquisition unit 2. As a result, the head motion tracker device 210 can be easily applied even to the existing aircraft 100.

Further, in the second embodiment, as described above, the control unit 204 is provided with the control unit side communication unit 204f. The mark unit 203 is provided with a mark unit side communication unit 234 capable of communicating with the control unit side communication unit 304f and a storage unit 232 for storing the position of the mark unit 203 with respect to the reference position of the aircraft 100 on which the pilot 11 is placed. When the control unit 204 acquires the mark unit 203 by the image acquisition unit 2, the position of the head of the pilot 11 and the position of the head of the pilot 11 based on the position of the mark unit 203 acquired from the storage unit 232 via the marker unit side communication unit 234. It is configured to control the acquisition of the angle. As a result, the accuracy of the position of the mark portion 203 can be improved as compared with the case of acquiring the position of the mark portion 203 from the image acquired by the image acquisition unit 2, so that the three-dimensional ambient environment map 5 can be used. It is possible to improve the accuracy of the position of the mark portion 203 of. As a result, the control unit 204 can acquire a more accurate position and angle of the head of the pilot 11.

Further, in the second embodiment, as described above, the head motion tracker device 210 is provided with a gyro sensor 242 for measuring the angular velocity of the head of the pilot 11. A gyro sensor 233b for measuring the angular velocity of the aircraft 100 carrying the pilot is provided on the mark portion 203. When the angle of the head of the pilot 11 cannot be acquired within a predetermined time by the control unit 204 based on the ambient environment map 5, the angular velocity of the head of the pilot 11 measured by the gyro sensor 242 and the angular velocity of the head of the pilot 11 are measured by the gyro sensor 233b. It is configured to control to acquire the angle of the head of the pilot 11 based on the difference from the angular velocity of the aircraft 100. As a result, when the angle of the head of the pilot 11 cannot be obtained based on the surrounding environment map 5, the angle of the head of the pilot 11 can be complemented by the gyro sensor 242 and the gyro sensor 233b, so that the head of the pilot 11 can be complemented. The angle of the part can be surely acquired. The other effects of the second embodiment are the same as the effects of the first embodiment.

[Third Embodiment]
Next, the head motion tracker device 310 of the third embodiment will be described with reference to FIGS. 16 and 17. Specifically, unlike the head motion tracker device 210 of the second embodiment, which includes the mark portion 203 having the light emitting unit 203a, the mark portion 304 reflects and emits light in the head motion tracker device 310 of the third embodiment. In the third embodiment, the same components as those in the second embodiment are designated by the same reference numerals and the description thereof will be omitted.

Specifically, as shown in FIGS. 16 and 17, the head motion tracker device 310 includes a head mounting unit 1, an image acquisition unit 302, a light emitting unit 303, a marker unit 304, and a control unit 305. I have.

The image acquisition unit 302 includes one camera 21 and a light receiving optical system (not shown). Specifically, the camera 21 is an area sensor camera. The camera 21 is sensitive to ultraviolet light and near infrared light. The reflected light emitted from the light emitting unit 303 and reflected on the inner surface of the machine body is incident on the camera 21 via the light receiving optical system. The light-receiving optical system includes a filter, a lens, and the like that allow only light in a specific wavelength region to pass through so that the camera 21 does not receive light other than the specific wavelength region.

The image acquisition unit 302 has a function of calculating the distance. Specifically, the image acquisition unit 302 is configured to be able to acquire a distance based on the difference between the wavelength of the light emitting unit 303 at the time of light emission and the wavelength of the reflected light of the light emitting unit 303.

The light emitting unit 303 is composed of an LED capable of irradiating ultraviolet light or near infrared light, a VCSEL, and the like. The optical axis of the light emitting unit 303 is arranged in the same direction as the optical axis of the camera 21. Here, the light emitting unit 303 is preferably light having a wavelength that is not easily affected by ambient light. The light emitting unit 303 is attached with a lens for irradiating light at a wide angle.

Here, in order to improve the function of calculating the distance, the image acquisition unit 302 is provided with a mechanism (motor or the like) capable of changing the optical axis direction angle of the camera 21 and the optical axis direction angle of the light emitting unit 303. It is preferable to have.

The mark portion 304 is arranged around the pilot 11 and is acquired as an image by the image acquisition portion 302. The mark portion 304 has a function as a marker. The mark portion 304 is composed of a retroreflector or a reflector having a reflection characteristic in a specific wavelength region in order to facilitate the feature point extraction process. A plurality (three) of the mark portions 304 are arranged on the moving body. In addition, the mark portion 304 may be arranged in 1, 2 or 4 or more.

The plurality of marking portions 304 are arranged in a region where estimation accuracy of the position and angle of the head of the pilot 11 is particularly required, a region where it is difficult to extract the feature point 12, and the like.

The control unit 305 is composed of a computer that performs various controls and arithmetic processing. The control unit 305 includes an image processing unit 41, a CPU unit 305a, a storage unit 305b, and a sensor unit 305c.

The CPU unit 305a has a function of executing the comparison process 42a, the position angle detection process 42b, and the sensor data process 350. The sensor data processing 350 performs a process of converting the measured value from the sensor unit 305c into the measured value information recognizable by the control unit 305.

The storage unit 305b is a storage medium having a RAM and a ROM. The storage unit 305b stores the front frame map 53, the base map 51, and the aircraft coordinate position 306.

The sensor unit 305c has an acceleration sensor 351, a gyro sensor 352, a tilt sensor 353, and an azimuth sensor 354. The gyro sensor 352 is an example of the "device-side gyro sensor" in the claims.

(Position and angle detection processing)
As shown in FIGS. 16 and 17. The control unit 305 of the third embodiment is configured to control to acquire the position and angle of the head of the pilot 11 based on the image acquired by the image acquisition unit 302. The position and angle of the head of the pilot 11 are shown as the position and angle of the midpoint of the tip of one camera 21.

Here, since one camera 21 is an area sensor camera, the image processing unit 41 can acquire information on the XZ plane in the three-dimensional map. Further, the image acquisition unit 302 can acquire information in the Y-axis direction in the three-dimensional map by the distance measurement function.

Specifically, the control unit 305 is configured to control to acquire the position and angle of the head of the pilot 11 based on the portion of the image acquired by the image acquisition unit 302 in the aircraft 100. There is. That is, the control unit 305 heads the pilot 11 based on the shape of the skeleton in the aircraft 100 and the feature points 12 such as the switch and the display arranged in the aircraft 100 in the image acquired by the image acquisition unit 302. It is configured to control the acquisition of the position and angle of the unit. Further, as the feature point 12 of the image, the mark portion 304 is also the feature point 12.

The control unit 305 is configured to control the light emitted from the light emitting unit 303 to acquire a plurality of feature points 12 by using the reflected light from the switch, the display, the mark unit 304, and the like. Here, the control unit 305 determines the angle and reflectance of the light from the light emitting unit 303 with respect to the surface of the machine body, the region where the light from the light emitting unit 303 does not reach, and the information in the image due to the contrast in the image. In order to suppress the occurrence of defects, it is configured to perform control to complement the information in the image by the distance calculated by the image acquisition unit 302. The other configurations of the third embodiment are the same as the configurations of the second embodiment.

(Effect of the third embodiment)
In the third embodiment, the following effects can be obtained.

In the third embodiment, as described above, the head motion tracker device 310 is provided with the image acquisition unit 302 attached to the head mounting unit 1. Further, the head motion tracker device 310 is provided with a control unit 305 configured to control the acquisition of the position and angle of the head of the pilot 11 based on the image acquired by the image acquisition unit 302. Thereby, the head motion tracker device 310 can be easily applied even to the existing aircraft 100. The other effects of the third embodiment are the same as the effects of the second embodiment.

[Modification example]
It should be noted that the embodiments disclosed this time are exemplary in all respects and are not considered to be restrictive. The scope of the present invention is shown by the scope of claims rather than the description of the above-described embodiment, and further includes all modifications (modifications) within the meaning and scope equivalent to the scope of claims.

For example, in the first to third embodiments, the aircraft 100 is an example of a "moving body" in the claims, but the present invention is not limited to this. In the present invention, the moving body may be a vehicle.

Further, in the first to third embodiments, the position / angle detection process 42b by the particle filter has been described as an example, but the present invention is not limited to this. In the present invention, the position / angle detection process may be performed by an extended Kalman filter or the like.

Further, in the first to third embodiments, the control unit 4 (204,305) has shown an example in which the position angle detection process 42b is performed at both the feature point 12 and the mark unit 3, but the present invention relates to this. Not limited. In the present invention, the control unit may perform the position / angle detection process at either the feature point or the mark unit. For example, as in the first modification shown in FIG. 18, in the region where the mark portion 3 is not arranged, the control unit performs the position angle detection process 42b only by the feature point 12 based on the frame of the aircraft 100. It may be configured as follows.

Further, in the first to third embodiments, the control unit 4 (204, 305) compares the front frame map 53 (second surrounding environment map) with the current frame map 52 (third surrounding environment map). Based on the above, an example configured to perform the position / angle detection process 42b has been shown, but the present invention is not limited to this. In the present invention, the control unit may be configured to perform the position / angle detection process based on the comparison between the base map and the third surrounding environment map. Further, the control unit may be configured to perform the position / angle detection process based on the comparison between the base map, the second surrounding environment map, and the third surrounding environment map.

Further, in the first to third embodiments, the head motion tracker device 10 (210, 310) has shown an example including the mark portion 3 (203, 304), but the present invention is not limited to this. .. In the present invention, the head motion tracker device may not have a marker portion.

Further, in the first to third embodiments, the pre-movement image 62 (second image) shows an example of one image, but the present invention is not limited to this. In the present invention, there may be a plurality of second images.

Further, in the first and second embodiments, the mark portion 3 is configured to emit light, but the present invention is not limited to this. In the present invention, the mark portion may have a configuration in which the shape of the mounting portion changes depending on the angle observed from the image acquisition portion, or has a configuration in which the shape of the light of the light emitting portion changes. You may be doing it.

Further, in the second embodiment, the control unit 204 has shown an example of acquiring the identification number 232a and the aircraft coordinate position 232b of the mark unit 3 by optical communication from the mark unit 203, but the present invention is limited to this. I can't. In the present invention, the control unit may obtain the position and identification number of the mark portion by the image acquisition unit by providing an AR (Augmented Reality) tag (QR (Quick Response) code, AprilTAG, etc.) on the mark portion. ..

Further, in the third embodiment, the head motion tracker device 310 shows an example including one image acquisition unit 302, but the present invention is not limited to this. In the present invention, as in the second modification shown in FIG. 19, the head motion tracker device 410 may include two image acquisition units 402. In this case, even if the head motion tracker device 410 includes a stereo image processing unit 403 that acquires a distance based on the parallax of the two image acquisition units 402 by synchronizing the two image acquisition units 402 for shooting. good.

Further, in the first to third embodiments, the control unit 4 (204, 305) controls to create a three-dimensional ambient environment map 5 around the pilot 11 (observer) by machine learning. Although the configured example is shown, the present invention is not limited to this. In the present invention, the control unit may create a three-dimensional ambient environment map by a method other than machine learning.

Further, in the first to third embodiments, only the pilot 11 (observer) is on board in the aircraft 100 (mobile body), but the present invention is not limited to this. In the present invention, a plurality of observers may be on board in the moving body. Each of the plurality of observers may be equipped with a head motion tracker device. In this case, the plurality of head motion tracker devices may each include an image acquisition unit capable of receiving different light (ultraviolet light and near-infrared light). As a result, a plurality of head motion tracker devices can be operated at the same time.

1 Head mounting part 2, 302, 402 Image acquisition part 3, 203, 304 Mark part 4, 204, 305 Control part 5 Surrounding environment map 10, 210, 310, 410 Aircraft head motion tracker device (head motion tracker device)
11 Pilot (observer)
12 Feature points 21 Camera 52 Current frame map (3rd surrounding environment map)
53 Previous frame map (second surrounding environment map)
54 Post-movement environment map (1st surrounding environment map)
62 Image before moving (second image)
63 Image after movement (first image)
100 Aircraft 204f Control unit side communication unit 232 Storage unit 233b Gyro sensor (mark part side gyro sensor)
234 Mark side communication part 242 Gyro sensor (device side gyro sensor)

Claims (13)

The head-mounted part that is attached to the observer's head,
An image acquisition unit that is attached to the head mounting unit and acquires an image in a direction along the direction of the observer's head, and an image acquisition unit.
A head motion tracker device including a control unit configured to control the position and angle of the observer's head based on the image acquired by the image acquisition unit. The control unit controls to create a three-dimensional surrounding environment map around the observer based on the image in the direction along the direction of the observer's head, and is based on the surrounding environment map. The head motion tracker device according to claim 1, further controlled to acquire the position and angle of the observer's head. The surrounding environment map includes a first surrounding environment map composed of a first image acquired by the image acquisition unit at a present time.
The second aspect of the present invention, wherein the control unit is configured to control to estimate the position and angle of the observer's head based on a plurality of feature points included in the first ambient environment map. Head motion tracker device. The surrounding environment map includes a second surrounding environment map composed of a second image before the first image.
The control unit is configured to control to estimate the position and angle of the observer's head based on at least the plurality of feature points included in the first ambient environment map and the second ambient environment map. The head motion tracker device according to claim 3. The surrounding environment map further includes a third surrounding environment map based on an image obtained by synthesizing the first image with the second image.
The control unit acquires the first surrounding environment map and estimates the position and angle of the observer's head based on the comparison between the second surrounding environment map and the third surrounding environment map. The head motion tracker device according to claim 4, which is configured to perform control. The control unit estimates the position and angle of the observer's head capable of acquiring all the plurality of feature points included in the first ambient environment map as the current position and angle of the observer's head. The head motion tracker device according to any one of claims 3 to 5, which is configured to perform control. The control unit creates a plurality of positions and angles of the observer's heads having different head positions and angles from each other based on the positions and angles of the observer's heads at a time point before the present time. At the same time, after changing the positions and angles of the heads of the plurality of observers created, the positions of the heads of the observers capable of acquiring all of the plurality of feature points included in the first surrounding environment map. The head motion tracker device according to claim 6, wherein the head motion tracker device is configured to control the selection of an angle. The head according to any one of claims 3 to 7, wherein the control unit is configured to control acquisition of the plurality of feature points based on the difference in contrast in the first image. Motion tracker device. Further provided with a marker portion arranged around the observer and acquired as the image by the image acquisition unit.
The control unit controls to acquire the position and angle of the observer's head based on the mark portion in the three-dimensional ambient environment map based on the image obtained from the mark portion. The head motion tracker device according to any one of claims 2 to 8, which is configured. The control unit includes a control unit side communication unit.
The mark is
A marker side communication unit capable of communicating with the control unit side communication unit,
Includes a storage unit that stores the position of the mark portion with respect to the reference position of the moving body on which the observer is placed.
When the mark portion is acquired by the image acquisition unit, the control unit determines the position of the observer's head based on the position of the mark portion acquired from the storage unit via the mark portion side communication unit. The head motion tracker device according to claim 9, wherein the head motion tracker device is configured to control the acquisition of an angle. Further equipped with a device-side gyro sensor for measuring the angular velocity of the observer's head,
The mark portion includes a mark portion side gyro sensor that measures the angular velocity of the moving body carrying the observer.
When the angle of the observer's head cannot be acquired within a predetermined time based on the ambient environment map, the control unit has the angular velocity of the observer's head measured by the device-side gyro sensor and the mark. The head according to claim 9 or 10, which is configured to control to acquire the angle of the observer's head based on the difference from the angular velocity of the moving body measured by the unit-side gyro sensor. Motion tracker device. The image acquisition unit has two cameras arranged side by side.
The control unit acquires the distance in the direction along the direction of the observer's head based on the parallax of the two cameras, and the three-dimensional said distance based on the distance acquired by the two cameras. The head motion tracker device according to any one of claims 2 to 11, which is configured to control the creation of an ambient environment map. Equipped with a head motion tracker device worn by the pilot as an observer inside the aircraft
The head motion tracker device is
A head-mounted portion attached to the pilot's head,
An image acquisition unit that is attached to the head mounting unit and acquires an image in a direction along the direction of the pilot's head, and an image acquisition unit.
An aircraft head motion tracker device comprising a control unit configured to control the acquisition of the position and angle of the pilot's head based on the image acquired by the image acquisition unit.

Images