JP2008026236A - Position and attitude measuring instrument, and position and attitude measuring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect not only a position of a marker but also an attitude thereof, using the single marker. <P>SOLUTION: This position and attitude measuring instrument for measuring the position and attitude of an object is provided with a marker device 100 provided with a pattern filter formed with a color-coded transmission pattern, an illumination light source for illuminating the transmission pattern of the pattern filter, and a projection optical system (fisheye lens) for projecting an illuminated transmission pattern image to a space, imaging devices 10, 20 for imaging the transmission pattern image projected by the projection optical system, and a position and attitude calculator 200 for calculating the position and attitude of the marker device, based on an image data imaged by the imaging devices 10, 20. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a position and orientation measurement apparatus and a position and orientation measurement method for measuring the position and orientation of a marker by photographing a marker.

  For example, a marker used in a conventional motion capture or position measurement device is configured to geometrically change the posture of the marker by fixing a plurality of light emitting points (for example, LEDs) at a fixed position in three dimensions. It was detected. As an example of the transducer for determining the position and orientation, for example, a special table 2004-510137 is cited. In addition, as an example of a passive marker, for example, a position is detected by mounting a marker using a reflective material or the like.

In addition, as a method for measuring an imaging position with a camera, there is a special table 2003-528478 for a positioning application for optical communication.
Special table 2004-510137 Special table 2003-528478

  However, the conventional techniques including the above cited references cannot cope with the point that the posture of the marker is detected by a single marker.

  The present invention has been made paying attention to such a problem, and its object is to provide a position / orientation measurement apparatus and position / orientation measurement that can detect the position and orientation of a marker using a single marker. It is to provide a method.

  In order to achieve the above object, according to a first aspect of the present invention, there is provided a position / orientation measuring apparatus for measuring a position / orientation of an object, the transmitting member having a coded transmitting pattern formed thereon, and the transmitting member. An illumination device that illuminates the transmission pattern, a marker optical system that projects the illuminated transmission pattern image into space, and an imaging device that captures the transmission pattern image projected by the projection optical system; A position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device.

  According to a second aspect of the present invention, in the first aspect, the projection optical system is a fisheye lens.

  According to a third aspect of the present invention, in the first aspect, the projection optical system is an all-around panoramic lens.

  According to a fourth aspect of the present invention, there is provided a position / orientation measurement apparatus for measuring a position / orientation of an object, wherein a first transmission member on which a coded transmission pattern is formed, and a transmission pattern of the transmission member are obtained. A first illumination device that illuminates; a first projection optical system that projects an illuminated transmission pattern image into space; a first optical path splitting element provided coaxially with the projection optical system; and the optical path split A first marker device provided along one of the optical path dividing paths by the element, a second transmissive member on which a coded transmissive pattern is formed, and the transmission A second illumination device that illuminates the transmission pattern of the member; a second projection optical system that projects the illuminated transmission pattern image into space; and a second optical path dividing element that is provided coaxially with the projection optical system And optical path splitting by the optical path splitting element A second marker apparatus comprising a second image sensor, the provided along one, comprises a.

  According to a fifth aspect of the present invention, there is provided a position / orientation measuring apparatus for measuring a position / orientation of an object, a light emitting element array arranged in a code pattern, and a light emission control apparatus for controlling light emission of the light emitting element array. A marker optical device that projects an array image from the light emitting element array into space, an image pickup device that picks up a transmission pattern image projected by the projection optical system, and an image picked up by the image pickup device And a position / orientation calculation unit that calculates the position and orientation of the marker device based on the image data.

  According to a sixth aspect of the present invention, there is provided a position / orientation measuring apparatus for measuring a position / orientation of an object, wherein the coded transmission pattern is formed and illumination for illuminating the transmission pattern of the transmission member. A marker device comprising a device, a projection optical system that projects an illuminated transmission pattern image into space, and an image sensor provided coaxially with the projection optical system; An imaging device that captures a transmission pattern image projected by the projection optical system; and a position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device.

  According to a seventh aspect of the present invention, in the sixth aspect, the marker device includes the retroreflective member and a diffuse reflective member including the retroreflective member.

  Further, an eighth aspect of the present invention is a position and orientation measurement method for measuring the position and orientation of an object, the step of illuminating a transmission pattern of a transmission member on which a coded transmission pattern is formed with an illumination device; The step of projecting the illuminated transmission pattern image into the space by the projection optical system, the step of imaging the transmission pattern image projected by the projection optical system, and the marker device based on the image data captured by the imaging device And calculating a position and orientation by a position and orientation calculation unit.

  According to the present invention, it is possible to detect the position and posture of a marker using a single marker.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of the position / orientation measurement apparatus according to the first embodiment of the present invention. The position / orientation measurement apparatus according to the present embodiment includes a marker device 100, imaging devices 10 and 20, and a position / orientation calculation device 200. Between the imaging devices 10 and 20 and the position / orientation calculation device 200, image transmission devices 11 and 21 are provided. The image data is transferred using It is assumed that the marker device 100 is installed at the center of the hemisphere of the dome-shaped screen.

  As shown in FIG. 2, the marker device 100 includes a fisheye lens 103, a pattern filter 104, and an illumination light source 105, and an illumination control device 101 that controls the illumination light source 105.

  The pattern filter 104 is a color-coded pattern filter that is installed on the focal plane of the fish-eye lens 103 that is an fθ lens and projects the coordinates of the space.

  FIGS. 3A and 3B show configuration examples of the pattern filter 104, where FIG. 3A is a simplified pattern and FIG. The pattern filter 104 in FIG. 3A includes a plurality of partitioned regions (regions 1, 2,..., N) formed by concentric divisions at equal intervals and divisions along the XY axes. Each area has a different color pattern and is color coded. The pattern filter 104 in FIG. 3B includes a plurality of partitioned regions (regions 1, 2,..., N) formed by concentric divisions at equal intervals and radial divisions. Each area has a different color pattern and is color coded.

  The pattern filter 104 is disposed on the focal plane of the fisheye lens 103. Here, the angle of view of the fisheye lens 103 is 180 degrees, and the pattern is divided into concentric circles at equal intervals as shown in FIG. If it is configured by division, the dome-shaped space on the extension line of the marker device 100 is divided into three in the vertical direction and eight in the peripheral direction. The imaging devices 10 and 20 are installed in any one of the divided spaces.

  On the other hand, the position / orientation calculation apparatus 200 includes an image reception apparatus 201, a marker detection apparatus 204, a calibration apparatus 205, a pattern storage apparatus 206, a position calculation apparatus 207, an attitude identification apparatus 208, and a data output apparatus 211. The marker storage device 203.

  Next, the operation of the first embodiment will be described. First, the principle of posture detection according to this embodiment will be described. When the pattern filter 104 is illuminated by the illumination light source 105, the image is projected by the fisheye lens 103 toward the space of the dome-shaped screen. Thereby, a color pattern is projected. The front aperture of the fisheye lens 103 is imaged by the two imaging devices 10 and 20.

  When the imaging device 10 or 20 captures the pupil image of the fisheye lens 103 of the marker device 100, the color of the pupil image of the fisheye lens 103 changes depending on the direction in which the imaging device 10 or 20 captures the marker device 100. That is, as illustrated in FIG. 4, when a marker is photographed from different positions such as the imaging device 10 and the imaging device 20, for example, a blue pupil image is captured by the imaging device 10, and a yellow pupil image is captured by the imaging device 20, for example. Will be.

  As described above, in the present embodiment, the dome-shaped space is color-coded according to the photographing color of the pupil image, and by processing the image data captured by the imaging devices 10 and 20 arranged in such a space, The relative posture of the marker device 100 with respect to the imaging devices 10 and 20 can be detected.

  Next, the flow of processing will be described.

  The illumination control device 101 adjusts the light amount of the illumination light source 105 to illuminate the pattern filter 104. The image of the illuminated pattern filter 104 is projected into space through the fisheye lens 103. By photographing this with the imaging devices 10 and 20, the pupil of the fisheye lens 103 is photographed. The captured images are transmitted to the position / orientation calculation apparatus 200 by the image transmission apparatuses 11 and 21. Image data is received by the image receiving device 201 of the position / orientation calculation device 200. The image reception device 201 transmits the received image (image data group 202) to the marker detection device 204. The marker detection device 204 performs image processing such as binarization and color recognition using the color code stored in advance in the marker storage device 203 from the received image data, so that the pupil image of the fisheye lens 103 in the image is displayed. Is detected.

  The position calculation device 207 calculates the position of the marker recognized based on the calibration data calculated in advance by the calibration device 205 and stored in the pattern storage device 206, and outputs position data 209. This process may be performed in the same manner as in an optical motion capture device using a conventional marker.

  If a stereo camera is used as an imaging device as shown in FIG. 1, it is possible to calculate a three-dimensional position with respect to the imaging device, and between an arbitrarily set world coordinate system and the imaging device. If the above calibration is performed, the three-dimensional position in the world coordinates can be calculated.

  In addition to the above, the present embodiment is characterized in that the posture of the marker can be recognized even with a single marker. That is, the posture identification device 208 identifies the posture of the marker based on the color code stored in the marker storage device 203 and the calibration data from the calibration device 205 and outputs the posture data 210. This is processing for identifying in which space of the coding space the imaging devices 10 and 20 are located from the color information of the pupil image. The data output device 211 outputs position data 209 and posture data 210 as position / posture data 212.

(Modification)
FIG. 5 is a diagram for explaining a modification of the first embodiment. The imaging devices 10 and 20 are generally configured to perform exposure adjustment according to the brightness within the imaging range. If the light emission amount of the marker becomes excessive due to a change in external light, the brightness becomes out of the dynamic range of the imaging devices 10 and 20, and color recognition becomes difficult. In addition, when the light emission amount of the marker is insufficient, it is difficult to extract the marker from the image data. For this reason, an external light sensor may be provided in the illumination control device 101, and the light emission amount may be adjusted according to the external light condition.

  FIG. 5 shows an example of an external light sensor, which includes a beam splitter 107 as an optical path splitting unit composed of a half prism and a half mirror for splitting the output light from the pattern filter 104, and the split light. And a photometric element 108 for performing photometry. Although the brightness of the entire environment may be observed by providing such an external light sensor, the image transmission device 21 and the image reception device 201 have a function of communicating exposure information from the imaging devices 10 and 20, The luminance of the illumination light source may be controlled by the illumination control device 101 in accordance with the exposure conditions of the imaging devices 10 and 20.

  Of course, various modifications and changes can be made to each configuration of the embodiment of the present invention. For example, FIG. 1 shows an example in which a plurality of stereo imaging devices are used, but a single specification or a monocular camera arranged so that it can be viewed in stereo may be used, and two or more cameras are in the same area in the space. As long as the image is captured. As the fish-eye lens, various optical systems for the purpose of panoramic viewing and panoramic viewing have been proposed as conventional techniques, and these may be used. Further, as shown in FIG. 6, the marker may be configured in a panoramic system using a catadioptric / circumferential panoramic lens 109. In this case, the direction can be recognized by changing the color depending on the viewing direction.

(Second Embodiment)
FIG. 7 is a block diagram showing a configuration of a position and orientation measurement apparatus according to the second embodiment of the present invention.

The position / orientation measurement apparatus according to the present embodiment includes a marker device 100, imaging devices 10 and 20, and a position / orientation calculation device 200. Between the imaging devices 10 and 20 and the position / orientation calculation device 200, image transmission devices 11 and 21 are provided. The image data is transferred using

  As shown in FIG. 8, the marker device 100 includes a fisheye lens 103, a pattern filter 104, an illumination light source 105, a beam splitter 107, and an image sensor 108, and an illumination control device 101 that controls the illumination light source 105. And an image transmission device 110 that transmits image data captured by the image sensor 108.

  The position / orientation calculation apparatus 200 includes an image reception apparatus 201, a marker detection apparatus 204, a calibration apparatus 205, a pattern storage apparatus 206, a position calculation apparatus 207, an attitude identification apparatus 208, and a data output apparatus 211. The marker storage device 203.

  As shown in FIGS. 8A and 8B, a color-coded pattern filter 104 that projects spatial coordinates is installed on the focal plane of the fish-eye lens 103. This is the same as in the first embodiment. Further, in the present embodiment, the image sensor 108 is provided at the tip of the optical path divided by the beam splitter 107 so as to be substantially the same focal position as the pattern filter 104. The arrangement of the illumination light source 105 and the image sensor 108 may be either the arrangement shown in FIGS.

  Next, the operation of this embodiment will be described. Similar to the first embodiment, the illumination control device 101 adjusts the light amount of the illumination light source 105 to illuminate the pattern filter 104. The illuminated image of the pattern filter 104 is projected into the space through the fisheye lens 103. By photographing this with the imaging devices 10 and 20, the pupil of the fisheye lens 103 is photographed. The captured images are transmitted to the position / orientation calculation apparatus 200 by the image transmission apparatuses 11 and 21. Image data is received by the image receiving device 201 of the position / orientation calculating device 200.

  The marker detection device 204 uses the color code stored in the marker storage device 203 in advance from the received image data, and performs image processing such as binarization and color recognition, so that the pupil image of the fisheye lens 103 in the image is displayed. Is detected.

  The position calculation device 207 calculates the position of the recognized marker based on the calibration data calculated in advance by the calibration device 205. This process may be performed in the same manner as in an optical motion capture device using a conventional marker.

  As shown in FIG. 7, if a stereo camera is used as the imaging devices 10 and 20, it is possible to calculate a three-dimensional position with respect to the imaging devices 10 and 20, and an arbitrarily set world coordinate system. If the calibration between the imaging devices is performed, the three-dimensional position in the world coordinates can be calculated.

  In addition to this, the present embodiment is characterized in that the posture of the marker can be recognized even with a single marker. The posture identification device 208 identifies the posture of the marker from the color code stored in the marker storage device 203 and the calibration data from the calibration device 205. This is processing for identifying in which space of the coding space the imaging devices 10 and 20 are located from the color information of the pupil image.

(Modification)
FIG. 9 is a diagram for explaining a modification of the second embodiment. Here, as shown in FIG. 9, markers having a plurality of image sensors (108-1 and 108-2 in FIG. 9) are installed facing each other, and posture recognition between the two-way markers is performed. It is aimed. Here, as described in the first embodiment, the identification is not performed based on the image data captured by the imaging device arranged separately from the marker, but by the imaging element in the marker arranged to face each other. The marker posture is identified based on the image data.

  As shown in FIG. 10A, the marker itself can be observed, but if it is not within the angle of view of the other party, the light emission of the pupil image is not detected. On the other hand, as shown in FIG. 10 (b), if both are within the angle of view of each other, light emission of the pupil image of the marker is detected by both.

  As a specific effect of the present embodiment, when the pattern filter and the image sensor have the same angle of view, self-imaging is performed at the other party's angle of view with both markers by performing coaxial projection and shooting. Recognize that the device has entered. As a result, it is possible to recognize whether or not the markers exist in the imaging region of the other party.

  Of course, various modifications and changes can be made to each configuration of the embodiment of the present invention. For example, as shown in FIG. 11, a plurality of markers can be integrated, for example, a form capable of recognizing the posture of the entire periphery can be taken. A marker device 170 shown in FIG. 11 is installed outside the housing 163 and is a combination of a configuration including the lighting device 162 and the fisheye lens 160 and a configuration installed inside the housing 163 and including the lighting device 164 and the fisheye lens 165. It is. Reference numeral 161 denotes a projection angle of the fisheye lens. Since the projection angle range of the two fisheye lenses covers the entire periphery, the image of at least one fisheye lens can always be taken by an external camera regardless of the posture of the marker device. For this reason, the posture of the marker device can always be detected, and the application range is wide.

  In this example, in the range where the projection angles of the two fisheye lenses 160 and 165 overlap, the pupil images of the two fisheye lenses can be taken simultaneously by the external camera. For this reason, the secondary information that the distance information to the marker can be estimated to some extent from the image of the two pupils whose relative distance is known and the detected posture of the marker device, even if the outside is a monocular camera. There is also a positive effect.

(Third embodiment)
FIG. 12 is a block diagram showing a configuration of a position and orientation measurement apparatus according to the third embodiment of the present invention. FIG. 13 is an enlarged view of a part of FIG. In the first and second embodiments, the pattern is projected by illuminating the pattern filter 104. However, the third embodiment is characterized in that a light emitting element array 109 that emits light is used.

  As the light emitting element itself, an array of a laser diode, a light emitting diode, plasma, an organic EL, or the like is used. FIG. 14 shows a configuration example in which color LEDs are arranged in an array. In the third embodiment, as shown in FIG. 13, the light emitting element array 109 is installed in the portion where the transmission pattern is arranged in the first and second embodiments. The light emitting element array 109 is controlled by the light emission control device 110 for its light emission pattern, blinking, color, and the like. Other configurations, operations, and effects are the same as those in the first and second embodiments, and thus description thereof is omitted here. As a result, more efficient use of light is possible, and the brightness of the marker can be effectively controlled.

  As in the second embodiment, as shown in FIG. 15, a beam splitter 107 is provided as an optical path splitting unit composed of a half prism, a half mirror, etc., and an image is picked up at substantially the same focal position as the light emitting element array 109. An element 108 may be provided. In this way, as shown in FIG. 16, it is possible to shoot with the marker devices 300, 301, 302 facing each other, and thus it is possible to estimate the posture of each other.

  In addition, as shown in FIG. 17, a plurality of markers can be separately imaged with a stereo camera, the position is measured, and the posture of the marker can be detected from the position of the light emitting element array and its color. In FIG. 17, the marker device 100-1 includes an image transmission device 110-1, an illumination control device 101-1, an imaging element 108-1, a light emitting element array 109-1, and a beam splitter 107-1. Is done. The marker device 100-2 includes an image transmission device 110-2, an illumination control device 101-2, an image sensor 108-2, a light emitting element array 109-2, and a beam splitter 107-2. .

  Furthermore, as a specific effect of the present embodiment, for example, the camera ID, status, and various types of information may be exchanged using a light emission pattern.

  Of course, as a configuration equivalent to this, it is also possible to have a configuration that provides the same effect as the light-emitting element array 109 by controlling the blinking of the pattern with an LCD or the like. However, at this time, the utilization efficiency of light from the illumination light source is lowered.

  Also, instead of a color code or an image sensor, a cross-splitting PD, a half prism, and a laser beam light source can be combined to be used as a laser axis alignment device that can position the optical axis in a range exceeding 180 degrees. Also good.

  Further, it can be applied as an optical communication device that transmits and receives signals from multidirectional laser transmitters in multiple channels.

(Fourth embodiment)
The fourth embodiment of the present invention will be described below. In the first and second embodiments, a color light projecting device is provided for the marker, but here, a marker using a retroreflective material is used instead of the surrounding imaging device. FIG. 18 shows an example of such a retroreflective marker 350, which is encompassed by a diffusion marker 351.

  In the fourth embodiment, the marker including the camera and the projection device of the first and second embodiments for projecting and imaging is fixedly arranged, and the retroreflective marker 350 itself corresponding to the conventional camera moves. Become.

  Next, the function and effect of the fourth embodiment will be described. As shown in FIG. 19, the retroreflective marker 350 reflects the projected code light in its projection direction. The light reflected in the projection direction is divided into optical paths by the half prism 354 and is incident on the camera 353 or the projector 352. Further, the code light projected from the other direction is also reflected in the projection direction by the retroreflection marker 350. The light reflected in the projection direction is divided into optical paths by the half prism 357 and is incident on the camera 356 or the projector 355. In this way, it is possible to obtain the same effect as that obtained by imaging with the camera by the camera provided coaxially with the pattern projection of the second embodiment.

  That is, the transmission pattern light projected from the half prism 354 is reflected by the retroreflective marker 350 and is color-coded only in the imaging device (camera 353) provided coaxially with the projection device (projector 352). The reflected color will be reflected. The same applies to the transmission pattern light projected from the half prism 357.

  As shown in FIG. 20, the imaging apparatus 400 optically has the marker (virtual marker 402) shown in the first to third embodiments at a folded position with the retroreflective marker 401 as a symmetry point. And the same conditions as when imaging it. This achieves the same effect as the second embodiment without having a light source or a lens.

  Considering only the mirror image position because reflection is used, the distance is halved compared to the case of imaging the marker shown in the previous embodiment. If the above correction is used, the processing is the same as in the second embodiment.

  However, regarding distance measurement, since the same color cannot be captured by a plurality of cameras, it is difficult to determine which retroreflective marker is the same marker during stereo shooting. At this time, it is possible to discriminate by the mixed color of the projection pattern of the diffuse reflection portion. However, when there are a plurality of markers in each projection color segment, it is necessary to determine the position by detecting the position of the marker from the result of the position / orientation calibration calibration between the cameras.

  FIG. 21 is a diagram showing a state where lightweight retroreflective markers 350-1 and 350-2 are attached to a person 361. FIG. Here, the retroreflective markers 350-1 and 350-2 are used for capturing the motion of the human 361. As shown in FIG. 19, it is possible to easily determine where retroreflective markers 350-1 and 350-2 exist by color-coded illumination that is divided and projected. .

  In addition, the human 361 wearing the marker device uses a camera to detect the marker device based on the color data reflected by the diffusion markers 351-1 and 351-2 arranged around the retroreflective markers 350-1 and 350-2. It is possible to easily know where in the captured space area. Since the position of itself in the space can be known from the color data, it can be used as a feedback of the spatial position when meaning is given to a specific area in the space, for example, in the gesture input.

  As is common to the above embodiments, the brightness control of the pattern or pattern illumination performed by the LED or the like may be controlled so as not to be saturated within the exposure amount of the image sensor. Further, saturation, the appropriate luminance adjustment value, and extinction may be combined in an arbitrary pattern, and may be repeated by code transmission using a blinking sequence such as a so-called Morse code.

  When the marker is mounted on a mobile body having a communication function, the ID of the marker itself, information on communication, and light emission timing are stored in a fixed network (for example, Ethernet (registered trademark), IEEE801.2 wireless LAN, It may be distributed to a wired IEEE 1394, USB, PLC, ZigBee, Bluetooth (registered trademark) or other short-range wireless network. By doing so, it is possible to collate the relationship between the transmission pattern information and the light emission pattern information of the light emitting element array with the marker photographed by the imaging element, and the marker whose position is known by wireless network or wired Therefore, it is possible to specify the position of the marker whose position is unknown.

  The specific embodiments described above include inventions having the following configurations.

1. A position and orientation measurement apparatus for measuring the position and orientation of an object,
A transmission member on which a coded transmission pattern is formed;
An illumination device that illuminates a transmission pattern of the transmission member;
A projection optical system that projects an illuminated transmission pattern image into space, and a marker device comprising:
An imaging device that captures a transmission pattern image projected by the projection optical system;
A position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device;
A position and orientation measurement apparatus comprising:

(Corresponding Embodiment of the Invention)
Embodiments relating to the present invention correspond to the first and second embodiments described above. The projection optical system in the configuration corresponds to the fisheye lens 103, the transmission pattern corresponds to the pattern filter 104, and the illumination device corresponds to a configuration including the illumination light source 105 and the illumination control device 101.

(Function)
The transmission pattern is illuminated by an illumination device, and light is projected into space by a projection optical system. By photographing this using an imaging device, a code of a transmission pattern in which a code pattern is formed is obtained as a pupil image. By decoding this, the posture of the marker is calculated.

(effect)
In addition to the effect of position measurement by photographing a marker such as a normal LED with an imaging device, the posture information of the marker itself can be calculated by photographing a coded pupil image, so the posture of the marker is also recognized. be able to.

2. 2. The position / orientation measurement apparatus according to 1, wherein the projection optical system is a fisheye lens.

(Corresponding Embodiment of the Invention)
Embodiments relating to the present invention correspond to the first and second embodiments described above. The projection optical system in the configuration corresponds to the fisheye lens 1-3, but also includes a panoramic lens as shown in the modification. The transmission pattern corresponds to the pattern filter 104. The illumination device corresponds to a configuration including the illumination light source 105 and the illumination control device 101.

(Function)
The transmission pattern is illuminated by an illumination device, and light is projected into space by a projection optical system. By photographing this using an imaging device, a code of a transmission pattern in which a code pattern is formed is obtained as a pupil image. By decoding this, the posture of the marker is calculated.

(effect)
In addition to the effect of position measurement by photographing a marker such as a normal LED with an imaging device, the posture information of the marker itself can be calculated by photographing a coded pupil image, so the posture of the marker is also recognized. be able to.

3. 2. The position / orientation measurement apparatus according to 1, wherein the projection optical system is an omnidirectional panoramic lens.

(Corresponding Embodiment of the Invention)
Embodiments relating to the present invention correspond to the first and second embodiments described above. The projection optical system in the configuration corresponds to the catadioptric / all-round panoramic lens 109, the transmission pattern corresponds to the pattern filter 104, and the illumination device corresponds to a configuration including the illumination light source 105 and the illumination control device 101.

(Function)
The transmission pattern is illuminated by an illumination device, and light is projected into space by a projection optical system. By photographing this using an imaging device, a code of a transmission pattern in which a code pattern is formed is obtained as a pupil image. By decoding this, the posture of the marker is identified.

(effect)
In addition to the effect of position measurement by photographing a marker such as a normal LED with an imaging device, the posture information of the marker itself can be calculated by photographing a coded pupil image, so the posture of the marker is also recognized. be able to.

  Here, the term “all-around panoramic lens” is used in this specification to refer to an omnivision optical system that combines a hyperboloidal mirror, a paraboloidal mirror, and an imaging lens, or an annular catadioptric optical system. It means an optical system that can acquire panoramic images.

4). A position and orientation measurement apparatus for measuring the position and orientation of an object,
A position and orientation measurement apparatus for measuring the position and orientation of an object,
A first transmission member formed with a coded transmission pattern;
A first illumination device that illuminates a transmission pattern of the transmission member;
A first projection optical system that projects an illuminated transmission pattern image into space;
A first optical path splitting element provided coaxially with the projection optical system;
A first marker device provided with a first imaging element provided along one of the optical path splitting paths by the optical path splitting element;
A second transmissive member having a coded transmissive pattern formed thereon;
A second illumination device that illuminates the transmission pattern of the transmission member;
A second projection optical system that projects the illuminated transmission pattern image into space;
A first optical path splitting element provided coaxially with the projection optical system;
A second imaging device comprising a second imaging element provided along one of the optical path splitting paths by the optical path splitting element;
A position and orientation measurement apparatus comprising:

(Corresponding Embodiment of the Invention)
The embodiment relating to the present invention corresponds to a modification of the above-described second embodiment. Here, the marker device includes an imaging element. However, as in the case of 1, an imaging device that photographs the marker from the outside may be used.

(Function)
The transmission pattern in which the code pattern is formed is illuminated by an illuminating device, projected onto a space through a projection optical system, and a spatial image received through an optical path splitting element installed coaxially with the projection optical system is photographed by an image sensor. To do. A plurality of such marker devices are provided, and a plurality of markers photograph each other.

(effect)
Conventionally, in a sensing environment using a plurality of cameras, it has been difficult to recognize the angle of view that the other party may have photographed from the captured images in applications in which opposing photographing is performed and the images are mutually recognized. According to this configuration, it is possible to recognize whether or not there is a marker within the shooting angle of view and further to recognize the relative posture and direction of the marker by performing shooting between the markers.

5. A position and orientation measurement apparatus for measuring the position and orientation of an object,
A light emitting element array arranged in a code pattern;
A light emission control device for controlling light emission of the light emitting element array;
A projection optical system that projects an array image from the light emitting element array into space, and a marker device,
An imaging device that captures a transmission pattern image projected by the projection optical system;
A position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device;
A position and orientation measurement apparatus comprising:

(Corresponding Embodiment of the Invention)
The embodiment relating to the present invention corresponds to the first embodiment described above. The light emitting element array in the configuration is, for example, an LD array, an LED array, an organic EL panel, or the like. Further, the light emitting element itself may be provided with a convex lens as represented by an LED by a mold or the like so that light converged in the optical axis direction is emitted.

(Function)
The light emitting element emits light in accordance with a predetermined pattern, and the light is projected by a projection optical system that projects directly onto the space.

(effect)
By using a self-luminous element, the illumination optical system is simplified, and as the characteristic of the light-emitting element, light converged in parallel with the optical axis is emitted, so that the projection efficiency is high. In addition, it is possible to control light emission while changing the light emission pattern with sufficient luminance for the transmission pattern.

6). A position and orientation measurement apparatus for measuring the position and orientation of an object,
A transmission member on which a coded transmission pattern is formed;
An illumination device that illuminates a transmission pattern of the transmission member;
A projection optical system that projects an illuminated transmission pattern image into space;
An imaging device provided coaxially with the projection optical system, and a marker device formed of a retroreflective material;
An imaging device that captures a transmission pattern image projected by the projection optical system;
A position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device;
A position and orientation measurement apparatus comprising:

(Corresponding Embodiment of the Invention)
The embodiment relating to the present invention corresponds to the above-described fourth embodiment.

(Function)
The transmission pattern is illuminated by an illumination device, and light is projected into space by a projection optical system. By photographing this using an imaging device, a code of a transmission pattern in which a code pattern is formed is obtained as a pupil image. By decoding this, the posture of the marker is calculated.

(effect)
It is possible to easily recognize at which position within the camera angle of view the marker exists.

7). 7. The position / orientation measuring apparatus according to claim 6, wherein the marker device includes the retroreflective member and a diffuse reflective member including the retroreflective member.

(Corresponding Embodiment of the Invention)
The embodiment relating to the present invention corresponds to the above-described fourth embodiment.

(Function)
Based on the color data reflected by the diffusion marker arranged around the retroreflective marker, it is detected where the marker is located in the spatial region imaged by the camera.

(effect)
It is possible to recognize in which spatial coordinates the object (for example, a human) wearing the retroreflection marker is located.

It is a block diagram which shows the structure of the position and orientation measuring apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the example of 1 structure of the marker apparatus 101 of 1st Embodiment. (A), (B) is a figure which shows the structural example of the pattern filter 104. FIG. 6 is a diagram illustrating how the color of the pupil image of the fisheye lens 103 changes depending on the direction in which the imaging devices 10 and 20 capture the marker device 100. FIG. It is a figure for demonstrating the modification of 1st Embodiment. It is a figure for demonstrating the other modification of 1st Embodiment. It is a block diagram which shows the structure of the position and orientation measuring apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of 1 structure of the marker apparatus 101 of 2nd Embodiment. It is a figure for demonstrating the modification of 2nd Embodiment. (A) shows a state where one marker is not within the angle of view of the other marker, and (b) is a diagram showing a state where both markers are within the angle of view of each other. It is a figure for demonstrating the modification of 2nd Embodiment. It is a block diagram which shows the structure of the position and orientation measurement apparatus which concerns on 3rd Embodiment of this invention. It is a figure which expands and shows a part of FIG. It is a figure which shows one structural example which arrange | positions color LED in an array form. It is a figure for demonstrating the modification of 3rd Embodiment. It is a figure for demonstrating the advantage at the time of using the structure of FIG. It is a figure for demonstrating the other modification of 3rd Embodiment. It is a figure which shows an example of the retroreflection marker 350. FIG. It is FIG. (1) for demonstrating the effect of 4th Embodiment. It is FIG. (2) for demonstrating the effect of 4th Embodiment. It is a figure which shows the retroreflective marker 350-1,350-2 having been mounted | worn with the person 361. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Image transmission device 20 Imaging device 21 Image transmission device 100 Marker device 101 Illumination control device 102 Illumination system 103 Fisheye lens 104 Pattern filter 105 Illumination light source 200 Position and orientation calculation device 201 Image reception device 203 Marker storage device 204 Marker detection device 205 Calibration device 206 Pattern storage device 207 Position calculation device 208 Posture identification device 211 Data output device

Claims (8)

A position and orientation measurement apparatus for measuring the position and orientation of an object,
A transmission member on which a coded transmission pattern is formed;
An illumination device that illuminates a transmission pattern of the transmission member;
A projection optical system that projects an illuminated transmission pattern image into space, and a marker device comprising:
An imaging device that captures a transmission pattern image projected by the projection optical system;
A position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device;
A position and orientation measurement apparatus comprising:   The position and orientation measurement apparatus according to claim 1, wherein the projection optical system is a fisheye lens.   The position / orientation measurement apparatus according to claim 1, wherein the projection optical system is an all-around panoramic lens. A position and orientation measurement apparatus for measuring the position and orientation of an object,
A first transmission member formed with a coded transmission pattern;
A first illumination device that illuminates a transmission pattern of the transmission member;
A first projection optical system that projects an illuminated transmission pattern image into space;
A first optical path splitting element provided coaxially with the projection optical system;
A first marker device provided with a first imaging element provided along one of the optical path splitting paths by the optical path splitting element;
A second transmissive member having a coded transmissive pattern formed thereon;
A second illumination device that illuminates the transmission pattern of the transmission member;
A second projection optical system that projects the illuminated transmission pattern image into space;
A second optical path splitting element provided coaxially with the projection optical system;
A second imaging device comprising a second imaging element provided along one of the optical path splitting paths by the optical path splitting element;
A position and orientation measurement apparatus comprising: A position and orientation measurement apparatus for measuring the position and orientation of an object,
A light emitting element array arranged in a code pattern;
A light emission control device for controlling light emission of the light emitting element array;
A projection optical system that projects an array image from the light emitting element array into space, and a marker device,
An imaging device that captures a transmission pattern image projected by the projection optical system;
A position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device;
A position and orientation measurement apparatus comprising: A position and orientation measurement apparatus for measuring the position and orientation of an object,
A transmission member on which a coded transmission pattern is formed;
An illumination device that illuminates a transmission pattern of the transmission member;
A projection optical system that projects an illuminated transmission pattern image into space;
An imaging device provided coaxially with the projection optical system, and a marker device formed of a retroreflective member;
An imaging device that captures a transmission pattern image projected by the projection optical system;
A position and orientation calculation unit that calculates the position and orientation of the marker device based on image data captured by the imaging device;
A position and orientation measurement apparatus comprising:   The position and orientation measurement apparatus according to claim 6, wherein the marker device includes the retroreflective member and a diffuse reflection member including the retroreflective member. A position and orientation measurement method for measuring the position and orientation of an object,
Illuminating the transmission pattern of the transmission member on which the encoded transmission pattern is formed with a lighting device;
Projecting the illuminated transmission pattern image into the space by the projection optical system;
Imaging a transmission pattern image projected by the projection optical system with an imaging device;
Calculating a position and orientation of the marker device based on image data captured by the imaging device by a position and orientation calculation unit;
A position and orientation measurement method comprising:

Images