JP2013117451A - Position attitude estimation mark and position attitude estimation device using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a position attitude estimation mark for suppressing configuration from becoming complicate, and for estimating the position and attitude of an object, and a position attitude estimation device using the same.SOLUTION: A position attitude estimation device 10 includes: a camera 13 for imaging a space including a marker 33; and a PC 14 for estimating the relative position and attitude of the marker 33 to the camera 13 on the basis of a photographic image photographed via the camera 13. A mark consisting of a polygon having four or more vertexes is attached to a predetermined face among faces constituting the marker 33. The PC 14 performs mark extraction determination of determining whether or not the mark has been extracted from the photographic image. Also, when determining that the mark has been extracted, the PC 14 performs position coordinate estimation processing of estimating the relative position and attitude of the mark 33 to the camera 13 on the basis of the position coordinates of each of the vertexes constituting the mark.

Description

  The present invention relates to a position / orientation estimation mark and a position / orientation estimation apparatus using the same.

2. Description of the Related Art Conventionally, there has been provided an apparatus for imaging a subject using an imaging device such as a camera and estimating the position and orientation of the subject in real space.
As such an apparatus, there is an apparatus described in Patent Document 1, for example. In this device, first, a real space including a subject with a marker is imaged by a plurality of imaging devices arranged at predetermined locations. Then, the position and orientation of the marker are estimated based on the difference in the position coordinates of the marker in each captured image. By estimating the position and orientation of the marker in this way, the position and orientation of the subject can be estimated.

JP 2000-298427 A

  By the way, in many cases, the captured image captured by the imaging device captures images of objects other than these subjects and markers as well as the subjects and markers. For example, in the invention described in Patent Document 1, a human head model, a table, and the like to which a dental model is fixed are imaged together with an object. For this reason, there is a possibility that it is difficult to appropriately extract the marker, or that other than the marker is erroneously recognized as the marker.

  In Patent Document 1 described above, a plurality of spherical markers are attached to an object so as to form a predetermined shape, and in order to appropriately extract these markers in a captured image, a color unique to the markers is colored, Lamps that produce unique colors are used. However, when such a configuration is adopted, the configuration of the position / orientation estimation apparatus becomes complicated.

  On the other hand, when the marker configuration is simplified, it is very difficult to appropriately extract the marker from the captured image.

  The present invention has been made to solve the above-described problems of the prior art, and suppresses the complexity of the configuration and can estimate the position and orientation of an object. And a position and orientation estimation apparatus using the same.

  A first aspect of the present invention is made to solve the above-described problem, and is based on an imaging unit that images a space including an object, and a captured image that is captured through the imaging unit. A position / orientation estimation apparatus including an estimation unit that estimates a relative position and orientation with respect to the imaging unit, wherein the object has four or more positions on a predetermined surface among the surfaces constituting the object. A mark composed of a figure having a corner is set, and the estimating means is configured to perform mark extraction determination for determining whether or not the mark composed of the figure is extracted from the photographed image. When it is determined that the mark made of the graphic has been extracted, the relative position and orientation of the object with respect to the imaging means are estimated based on the position coordinates of the corners constituting the mark. It is characterized by performing the estimation process.

  In this configuration, the mark for estimating the relative position and orientation of the object with respect to the imaging means is a figure having four or more corners, and the estimation process is performed by extracting the figure. . In this way, the shape of the mark is a figure having four or more corners that are not easily confused with the shape of the object other than the mark, so it is determined that the object other than the mark erroneously constitutes the mark. It can suppress performing an estimation process. Therefore, it becomes possible to prevent the position and orientation of the object from being estimated or erroneously estimated.

  A second aspect of the present invention is the position / orientation estimation apparatus according to the first aspect, wherein the estimation means extracts a plurality of line segments constituting the mark from the photographed image, and ends the extracted line segments. Dividing each part into a center part and a center part, calculating the intersection of the line segments based on the position of the center part on the image, and setting the intersection of the line segments as the position on the corner image It is characterized by.

  When extracting a mark, the error of the position of the corner on the image often increases. In this respect, according to the same configuration, since the position coordinates of the corners are calculated based on the center part of each line segment, it is possible to suppress occurrence of errors when extracting the positions of the corners on the image. can do.

  A third aspect of the present invention is the position / orientation estimation apparatus according to the first aspect or the second aspect, wherein the object is a polyhedron, and the mark is set on at least two surfaces of the object. It is characterized by being.

  A surface other than the marked surface may be imaged through the imaging means, such as when the object is inclined with respect to the imaging means. In such a case, it is difficult for the estimation means to appropriately extract the mark. In this regard, according to the same configuration, since the mark is set on at least two surfaces of the object, it is possible to prevent the mark from being difficult to be imaged by the imaging unit or being unable to be imaged.

  A fourth aspect of the present invention is a position / orientation estimation apparatus according to any one of the first to third aspects, wherein the position / orientation estimation apparatus includes a head model including a dental model and a treatment used for dental treatment. The object is applied to a dental training apparatus including an instrument, and the object is fixed to the treatment instrument.

  In dental practice, it is difficult for the instructor to directly view the inside of the oral cavity, so the instructor can check in real time whether the instructor is bringing the treatment device into contact with the dental model in an appropriate position and posture. It is difficult to evaluate. In this respect, according to the same configuration, since the position and posture of the treatment instrument can be clarified based on the photographed image, is the practitioner contacting the treatment instrument with the dentition model at an appropriate position and posture? The leader will be able to evaluate in real time whether or not.

  A fifth aspect of the present invention is the position / orientation estimation apparatus according to the fourth aspect, characterized in that the object is fixed to the dentition model.

  In dental practice, the position and posture of the dentition model may be changed during the practice. However, if the position and posture of the dentition model change, the positional relationship between the treatment instrument and the dentition model changes, so it is difficult for the instructor to visually recognize the relationship between the dentition model and the treatment position from the outside. End up. In this regard, according to the same configuration, since the position and posture of the dental model can be clarified in real time, even if the position and posture of the dental model changes during the practice, the instructor It becomes possible to visually recognize the positional relationship between the treatment device and the treatment device.

  A sixth aspect of the present invention is a position / orientation estimation apparatus according to the fourth or fifth aspect, characterized in that the treatment instrument and the dentition model can be displayed as a CG image.

  According to this configuration, the instructor can easily visually recognize the positional relationship between the dentition model and the treatment instrument.

  According to a seventh aspect of the present invention, the relative position and orientation of the object with respect to the imaging unit are estimated based on an imaging unit that captures a space including the object and a captured image captured through the imaging unit. A mark applied to a position / orientation estimation apparatus including an estimation unit, wherein the estimation unit is configured to perform mark extraction determination to determine whether the mark is extracted from the captured image, When it is determined that the mark has been extracted, an estimation process is performed to estimate a relative position and orientation of the target object with respect to the imaging unit based on the position coordinates of each corner constituting the mark. The mark is attached to a predetermined surface among the surfaces constituting the object, and is characterized by a figure having four or more corners.

  According to the present invention, it is possible to provide a position / orientation estimation mark that can suppress the complexity of the configuration and can estimate the position and orientation of an object, and a position / orientation estimation apparatus using the position / orientation estimation mark. it can.

(A) is a general view showing a schematic configuration of a dental training apparatus to which the position / orientation estimation apparatus according to the first embodiment of the present invention is applied, and (b) shows a schematic configuration of a dental model according to the embodiment. Overall view. (A) And (b) is a perspective view which shows schematic structure of the marker and therapeutic device concerning the embodiment. The flowchart which shows the flow of the whole process concerning the embodiment. The flowchart which shows the process sequence of the position and orientation estimation process concerning the embodiment. (A) is the whole figure of the mark displayed on the picked-up image concerning 2nd embodiment of this invention, (b)-(d) is an enlarged view which expands and shows the vertex of a mark. The flowchart which shows the flow of the whole process concerning the embodiment. The flowchart which shows the process sequence of the vertex coordinate calculation process concerning the embodiment. The perspective view which shows schematic structure of the marker and treatment instrument concerning 3rd embodiment of this invention. (A) is a schematic block diagram of the camera, marker, and therapeutic device concerning the embodiment, (b) is the top view of the marker displayed on the picked-up image concerning the embodiment. (A) is a perspective view which shows the whole structure of the marker concerning 4th embodiment of this invention, (b)-(d) is the whole figure of the mark concerning the embodiment. The flowchart which shows the flow of the whole process concerning the embodiment. The whole figure which shows schematic structure of the camera concerning the other embodiment of this invention, a dentition model, and a marker. The whole figure of the mark concerning other embodiments of the present invention.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. In the present embodiment, an example in which the position / orientation estimation apparatus 10 is applied to the dental training apparatus 50 will be described.
First, a dental training apparatus 50 according to the present embodiment will be described with reference to FIG.

  As shown in FIG. 1, the dental training apparatus 50 according to the present embodiment includes a chair 51 on which a trainee is seated, a dental treatment table 52, and a position / orientation estimation apparatus 10. The chair 51 and the dental treatment table 52 can be moved in the vertical direction according to the height of the trainee. The dental treatment table 52 includes a support column 53 that extends to the upper side in the vertical direction at the end of the dental treatment table 52.

Next, the position / orientation estimation apparatus 10 will be described.
As shown in FIG. 1A, a position / orientation estimation apparatus 10 according to this embodiment includes a head model 11, a dental model 12 (not shown) that can be attached to and detached from the head model 11, a head, A camera 13 (corresponding to “imaging means” of the present invention) that can image the part model 11, and a personal computer (hereinafter referred to as “PC”, corresponding to “estimating means” of the present invention) 14 connected to the camera 13; The treatment instrument 31 is configured.

  FIG. 1B shows a schematic configuration of the dentition model 12. As shown in FIG. 1B, the dentition model 12 includes an upper jaw portion 21 and a lower jaw portion 22. The dentition model 12 in this embodiment is attached to the head model 11 with the positions and postures of the upper jaw portion 21 and the lower jaw portion 22 fixed. That is, in the present embodiment, the position and posture of the dentition model 12 are determined in advance to a predetermined position and posture. The dentition model 12 can be appropriately selected by a practitioner according to the treatment technique to be acquired and can be attached to and detached from the head model 11.

The camera 13 is fixed to the column 53 of the dental treatment table 52. The camera 13 is preferably fixed so that the center line of the camera 13 is positioned above the midline of the dentition model 12 so that the dentition model 12 can be uniformly imaged. An image captured by the camera 13 (hereinafter referred to as “captured image”) is output to the PC 14.
Note that internal parameters of the camera 13 (for example, lens distortion, focal length, and camera center position) are input to the PC 14 in advance.

  The PC 14 displays the dentition model 12 on the display surface as a CG image. As described above, in the present embodiment, the position and posture of the dentition model 12 are determined in advance. For this reason, when the practitioner inputs which dentition model 12 has been selected to the PC 14, the PC 14 displays a CG image of the dentition model 12 predetermined according to the selected dentition model 12 on the display surface. indicate.

Next, the marker 33 and the treatment instrument 31 will be described with reference to FIG.
As shown in FIGS. 2A and 2B, a marker 33 (corresponding to the “object” of the present invention) is fixed to the proximal end portion of the treatment instrument 31. Thus, since the marker 33 is fixed with respect to the treatment instrument 31, the position and posture of the distal end portion of the treatment instrument 31 can be determined based on the position and posture of the marker 33.

  Of the surfaces constituting the marker 33, a mark 32 made of a closed star-shaped polygon (corresponding to the “figure” of the present invention) is attached to the surface opposite to the surface in contact with the treatment instrument 31. . The shape of the mark 32 is preferably a regular polygon in order to reduce the calculation amount of the PC 14.

  A reference line S for identifying the vertices constituting the mark 32 is attached to the mark 32. This reference line S is a line segment connecting the center point of the mark 32 and one vertex (reference vertex) among the vertices constituting the mark 32. The center point is a coordinate obtained by averaging the coordinates of the vertices constituting each mark 32.

As described above, the mark 32 is a regular polygon. For this reason, when the reference line S is not attached, when the marker 33 is rotated around the center point of the mark 32, the shape of the mark 32 displayed in the captured image before and after the rotation is as follows. Even if it is the same shape, the direction of the therapeutic instrument 31 may be different. That is, if each vertex cannot be identified, the posture of the treatment instrument 31 may be different even if the shape of the mark 32 displayed in the captured image is the same.
Therefore, in this embodiment, the reference vertex can be identified from other vertices by adding the reference line S. As described above, when the reference vertex can be identified from other vertices, each vertex constituting the mark 32 can be identified.

  The PC 14 estimates the position and orientation of the treatment instrument 31 based on the mark 32 displayed on the captured image, and displays the CG image (hereinafter referred to as “tool CG model”) on the display surface together with the dentition model 12.

  By the way, in the prior art, when estimating the position and orientation of the marker by imaging the marker with a camera, generally, a method of imaging the marker with a plurality of cameras fixed at a predetermined position is used. It is done.

In this method, the positional relationship of each camera in the three-dimensional space (for example, the distance between the cameras) and the coordinate system of at least one camera (hereinafter referred to as “camera coordinate system”, the viewpoint position when imaged from the center of the camera, and A coordinate system determined by the orientation of the viewpoint is obtained in advance.
The PC extracts a mark from each captured image output from each camera, and obtains the position coordinates of the mark in each captured image. After that, based on the difference in the position coordinates of the mark in each captured image and the positional relationship of each camera in the three-dimensional space, the position and orientation of the mark in the camera coordinate system, and thus the position and orientation of the marker are determined. Ask.

On the other hand, in the present invention, the mark 32 is detected using the single camera 13 and the coordinate system (marker coordinate system) of the marker 33 is estimated.
That is, as shown in FIG. 2A, the PC 14 first has a point in the vicinity of the mark 32 as an origin, an x axis parallel to one side of the marker 33, a y axis orthogonal to the x axis, and the x axis. And a coordinate system (marker coordinate system) composed of the z axis orthogonal to the y axis is set. Then, the PC 14 calculates in advance the coordinates (Xi, Yi, Zi) of each vertex constituting the mark 32 in the marker coordinate system. Since the mark 32 is attached to the marker 33, Zi is “0”.

  When the position / orientation estimation apparatus 10 is activated and a captured image captured by the camera 13 is input to the PC 14, the PC 14 extracts a mark 32 from the captured image. Then, the coordinates (Ui, Vi) on the captured image of each vertex (corresponding to the “corner” of the present invention) constituting the extracted mark 32 are calculated.

Next, the PC 14 associates the coordinates (Ui, Vi) of each vertex with the coordinates (Xi, Yi, Zi) of each vertex of the mark 32 calculated in advance by homography. Thereby, the position and orientation of the marker coordinate system in the camera coordinate system can be estimated.
For example, when the marker 33 is tilted with respect to the camera 13 as shown in FIG. 2B, the marker coordinate system is positioned according to the position and orientation of the marker 33 as shown in FIG. And attitude.

  By calculating the marker coordinate system in the camera coordinate system in this way, the PC 14 can calculate the position and orientation of the marker 33 in the camera coordinate system, and thus the position and orientation of the treatment instrument 31 in the camera coordinate system.

Next, the overall flow of processing according to the present embodiment will be described with reference to FIG.
As shown in FIG. 3, the PC 14 first acquires a captured image from the camera 13 (step S101).
Then, position / orientation estimation processing (corresponding to “estimation processing” of the present invention) for estimating the position and orientation of the treatment instrument 31 is performed (step S102), and the tool CG model that is a CG image of the treatment instrument 31 described above is stored in the PC 14. Display on the display surface (step S103).

Next, the position / orientation estimation apparatus will be described in detail with reference to FIG.
As shown in FIG. 4, the PC 14 first performs binarization processing on the captured image (step S201).

Next, a contour point sequence is extracted from the binarized captured image (step S202), and a mark 32 is extracted from the extracted contour point sequence (step S203).
As described above, the mark 32 is formed of a star-shaped closed regular polygon. In this embodiment, after extracting a contour point sequence, a vertex sequence constituting the mark 32 is extracted using a vertex extraction algorithm by Douglas Poker, which is a method for simplifying an approximate curve.

  The PC 14 calculates the number of vertices of the extracted polygon based on the contour of the extracted mark 32, and determines whether or not this number is the same as the number of vertices of the mark 32 (“ Equivalent to “mark extraction judgment”). Then, when the number of vertices of the extracted polygon is the same as the number of vertices of the mark 32 stored in advance, it is determined that the mark 32 has been extracted from the contour point sequence.

  Next, the PC 14 identifies each vertex constituting the mark 32 based on the reference line S. Then, a marker coordinate system in the camera coordinate system is calculated based on the coordinates on the captured image of each vertex constituting the mark 32 (step S204). Since the calculation method of the marker coordinate system has been described above, a detailed description thereof is omitted here.

  Then, the PC 14 estimates the position and posture of the treatment instrument 31 based on the calculated marker coordinate system (step S205).

  According to the tool CG model according to the present embodiment, since the position and posture of the treatment instrument 31 can be converted into CG, the positions of the dentition model 12 and the treatment instrument 31 in the oral cavity that are difficult to visually recognize directly. The relationship can be clarified.

According to the present embodiment, the following effects can be obtained.
(1) In conventional dental practice, since it is difficult for the instructor to directly view the inside of the oral cavity, whether or not the instructor makes the treatment instrument 31 contact the dental model 12 at an appropriate position and posture. It was difficult for leaders to evaluate in real time. In this regard, according to the present embodiment, since the position and posture of the treatment instrument 31 can be clarified based on the captured image, the practitioner touches the dentition model 12 with the treatment instrument 31 at an appropriate position and posture. Instructors will be able to evaluate in real time whether or not they are in contact.

  (2) Since the dentition model 12 and the treatment instrument 31 are displayed as CG images, the instructor visually recognizes whether the practitioner makes the treatment instrument 31 contact the dentition model 12 in an appropriate position and posture. It becomes easy to do.

  (3) Since the shape of the mark 32 is a star-shaped closed regular polygon that is difficult to be confused with the shape other than the mark 32, it is determined that the mark 32 is mistakenly constituted by the mark 32. Therefore, it is possible to suppress the position / orientation estimation processing based on this. Accordingly, it is possible to suppress erroneous estimation of the position and orientation of the marker 33.

  (5) Further, the PC 14 estimates the position and orientation of the marker coordinate system in the camera coordinate system by associating the coordinates of each vertex extracted from the captured image with the spatial coordinates of each vertex of the mark 32 by homography. I have to. For this reason, it becomes possible to estimate the position and orientation of the marker 33 by using a single camera 13 without using a plurality of cameras 13. For this reason, the configuration of the position / orientation estimation apparatus 10 can be simplified.

  (6) Moreover, since each vertex which comprises the mark 32 can be identified with the reference line S, it becomes possible to suppress erroneously estimating the posture of the treatment instrument 31.

(Second embodiment)
Hereinafter, a second embodiment of the present invention will be described based on FIGS. 5 to 7 with a focus on differences from the first embodiment. In addition, in these FIGS. 5-7, the code | symbol same about each element same as each element demonstrated in 1st embodiment is each shown, and the overlapping description about these elements is omitted.

As described in the first embodiment, the PC 14 extracts the coordinates (Ui, Vi) on the captured image of each vertex constituting the mark 32 based on the captured image subjected to the binarization process. ing.
Incidentally, the PC 14 displays a photographed image in units of pixels. For this reason, each vertex of the mark 32 extracted by the PC 14 may be displayed at slightly shifted coordinates. In addition, when the binarized processing is performed on the captured image as described above, such a shift may be increased.

Furthermore, since noise may remain in the image when binarization processing is performed, noise removal processing such as smoothing processing may be performed in advance when performing binarization processing. Even with such a smoothing process, the coordinates of the vertices of the mark 32 may change in the same manner.
Therefore, in the present embodiment, the PC 14 performs vertex coordinate calculation processing.

Hereinafter, the overall flow of processing according to the present embodiment will be described with reference to FIG.
As shown in FIG. 6, when this process is started, the PC 14 first performs a binarization process on the captured image (step S301).

For example, as shown in FIG. 5B and FIG. 5C, among the pixels constituting the captured image, a pixel having a luminance equal to or higher than a predetermined threshold is a black pixel and has a luminance lower than the predetermined threshold. Convert pixels to white pixels.
Note that, as shown in FIGS. 5B and 5C, the pixels near the vertex often have a small value, and can be converted into white pixels by such binarization processing. Many.

  Next, a contour point sequence is extracted from the binarized captured image (step S302, see FIG. 5D), and a mark 32 is extracted from the contour point sequence (step S303). Since the method for extracting the mark 32 has been described in the first embodiment, a detailed description is omitted.

Thereafter, the PC 14 performs a vertex coordinate calculation process (step S304).
After performing the vertex coordinate calculation process (step S304), the PC 14 calculates a marker coordinate system in the camera coordinate system (step S305), and estimates the position and orientation of the treatment instrument 31 based on this marker coordinate system ( Step S306), this process is temporarily terminated.

Next, the vertex coordinate calculation process will be described in detail with reference to FIG.
As shown in FIG. 7, when the vertex coordinate calculation process is started, the PC 14 first extracts the point sequence for each extracted side (corresponding to “each line segment” of the present invention, the points shown in FIG. 5D). Column) is acquired (step S401).

  Next, the number of point sequences of the sides is calculated from the sequence of points constituting each side. After that, among each point sequence, a point sequence (corresponding to the “end portion” of the present invention) in which the number of point sequences from the end points is 5 to 45% (preferably 10%) of the number of point sequences on the side is deleted. (Step S402). Note that the end point indicates a point that constitutes a vertex among each point sequence that constitutes the mark 32.

  Next, the PC 14 determines each side constituting the polygon based on the point sequence (corresponding to the “central portion” in the present invention) obtained by deleting the point sequence that is 5 to 45% (preferably 10%) from the end point. (Step S403), the intersection of adjacent sides (point A shown in FIG. 5D) is calculated, and the coordinates of this intersection are used as the coordinates of the vertex (step S404). Exit once.

As shown in FIG. 5B, the PC 14 displays a captured image in units of pixels. For this reason, each vertex of the mark 32 extracted by the PC 14 may be displayed at slightly shifted coordinates (for example, coordinates (Ui, Vi) shown in FIG. 5B).
In particular, when the resolution of the camera 13 or the PC 14 is low, such a shift may increase.
Moreover, such a shift may be further increased by performing binarization processing.

  However, according to the present embodiment, as shown in FIG. 5D, vertices based on a point sequence in which a point sequence in the range of 5 to 45% (preferably 10%) is deleted from the end points (FIG. 5D). Since the coordinates of the point A) shown in 5 (d) are calculated, it is possible to prevent the vertex coordinates from being shifted due to the binarization processing or the like as described above.

According to the second embodiment described above, the following functions and effects can be achieved in addition to the functions and effects (1) to (5) described above.
(6) According to the present embodiment, among the point sequences constituting each side, a point sequence in which the number of point sequences from the end points is within 5 to 45% (preferably 10%) of the number of point sequences of the side is selected. Since the coordinates of the vertices are calculated based on the deleted point sequence, it is possible to suppress the extracted vertex coordinates from being different from the actually captured coordinates.

(Third embodiment)
Hereinafter, a third embodiment of the present invention will be described based on FIGS. 8 and 9 with a focus on differences from the first embodiment. In FIGS. 8 and 9, the same elements as those described in the first embodiment are denoted by the same reference numerals, and redundant descriptions of these elements are omitted.

In the first embodiment described above, the mark 32 is attached to only one surface of the marker 33. On the other hand, in the present embodiment, as shown in FIG. 8, the marker 33 is a cube, and the mark 32 is attached to all the surfaces constituting the marker 33.
As shown in FIG. 8, these marks 32 have different shapes. Each is a closed polygon with four or more vertices.

  In the present embodiment, as shown in FIG. 8, the PC 14 first has a point in the vicinity of the mark 32 attached to the surface F1 as an origin, an x axis parallel to one side of the surface F1, and orthogonal to the x axis. A marker coordinate system consisting of a y-axis and a z-axis orthogonal to the x-axis and the y-axis is set. Then, the PC 14 calculates in advance the coordinates (Xi, Yi, Zi) of each vertex constituting the mark 32 in the marker coordinate system. The Zi of the mark 32 attached to the surface F1 is “0”.

  As described above, when the position / orientation estimation apparatus 10 is activated and a captured image captured by the camera 13 is input to the PC 14, the PC 14 extracts each mark 32 from the captured image. At this time, the PC 14 identifies the vertex of each mark 32 based on the reference line S. Then, the coordinates (Ui, Vi) on the captured image of each vertex constituting each extracted mark 32 are calculated.

Next, the PC 14 associates the coordinates (Ui, Vi) of each vertex with the coordinates (Xi, Yi, Zi) of each vertex of each mark 32 calculated in advance.
In the first embodiment described above, since the mark 32 is attached to only one surface, the coordinates (Ui, Vi) and the coordinates (Xi, Yi, Zi) are associated using homography. On the other hand, in this embodiment, without using homography, coordinates (Ui, Vi) and coordinates (Xi, Yi, Zi) are associated with each other, and the position and orientation of the marker coordinate system in the camera coordinate system are determined. Ask.

  In the present embodiment, the PC 14 recognizes which surface is detected based on the number of extracted polygonal vertices, and then coordinates of the vertex (Ui) of the mark 32 on the detected surface. , Vi) and the coordinates (Xi, Yi, Zi) of each vertex of the mark 32 are associated with each other.

  For example, when the treatment instrument 31 is tilted with respect to the camera 13 as shown in FIG. 9A, the surfaces F1 and F2 among the surfaces constituting the marker 33 as shown in FIG. 9B. Are displayed on the captured image.

At this time, the PC 14 extracts a mark 32 made of a regular hexagon and a mark 32 made of a regular heptagon from the photographed image. The PC 14 recognizes that the surface with the regular hexagon is the surface F1, and the surface with the regular heptagon is the surface F2.
Then, by associating the coordinates (Xi, Yi, Zi) in the marker coordinate system of each vertex constituting each polygon with the coordinates (Ui, Vi) in the captured image of the vertex, the marker coordinate system in the camera coordinate system Is calculated.

According to the present embodiment, in addition to the operational effects (1) to (5) described above, the following operational effects can be achieved.
(7) According to the present embodiment, since the mark 32 is attached to all the surfaces constituting the marker 33, any mark is present even when the marker 33 is largely inclined with respect to the camera 13. 32 will be imaged. For this reason, it becomes possible to prevent the position and orientation of the marker 33 from being estimated or difficult to estimate by the mark 32 not being displayed in the captured image.

  (8) Further, since the position and orientation of the marker 33 can be estimated based on the marks 32 attached to the plurality of surfaces, the position and orientation of the marker 33 can be estimated more accurately.

(Fourth embodiment)
Hereinafter, a fourth embodiment embodying the present invention will be described based on FIG. 10 and FIG. 11 with a focus on differences from the third embodiment. 10 and 11, the same elements as those described in the third embodiment are denoted by the same reference numerals, and redundant descriptions of these elements are omitted.

  In the third embodiment described above, polygons having different shapes are attached to each surface of the marker 33. On the other hand, in this embodiment, as shown in FIG. 10A, marks 32 each having a regular hexagon are attached to each surface of the marker 33.

  Further, as shown in FIGS. 10A and 10B, a reference line L1 connecting the center point C1 and the vertex V1 is attached to the mark 32 attached to the surface F1. Also, as shown in FIGS. 10A and 10C, the mark 32 attached to the surface F2 includes a reference line L2 connecting the center point C2 and the vertex V7, and a center point C2 and the vertex V8. A connecting reference line L3 is attached. Further, as shown in FIGS. 10A and 10D, the mark 32 attached to the surface F3 includes a reference line L4 connecting the center point C3 and the vertex V13, and the center point C3 and the vertex V14. A connecting reference line L5 and a reference line L6 connecting the center point C3 and the vertex V15 are attached.

In the present embodiment, the PC 14 identifies the reference vertex of each mark 32 based on the line segment attached between the center point and the vertex, and the mark 32 with the extracted mark 32 attached to any surface. It is made to recognize whether it is.
Hereinafter, the flow of the entire processing according to the embodiment will be described with reference to FIG.

  As shown in FIG. 11, when this process is started, the PC 14 first performs binarization processing on the captured image (step S501), and contour points are obtained from the binarized captured image. A column is extracted (step S502). Next, the PC 14 extracts a mark 32 from the contour point sequence (step S503), and calculates a center point for each extracted mark 32 (step S504).

For example, when the mark 32 attached to F1 (see FIG. 10B) is extracted and the coordinates of the center point C1 are calculated based on the vertices V1 to V6 (step S504), the vertices V1 to V1 constituting the mark 32 are calculated. Of the line segments connecting V6 and the center point C1, only the reference line L1 is displayed as a black line segment.
Also, for example, when the mark 32 (see FIG. 10C) attached to F2 is extracted and the coordinates of the center point C2 are calculated based on the vertices V7 to V12 (step S504), each vertex constituting the mark 32 Of the line segments connecting V7 to V12 and the center point C2, only the reference line L2 and the reference line L3 are displayed as black line segments.

  Therefore, the PC 14 calculates a point sequence that connects the vertices constituting the mark 32 and the center point (step S505), and points that are composed of black pixels in the point sequence that connects the vertices and the center point. A point sequence including a predetermined ratio or more is set as a reference line (step S506).

  Based on the reference line, the reference vertex and the mark 32 are identified (step S507).

  Thereafter, the PC 14 calculates the position and orientation of the marker coordinate system in the camera coordinate system (step S508), and estimates the position and orientation of the treatment instrument 31 (step S509).

According to the said embodiment, in addition to the effect (1)-(5), (7) and (8) mentioned above, the following effects can be show | played.
(9) According to this embodiment, the marks 32 having the same shape can be distinguished from each other. For this reason, in estimating the position and orientation of the marker 33, it is possible to suppress erroneous estimation of the position and orientation of the marker 33 by confusing the marks 32 with each other.

(Other embodiments)
Note that the position / orientation estimation apparatus 10 according to the present invention is not limited to the configuration illustrated in the above embodiment, and can be implemented as, for example, the following form in which the embodiment is appropriately changed. The following modifications are not applied only to the above-described embodiment, and different modifications can be combined with each other.

  In each of the above embodiments, the position and posture of the dentition model 12 are fixed. However, the present invention is not limited to this, and the positions of the upper jaw portion 21 and the lower jaw portion 22 constituting the dentition model 12. And the posture may be changed. In this case, as shown in FIG. 12, markers 33 are fixed to the upper jaw portion 21 and the lower jaw portion 22, the markers 33 are imaged by the camera 13, and the upper jaw portion is based on the position and posture of the marker 33. It is preferable to estimate the positions and postures of 21 and the lower jaw 22.

  In dental practice, the position and posture of the dentition model 12 may be changed during the practice. According to this modification, since the position and posture of the dentition model 12 can be clarified in real time, even if the position and posture of the dentition model 12 changes during the practice, the instructor can The positional relationship between 12 and the treatment instrument 31 can be visually recognized.

  In the above modification, the positions and postures of the upper jaw portion 21 and the lower jaw portion 22 can be changed. However, the present invention is not limited to this, and the position and posture of the lower jaw portion 22 are fixed. Only the position and posture of the upper jaw 21 may be changed. In this case, the marker 33 may be fixed only to the upper jaw part 21, and the marker 33 may not be attached to the lower jaw part 22.

  Alternatively, the position and posture of the upper jaw portion 21 may be fixed and only the position and posture of the lower jaw portion 22 may be changed. In this case, the marker 33 may be fixed only to the lower jaw part 22 and the marker 33 may not be attached to the upper jaw part 21.

  -In 3rd Embodiment and 4th Embodiment mentioned above, as demonstrated in 2nd Embodiment mentioned above, among the point sequences which comprise each side of the mark 32, the number of point sequences from an end point is a point sequence. The equation of each side may be calculated based on the point sequence from which the point sequence in 4 to 45% (preferably 10%) of the number is deleted, and the coordinates of the intersection of each side may be used as the vertex coordinates.

  In the third embodiment described above, the marks 32 are attached to all surfaces constituting the marker 33. However, the present invention is not limited to this, and the marks 32 are attached only to two surfaces of the marker 33. It may be. Further, the mark 32 may be attached to only three surfaces of the marker 33. Furthermore, the mark 32 may be attached only to the four surfaces of the marker 33.

  In the second embodiment, the point sequence whose end point number is 4 to 45% (preferably 10%) of the side point sequence number is deleted, but the present invention is not limited to this. The coordinates of the vertices may be calculated again using all the extracted point sequences.

  In the third embodiment, the shape of the marker 33 is a cube, but the shape of the marker 33 in the present invention is not limited to a cube, and may be another polyhedron.

  In the above embodiments and modifications, the world coordinate system (the coordinate system representing the position and orientation of the camera 13) is matched with the camera coordinate system, but the world coordinate system in the present invention is limited to the camera coordinate system. Rather, it can be determined arbitrarily.

  In each of the above embodiments and modifications, the origin of the marker coordinate system is set to the plane F1, but the marker coordinate system in the present invention is not limited to this, and is integrated with the marker 33 and the marker 33. It can be arbitrarily determined in the treatment instrument 31.

  In each of the above embodiments and modifications, the shape of the mark 32 is a star-shaped regular polygon or pentagon, or a hexagon or heptagon, but the shape of the mark 32 in the present invention is not limited to these. Any polygon that has four or more vertices may be used.

  In each of the above-described embodiments and modifications, the mark 32 is made of a straight line having a predetermined thickness. However, the mark 32 of the present invention is not limited to this, and the PC 14 has the outermost point sequence. Anything that can be extracted may be used. For example, the inside of the mark 32 may be filled with the same color as the outer contour of the mark 32.

  In each of the above embodiments and modifications, the direction of the mark 32 is identified by identifying the reference vertex using the reference line S, the reference line L1, the reference line L2, or the reference line L3. The invention is not limited to this. For example, as shown in FIG. 13, even when the marker 33 is rotated, the mark 32 having a shape that does not have the same shape after the rotation and before the rotation may be used. According to this modification, the direction of the mark 32 can be identified without using the reference line.

  In each of the above embodiments and modifications, the reference line S is used to identify the reference vertex, but the reference vertex identifying method in the present invention is not limited to this. For example, a part of the mark 32 may be filled.

  In each of the above embodiments and modifications, the mark 32 is directly attached to the marker 33. However, the present invention is not limited to this, and the mark 32 is set at a position away from the marker 33. Also good. It is only necessary that the position and coordinates of the marker 33 can be determined based on the position and coordinates of the mark 32.

  In each of the above-described embodiments and modifications, the position and orientation of the treatment instrument 31 are displayed on the display surface of the PC 14 using the tool CG model. However, the method for displaying the position and orientation of the treatment instrument 31 in the present invention is as follows. The method is not limited to the CG model, and other methods may be used. For example, it may be displayed as an animation or a simple model.

  In each of the above embodiments and each modification, the example in which the position / orientation estimation apparatus 10 is applied to the dental training apparatus 50 has been described. However, the position / orientation estimation apparatus 10 according to the present invention is not limited thereto, and may be an AV device, It can be applied to other devices such as game devices.

DESCRIPTION OF SYMBOLS 10 ... Position and orientation estimation apparatus 11 ... Head model 12 ... Dental row model 13 ... Camera 14 ... Personal computer (PC)
DESCRIPTION OF SYMBOLS 21 ... Upper jaw part 22 ... Lower jaw part 31 ... Treatment instrument 32 ... Mark 33 ... Marker 50 ... Dental practice apparatus 51 ... Chair 52 ... Dental treatment table 53 ... Post

Claims (7)

Position and orientation estimation comprising imaging means for imaging a space including an object, and estimation means for estimating a relative position and orientation of the object with respect to the imaging means based on a captured image captured through the imaging means A device,
The object is set with a mark made of a figure having four or more corners on a predetermined surface among the surfaces constituting the object,
The estimation means is configured to perform a mark extraction determination for determining whether or not a mark made of the graphic is extracted from the photographed image,
Performing an estimation process of estimating a relative position and orientation of the object with respect to the imaging unit based on the position coordinates of each corner constituting the mark when it is determined that the mark made of the graphic has been extracted. A position and orientation estimation device characterized by the above. The position and orientation estimation apparatus according to claim 1,
The estimation means extracts a plurality of line segments constituting the mark from the photographed image, divides each extracted line segment into an end part and a central part, and based on the position coordinates of the central part A position / orientation estimation apparatus characterized in that an intersection between line segments is calculated and the intersection between the line segments is used as a position coordinate of the corner. The position and orientation estimation apparatus according to claim 1 or 2,
The object is a polyhedron,
The position and orientation estimation device, wherein the mark is set on at least two surfaces of the object. The position and orientation estimation apparatus according to any one of claims 1 to 3,
The position and orientation estimation device is applied to a dental training device including a head model including a dentition model and a treatment instrument used for dental treatment,
The target object is fixed to the treatment instrument. The position and orientation estimation apparatus according to claim 4,
The object is fixed to the dentition model. A position and orientation estimation apparatus, wherein: The position and orientation estimation apparatus according to claim 4 and 5,
The position and orientation estimation apparatus characterized in that the treatment instrument and the dentition model can be displayed as a CG image. Position and orientation estimation comprising imaging means for imaging a space including an object, and estimation means for estimating a relative position and orientation of the object with respect to the imaging means based on a captured image captured through the imaging means A mark applied to the device,
The estimation means is configured to perform mark extraction determination for determining whether or not the mark has been extracted from the captured image,
When it is determined that the mark has been extracted, an estimation process for estimating a relative position and orientation of the target object with respect to the imaging unit is performed based on the position coordinates of each corner portion constituting the mark,
The mark is a mark that is attached to a predetermined surface among the surfaces that constitute the object and includes a figure having four or more corners.

Images