JP2008276743A - Method and system for scanning surface and preparing three-dimensional object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a device for improving accuracy in mapping an object. <P>SOLUTION: An image is projected on the surface of a three dimensional object 611, and the image can include a pattern including a plurality of individual shapes used to measure and map the surface. The image is further provided with a feature part containing an encoded information for identifying the plurality of shapes individually. The feature part containing the encoded information is oriented to retrieve the encoded information along a line perpendicular to a plane formed by points along a projection axis and a visual line (613). The feature part is used to execute multiframe independent scanning (612). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

Detailed Description of the Invention
(Related application)
No. 09 / 560,131, filed Apr. 28, 2000, U.S. Patent Application No. 09 / 560,132, filed Apr. 28, 2000, Apr. 28, 2000. US patent application 09 / 560,583 filed on the same day, US patent application 09 / 616,093 filed on July 13, 2000, US patent application filed on April 28, 2000 No. 09 / 560,645, U.S. Patent Application No. 09 / 560,644, filed April 28, 2000, U.S. Patent Application No. 09 / 560,584, filed Apr. 28, 2000, and No. 09 / 560,133, filed Apr. 28, 2000, the priority of which is hereby incorporated by reference in its entirety.

(Field of Invention)
The present invention is primarily concerned with object mapping, and more specifically, creating a three-dimensional model of an object, creating a model of an object by aligning portions of the object, and scanned portions of the object To provide 3D objects, scan and treat and diagnose anatomical structures, develop and manufacture medical and dental devices and instruments, and provide specific images Assisting with object mapping.

(Background of the Invention)
It is well known that anatomical devices such as prosthetics and orthodontics and instruments such as orthodontics can be made. Current methods of making anatomical devices are personal, so that the physician can remember his views on his anatomical structure, where the device is used, and his experience and similar circumstances. Specify or design anatomical devices based on personal criteria such as Such personal criteria will result in the development of anatomical devices that can vary greatly by the physician and prevent obtaining a database of knowledge that can be used by others.
Attempts to develop anatomical devices that are not very personal include taking the form of anatomical structures. From a mold that is a female representation of an anatomical structure, a male model of the anatomical structure can be created. However, molds and models made from molds are prone to distortion, wear and damage, have short expiration dates, are inaccurate, have additional costs to make multiple copies, and accuracy is easy It cannot be verified. Therefore, it cannot be easily verified whether the type or model of the structure is an authentic representation of the anatomical structure. Furthermore, the mold processing is generally unpleasant and inconvenient for the patient, has to go to the doctor and is time consuming. Furthermore, if multiple models are needed, either multiple types must be created, or multiple models must be created from a single type. In any case, there is no reliable standard reference that guarantees the similarity of each of the models. Furthermore, the model still has to be judged visually by one doctor, resulting in personal processing.

  Another attempt to develop less personally involves using two-dimensional images. However, using two-dimensional images, as is well known, cannot provide precise structure information and must still be objectively interpreted by a physician. Furthermore, the manufacture of the device still relies on an objective interpretation.

  When the mold is transported from the doctor to the manufacturing facility, the 3D model used to design the prosthetic device is only available at the manufacturing facility and therefore belongs to the model or device being manufactured Communication between the doctor and technician regarding the problem is hindered. Even when there are multiple models, they are physically separate objects and cannot be viewed simultaneously from the same viewpoint because there is no interactive way of referring to the multiple models.

  In addition to molds and molds, other types of records maintained by doctors such as dentists and orthodontists are easily lost or damaged and are expensive to make duplicates. Therefore, methods or systems that overcome these disadvantages are useful.

  The use of scanning techniques to map the surface of an object is well known. Prior art FIG. 1 illustrates an object 100 having visible surfaces 101-104. In general, the visible surfaces 101 to 103 form a rectangular shape and lie on a substantially flat surface 104.

  An image is projected onto the object 100, which includes a line 110. In operation, an image of line 110 is received by an observation device such as a camera (not shown) and processed to determine the shape of that portion of object 100 where line 110 is. By moving the line 110 across the object 100, the entire object 100 can be mapped. A limitation associated with using an image with a single line 110 is that it takes a great deal of time to scan the object 100 to provide an accurate map, and it is a fixed reference to either the scanner or the object. The point is necessary.

  FIG. 2 illustrates a prior art solution that reduces the amount of time it takes to scan an object. Specifically, FIG. 2 illustrates an image that includes lines 121-125. By providing multiple lines, it is possible to scan a larger surface area at a time, thus allowing more efficient processing of data associated with the object 100. The limitations of using a pattern as illustrated in FIG. 2 are mapped due to the need for fixed reference points and possible improper processing of data due to overlapping discontinuities in the image. It is possible that the surface resolution that can be reduced.

  To better understand the concept of overlap, it is useful to understand the scanning process. Prior art FIG. 3 illustrates the shape of FIGS. 1 and 2 from the side so that only the surface 102 is visible. For consideration, a projection device (not shown) projects the pattern in a direction perpendicular to the surface 101 that forms the top edge of the surface 102 of FIG. The point from the center of the projection lens to the surface is called the projection axis, the rotation axis of the projection lens or the center line of the projection lens. Similarly, an imaginary line from the center point of the observation device (not shown) is called a visual axis, a rotation axis of the observation device, or a center line of the observation device, and extends in a direction in which the observation device is oriented.

  The physical relationship between the projection axis and the visual axis relative to each other is almost known. In the particular illustration of FIG. 3, the projection axis and the viewing axis are in a common plane. The relationship between the projection system and the observation system is physically calibrated so that the relationship between the projector and the observation device is known. Note that the term “reference (criteria or reference) point” describes a “reference (criteria or reference)” from which a third party such as a reader views an image. For example, in FIG. 2, the reference point is above and on the side formed by the surfaces 101, 102, and 103.

  FIG. 4 illustrates the object 100 with the projected image of FIG. 2 (projection view) when the reference point (reference point) is equal to the projection angle. When the reference point is equal to the projection angle, there is no break in the projected image. In other words, the lines 121 to 125 appear straight on the object 100. However, when the reference point is equal to the projection angle, the line is not distorted, so that data useful for mapping the object cannot be obtained.

FIG. 5 illustrates the object 100 from a reference point equal to the viewing angle, which is the fleet of FIG.
In FIG. 5, the surfaces 104, 103 and 101 are visible because the visual axis is substantially perpendicular to the line formed by the surfaces 101 and 103 and to the right of the plane formed by the surface 102. is there. Accordingly, the surface 102 is not illustrated in FIG. See FIG. Due to the angle at which the image is viewed or received by the viewing device, lines 121 and 122 appear as a single continuous straight line.
Similarly, line pairs 122 and 123, and 123 and 124 are matched to give a type that is a single sequence. Because line 125 is projected onto surface 104 at a single level of surface height, line 125 is a continuous single line.

  When the pattern of FIG. 5 is received by the processing device and performing the mapping function, line pairs 121 and 122, 122 and 123, and 123 and 124 are misinterpreted as being a single line. As a result, the two-stage object illustrated in FIG. 2 may actually be mapped as a single horizontal surface, or otherwise the processing step will distinguish between line pairs. Can not be displayed correctly.

  FIG. 6 illustrates a prior art solution that overcomes the problem described in FIG. Specifically, FIG. 6 illustrates a shape 100 having an image (projection view) projected onto it, whereby a plurality of lines having different line widths or thicknesses are used. FIG. 7 illustrates the pattern of FIG. 6 from the same reference point (reference point) as FIG.

  As illustrated in FIG. 7, a processing element that analyzes the received data can distinguish between pairs of lines that were previously indistinguishable. Referring to FIG. 7, line 421 is still in line with line 422, forming what appears to be a continuous line. However, because line 421 and line 425 have different thicknesses, it is possible to determine the exact identification of a particular line segment by analyzing the image. In other words, by analyzing the received image, it can be determined that the line 422 projected onto the surface 104 and the line 422 projected onto the surface 101 are actually common lines. Using this information, by analyzing the received image, it can be determined that a step type feature is present in the object being scanned, resulting in a mismatch between the two segments of line 422. .

  As illustrated in FIG. 7, the use of varying line thickness helps to identify line segments, but an object having the illustrated type of variation feature is included in the received image. There may still be errors during the analysis.

  FIG. 8 illustrates a structure having a surface 710 with sharply varying features from a side reference point. Surface 710 is illustrated as being substantially perpendicular to the reference point in FIG. In addition, the object 700 has side surfaces 713 and 715 and top surfaces 711 and 712. From the point of reference in FIG. 8, the actual surfaces 711, 712, 713 and 715 are not visible and only their edges are represented. Surface 711 is a relatively steeply inclined surface, while surface 712 is a relatively gentle inclined surface.

  Further illustrated in FIG. 8 are three projected lines 721-723 having various widths. The first line 721 has a width of 4. The second projected line 722 has a width of one. The third projected line 723 has a width of 8.

  Line 721 has a width of 4 and is projected onto a relatively flat surface 714. Due to the angle between the projection axis and the viewing axis, the width of the actual line 721 seen on the flat surface 714 is approximately two. If lines 722 and 723 are also projected onto a relatively flat surface 714, their respective widths are approximately the same as the width of 721 so that the thickness can be detected during the analysis step of mapping the surface. Fluctuates by the percentage amount. However, since the line 722 is projected onto the inclined surface 711, the viewpoint from the observation device along the visual axis is such that the line 722 appears to have a width of two.

  Due to the steep angle of surface 710, line 722 appears to have a width of 2 because a larger portion of projected line 722 is projected onto a larger area of surface 711. It is this large area of the surface 722 being viewed that gives the recognition that the projected line 722 has a width of two.

  In the opposite manner that line 722 is affected by surface 711, line 723 is affected by surface 712, giving the realization that projected line 723 having a width of 8 actually has a width of 2. This occurs because the angle of the surface 712 relative to the viewing device causes the surface area with the projected line 723 to appear to have a width of two. The result of this phenomenon is further illustrated in FIG.

  FIG. 9 illustrates the shape 700 of FIG. 8 in terms of a visual axis reference. Lines 721 to 723 are projected onto the surface 714 so that the difference between the line thicknesses can be easily determined from the reference point of the visual axis. Thus, when the surface area 714 is analyzed, the lines can be easily identified based on the viewed image. However, when the analysis includes surfaces 711 and 712, not only are the widths the same, but because the line 722 of the surface 711 is aligned with the line 721 of the surface 714, the line 722 is mistakenly identified as the line 721. There can be. Similarly, line 723 has a projected width of 8, but the width seen is 2. Therefore, it is not possible to distinguish between lines 721, 722 and 723 on surfaces 711 and 712 during analysis of the received image. Since such lines cannot be distinguished, the result can be an incorrect analysis of the surface.

One proposed method of scanning used a pattern in which black and white triangles and rectangular columns run parallel to the triangulation plane, as disclosed in the foreign patent DE 19821611.4. The column used a measurement feature containing a digital encryption pattern.
However, when the surface being scanned causes shadowing and / or undercutting, a portion of the pattern is hidden, which can result in interruption of the sequence. Furthermore, since it is impossible to know which part of the pattern is lost, the disclosed encryption pattern may result in the pattern not being able to be decoded as a result of a sequence interruption. . A further limitation of the coded type described is that the distortion can make one coded feature resemble another coded feature. For example, a triangle can look like a rectangle.

  Therefore, methods and apparatus that can overcome the problems associated with the prior art of mapping objects are advantageous.

Detailed Description of Preferred Embodiments
In accordance with certain embodiments of the invention, an image is projected onto the surface. The image can include a pattern having a plurality of individual shapes used to measure and map the surface. The plurality of individual shapes includes a feature that is detectable in a direction parallel to a plane formed by points associated with the projected and projected axes of the projected shape. The image further comprises features that include coded information for individually identifying a plurality of shapes. The encoded feature varies in a direction substantially perpendicular to the plane formed by the projection and viewing axis points, and can be a separate feature from each of a plurality of individual shapes. It can be a feature that is integral to the shape and / or can be displayed at different time intervals from multiple individual shapes. The feature containing the coded information is oriented so that the coded information is retrieved along a line substantially perpendicular to the plane formed by the points along the projection axis and the viewing axis. The use of features is used to perform multi-frame reference independent scanning. In certain embodiments, the scanned frames are aligned with respect to each other.

  Certain embodiments of the present invention are best understood with reference to the accompanying FIGS. FIGS. 10 and 11 represent a system implementing a particular embodiment of the invention, FIGS. 12 and 19 to 22 illustrate a particular method according to the invention, and FIGS. 2 illustrates a specific implementation of the method in combination with the system.

  FIGS. 42 through 52 illustrate specific methods and apparatus of another embodiment of the present invention using three-dimensional scanning data of anatomical structures, and can be obtained in the specific manner illustrated herein. The 3D scan data is transmitted to the remote facility for further use. For example, three-dimensional scan data can represent the anatomy of living tissue, which can be used to design anatomical devices, manufacture anatomical devices, or structure of living tissue. Monitoring anatomical changes, storing data belonging to anatomical structures for a period of time, performing an iterative analysis of closed loops of anatomical structures, consulting structures interactively, or structures Or a diagnosis related to the anatomical structure or a treatment plan based on the anatomical structure is determined.

  FIG. 10 illustrates a system controller 951 that provides control signals to the scanning device 980. The scanning device 980 projects an image delimited by lines 962 and 963 and retrieves or observes the images in the reflected lines 972 and 973.

  In one operation, the system controller 951 provides specific information to the scanner 980 to identify a specific image to be projected on the surface 991 of the object 990. The reflected image is captured by the scanning device 980, which in turn returns the captured information to the system controller 951. The captured information can be automatically returned to the system controller 951 or stored in the scanning device 980 and retrieved by the system 951. Image data received by the system controller 951 is analyzed to determine the shape of the surface 991. Note that analysis of the received data can be performed either by the system controller 951 or by an external processing device not shown.

  10 illustrates a scanning device 980, which includes a projection device (projector) 960 and an observation device (viewer) 970. Projector 960 is oriented so that an image is projected onto object 990. The projector 960 has a projection axis 961. Projection axis 961 starts at the center of the lens that projects the image and represents the direction of projection. Similarly, the viewer 970 has a viewing axis 971 that extends from the center of the lens associated with the viewer 970 and represents the direction in which the image is being received.

  Once the scanning device is calibrated, an analysis of the received signal can be performed to map the scanned surface. Those skilled in the art will appreciate that the angles shown in the drawings of this application are shown for illustrative purposes only. Actual angles and distances can vary significantly from those illustrated.

  FIG. 11 illustrates the system controller 951 of FIG. 10 in more detail. The system control device 951 further includes a data processor 952, a projection image display 953, a projector control device 954, and a viewer control device 955.

  The viewer controller 955 provides the necessary interface to receive data from the viewer 970 representing the reflected image data. The reflected image data is received from the viewer 970 at the viewer controller 955 and then provided to the data processor 952. In a similar manner, projector controller 954 provides the interface necessary to control projector 960. Projector controller 954 provides projector 960 with an image to be projected in a format supported by the projector. In response, the projector 960 projects an image on the surface of the object. Projector controller 954 receives or accesses projection image display 953 to provide images to the projector.

  In the illustrated embodiment, the projected image display 953 is an electronic representation of an image stored in a memory location. The stored image can represent a bitmapped image or other standard or custom protocol used to define an image projected by projector 960. If the projection image is a digital image (made electronically), the display can be stored in memory by the data processor 952 so that the data processor 952 can modify the projection image display. It is possible to change the image as required according to the present invention.

  In another embodiment, the projected image display 953 need not be present. Alternatively, the projection controller 954 can select one or more transparencys (not shown) associated with the projector 960. Such transparency can include any combination of film, plate, or other type of reticle device that projects an image.

  Data processor 952 controls the projection and receipt of data through controllers 954 and 955, respectively.

  FIG. 12 illustrates a method according to the present invention discussed with reference to the system of FIG. 10 and the accompanying drawings. In order to better understand the methods discussed herein, unique terms and characteristics are described in the present invention. The term “projection / observation plane” means a plane formed by at least one point of the projection axis and the viewing axis. The term projection / view plane is best understood with reference to FIG. Assume that FIG. 3 represents a cross section of the object 100. The projection axis shown is oriented so that it lies entirely in the plane formed by the sheet including FIG. Similarly, the visual axis of FIG. 3 is entirely within the plane represented by the paper of FIG. In this embodiment, the projection / observation plane formed by at least one point of the projection axis of FIG. 3 and the viewing axis of FIG. 3 includes the sheet on which the drawing is drawn.

  However, if the visual axis of FIG. 3 is actually oriented so that the end point close to the viewing device is on the plane of the paper, and the end of the arrow on the visual axis is pointing from the paper to the reader, the visual axis and projection It is not possible to form a plane that includes the entire axis. Thus, the projection / view plane can be described as including substantially all of the projection axes and at least one point of the viewing axis, or all of the viewing axes and at least one point of the projection axis. For the purpose of this study, it is assumed that the point of the visual axis closest to the observation apparatus is the point that should be included in the projection / observation plane. For example, referring to prior art FIG. 4, the projection / view plane described with reference to FIG. 3 is substantially orthogonal to surface 104 and orthogonal to each of lines 121-125. The projection / view plane is represented by line 99, which represents the plane from the edge view that intersects lines 121-125.

  In step 611 of FIG. 12, an image having coded (variable) features with one or more components that fluctuate orthogonally to the projection / view plane is projected. In FIG. 13, the projection / viewing plane is illustrated by line 936, indicating that the orientation of the projection / viewing plane is on the edge so that such a plane appears as being a line. Each shape or pattern 931 to 935 represents a coded feature.

  Each of the individual features 931-935 has one or more components that vary in a direction orthogonal to the projection / viewing plane. For example, the feature 933 varies perpendicular to the projection plane so that three individual lines are identified. By changing the thickness of the three individual lines, a unique pattern is associated with each of the features 931-935. For example, the bar code feature 933 varies in the orthogonal direction: no line, thin line, no line, thick line, no line, thin line and no line. Individual lines of the feature 933 are projected parallel to the projection / observation plane. Projection lines parallel to the projection / view plane reduce or eliminate the effect of surface terrain observation distortion on the line width. Thus, since the observed width of the individual lines that make up the feature 933 is not substantially distorted, the thickness or relative thickness of each individual line of the feature 933 can be easily determined regardless of the surface phase. Can be identified. As a result, the feature 933 can be identified substantially independently of the surface topography.

  FIG. 13 displays a specific embodiment of an image having five separate lines (measurement features) 431-435. The illustrated lines 431 to 435 have a length that runs substantially perpendicular to the projection / viewing plane and are uniformly spaced from each other in a direction parallel to the projection / viewing plane. By providing a plurality of lines that can be detected in a direction parallel to the projection / observation plane, a plurality of measurement lines can be observed and analyzed simultaneously. In one embodiment, lines 431-435. In addition to lines 431-435, five unique bar codes 931-935 are also illustrated. Each unique bar code (variable feature) 931-935 is associated with and repeats along with a respective measurement feature 431-435. In another implementation, each barcode can be repeated along the measurement feature more than once as illustrated. Note that the illustrated barcode is illustrated as a repeating set. In another implementation, barcodes need not be grouped in sets.

  In certain embodiments, lines 431-435 and barcodes 931-935 are made using low-intensity visible light so that the pattern is visibly resistant and skin-tolerant. For example, lines 431-435 can be viewed as white lines and barcodes 931-935 can be viewed as specific colors or combinations of colors. In another embodiment, depending on the application, high intensity or laser light can also be used.

  By associating a barcode with a particular line in the manner illustrated, it is possible to distinguish lines from each other even when the lines appear to be linearly coincident. For example, lines 432 and 433 appear to be continuous lines at the edge of object 101. However, lines 432 and 433 can be distinguished from each other by analyzing the (coded feature) barcode associated with each line. In other words, if the lines 432 and 433 appear to be a common line to the viewer, the bar code associated with the left line 432 is not the same as the bar code associated with the right line 433, so two different It can be easily determined that it is a line.

  In the particular embodiment illustrated in FIG. 13, by analyzing the retrieved image, line segments 432 and 433 are placed either between the leftmost barcode 932 and the rightmost barcode 933. It is determined that there is a discontinuity that shows as a common line. In certain embodiments, such edge locations can be more precisely determined by repeating the barcode pattern relatively close to each other. For example, the edge where surface 102 meets surface 101 can only be determined with an accuracy equal to the spacing between adjacent barcodes. This is because when analyzing what appears to be a single line with two different barcodes, it is not known where the discontinuity occurred between the two barcodes. Therefore, discontinuous locations can be more accurately identified by repeating the barcode more frequently along the measurement line of FIG.

  The coded features 931 to 935 of FIG. 13 are not repeated because no two barcodes are identical. However, the coded value or sequence can be repeated in the projected image as long as it does not cause ambiguity. For example, if the image contains 60 lines (measurement features) that use binary coded, 6 bits of data are required to uniquely identify each line. However, due to the fact that the range of focus of the scanner is limited by the depth of the field, each individual line of 60 lines can appear as a recognizable image only within a certain range.

  FIGS. 25 and 26 better illustrate how the depth of the field affects feature repetition. FIG. 25 illustrates a projector that projects a shape along path 2540. As the shape is projected onto the surface, the image reflects to the viewing device 2506 along the reflection path. For example, a reflection path 2544 results when the shape reflects from the surface at location 2531, a reflection path 2541 results when the shape reflects from the surface at location 2532, and a shape reflects from the surface at location 2533. Results in a reflection path 2542 and results in a reflection path 2543 when the shape reflects from the surface at location 2534.

  FIG. 26 shows the shape that the viewer 2506 shows. Specifically, the image reflected from the surface 2531 which is the surface closest to the project is seen in the rightmost image in FIG. 26, and the image reflected from the surface 2534 which is the surface farthest from the project is shown in FIG. It can be seen in the leftmost image. However, it should be noted that the rightmost and leftmost images are the farthest and closest to the project 2505, respectively, and are out of focus. Because it is out of focus, it cannot be accurately detected based on the image received by the viewing device 2506.

  Referring back to FIG. 25, any surface closer to the projection device 2505 than the plane 2525 or any surface farther from the projection device 2505 than the plane 2526 is outside the visible range 2610 or out of the field of view. Therefore, the usable shape cannot be reflected. Thus, as long as the repeated shape cannot be seen within range 2610 of FIG. 6, the shape can be repeated and still uniquely identified.

  In certain embodiments, the projector projects approximately 80 lines. For example, when using three colors (red, blue, green), a coded feature with three color locations uniquely identifies 27 different lines. If a field of view is provided so that the same coded line cannot be seen at the same location, this 27 coding sequence can be repeated three times to cover all 80 lines. In another embodiment, five color locations can be added with or without increasing the number of lines in the sequence in order to provide a recognition capability where specific color locations can be lost.

  This means that the coding features may be repeated as long as the fields of view where each of the repeating features cannot be seen do not overlap. Thus, a sequence of 12 unique coded features encodes all 60 lines if only 4 bits of binary data are needed and the feature has no opportunity to be seen in the same place Can be repeated 5 times.

  Reference independent scanning is achieved by providing a pattern having multiple measurement features with associated coding features. Specifically, neither the object nor the scanner need be fixed in space or need to be referenced with respect to each other. Instead, on a frame-by-frame basis, the reference independent scanner retrieves enough measurement information (3D cloud), which is accurate due to the coding feature and aligns with its neighboring frames Can do. Registration is a process of determining overlapping features on adjacent frames and forming an integrated map of objects.

  FIG. 14 illustrates the object of FIG. Here, the measurement lines 431 to 435 have varying thicknesses. However, the thickness of lines 431-435 is subject to distortion. Thereby, identifying individual lines 431-435 based solely on their thickness is prone to error. This is better illustrated with reference to FIG.

  FIG. 15 illustrates the object 700 of FIGS. 8 and 9 having a pattern projected onto its surface in accordance with the present invention. FIG. 15 illustrates the projection of lines 721 to 723 having varying widths. As discussed above, lines 722 and 723 appear to have the same line thickness as line 721 when projected onto surfaces 711 and 712, respectively. Therefore, simply measuring a line of varying thickness cannot determine which line is which by analysis of the image. However, by further incorporating coded features 451-453 to have components that vary orthogonal to the projection / viewing plane, identification of lines 721-723 and subsequent mapping analysis is On the other hand, it is improved.

  The illustrated embodiment in which the encoded feature is projected to have a portion that is orthogonal to the projection / view plane more accurately identifies specific lines associated with the pattern by analyzing the received image Those skilled in the art will recognize that this is advantageous over the prior art. Those skilled in the art will further appreciate and understand that the specific embodiments described herein are described with reference to lines and barcodes. However, other patterns, shapes and features can also be used.

  Referring to FIG. 16, a table illustrating a particular shape set used in a direction orthogonal to the projection / viewing plane is illustrated. Column 1 of Table 16 represents a unique feature identifier. Columns 2 through 4 of Table 16 illustrate specific ways in which each feature identifier can be represented. Column 2 shows the barcode. Column 3 shows the colors that can be used alone or with other coded features. Note that some types of coded features, including color features, can be implemented as an integral part of the measurement feature or as a coded feature separate from the measurement feature. . Similarly, other types of coding can be based on measurements and / or features and the intensity with which the coded features are projected. Column 4 represents a pattern that can be used separately from the shape identifying shape or in combination as part of the shape. In other words, a line including a repetitive pattern sequence of the type illustrated in column 4 can be provided. In this way, the pattern change in the direction orthogonal to the projection / viewing plane can be relative to the actual shape itself. In addition, those skilled in the art will recognize that many variations on variable components are anticipated by the present invention.

  FIG. 17 illustrates the use of a unique non-repeating identifier for each line in tabular form. For example, referring to the first column of FIG. 17, sequences 0 through F are presented sequentially. In one implementation, each value from 0 to F represents a unique code associated with a particular line. Those skilled in the art will recognize that some type of spacer needs to be present between each individual code to identify a particular code. For example, long spaces or unique codes can be used.

  In a system for projecting and analyzing four lines each having one of the sequences illustrated in FIG. 17, once three code sequences have been retrieved, one of the four lines is analyzed. It is possible to identify what is being done. In general, since codes vary orthogonal to the projection / viewing plane, lost codes do not cause misidentification problems.

FIG. 18 illustrates four unique repetitive code sequences. The letter S in Table 18 is used to represent the spacer used during the repeating sequence.
The spacer may be a unique identifier that identifies where each repeated code of the encoded sequence begins and / or ends.

  Returning to the flow of FIG. 12, once an image with coded features orthogonal to the projection / view plane is projected, a display of the surface image is received at the viewer. This is similar to the discussion of FIG. 10, whereby the viewer 970 receives the reflected image. Next, at step 613, the location of the point associated with the object is determined based on the orthogonally varying feature. In a particular embodiment of the invention, each one of the shapes, eg lines, is quantified into a unique code pattern before being used for object analysis, so the points are based on variable components.

FIG. 19 illustrates the sub-steps associated with step 611 of FIG. In step 621, the first image is projected, while in step 622, the second feature is projected. Referring to FIG. 14, the first image can be similar to the combination of the measurement line 431 and its associated coded feature 931. In a similar manner, the second feature can be represented by a combination of measurement line 432 and its encoded feature 932. In addition to being able to analyze the line 431 against the feature 931, in another embodiment, it is also possible to determine the identification of the line 431 based on the encoded feature 932. I want to be.
In other words, a particular line in a group of lines, as illustrated in FIG. 14, can be identified based on two or more of the various coded features. However, in certain embodiments, only a single contiguous set of coded features, or multiple contiguous sets of coded features are used. In addition, steps 621 and 622 can occur at different times, as discussed with reference to FIG.

  FIG. 21 illustrates another method according to the present invention. In step 631, a plurality of first features and a plurality of second features are projected. These features may be projected simultaneously or may be projected to separate locations.

  In step 632, one of the plurality of first features is determined or identified based on the second feature. Referring to FIG. 14, the plurality of first features include lines 431 to 435 to be measured. A second feature, bar code 931-935, can be used to identify a particular one of lines 431-435.

  In step 633, the location of the point on the surface is determined based on a particular one of the plurality of parallel first features.

  This particular embodiment is advantageous over the prior art because lines identified by analysis of the received shape are not used until their identity is verified based on the encoded information.

  FIG. 22 illustrates another method according to the present invention. In step 641, parallel first and second discontinuous shapes are projected. Examples of such discontinuous shapes include lines 431 and 432 in FIG. However, those skilled in the art will recognize that various other parallel shapes can be projected.

  In step 642, the coded features for the first discontinuous shape are projected. Referring again to FIG. 14, the coded features for line 432 can include coded features 932 or even coded features 933.

  In step 643, the coded features for the second discontinuous shape are projected.

  In step 644, a first discontinuous shape is identified based on the first encoded feature. This is accomplished in a manner similar to that previously discussed.

  In step 645, the location of the particular point of the object is determined based on the first discontinuous shape.

  FIG. 23 illustrates another embodiment of the present invention. Specifically, FIG. 23 illustrates a series of images projected at times T1, T2, T3, and T4. At time T1, the projected image includes measurement features 1011 to 1013. During the time T1, the coded features are not projected. During time T2, an image including coded features 1021 to 1023 is projected. The pattern of times T1 and T2 is repeated during times T3 and T4, respectively. As a result of alternating projection of coded and measurement features, denser patterns can be used and more information can be obtained. Note that the image at time T4 shows the coded features 1021 to 1023 lying on top of the measurement features 1011 to 1013. However, in one embodiment, the measurement features are included for illustrative purposes only and are generally not present at the same time as the coded features.

  In yet another embodiment of the invention, FIG. 24 illustrates an image having features with different characteristics. Specifically, FIG. 24 illustrates an image 1100 with lines 1131 through 1134 and a distance X between individual lines, while the distance between lines 1134, 1135 and 1136 separates the lines. Illustrate a substantially larger distance Y. By allowing the features to have different isolation properties, it is possible to provide a high resolution feature. In other words, line 1135 can be used to map surface features that cannot (if not) be mapped. Note that pattern 1100 can be used with or without the coding techniques described herein.

Once the scanner receives or observes the projected frame pattern, the frame pattern is digitized into a plurality of 2D points (2D image frames). Since the projection axis and visual axis of the scanner are fixed and known, if each 2D point of the 2D image frame can be correlated to the projected point, the 2D image can be generated using conventional 3D image forming techniques. Each 2D point of the frame can be converted to a 3D point. By using a projected frame pattern with coded features, a 2D image point can be correlated to each projected point.
Multi-frame reference independent scanning is described herein in accordance with another aspect of the present disclosure. In certain embodiments, multiple 3D images are received by scanning the object one frame at a time using a portable scanner to obtain multiple frames, each frame capturing only a portion of the object. To do. Referring to multiple frames, the reference independent scan has a spatial position that can vary from frame to frame with respect to the object being scanned, and the spatial position is not fixed relative to the reference point, or Not tracked. For example, there is no fixed reference point for the object being scanned.

One type of reference independent scanner disclosed herein includes a portable scanner that projects a pattern onto a continuous frame having a measurement feature and a coded feature.
This allows each observed point of the frame to have a known corresponding projected point, thereby converting 2D frame data to 3D frame data.

  27-28 are used to consider multiple frame reference independent scans.

  27, 28 and 30 illustrate the object 2700 from different viewpoints. As illustrated in FIG. 27, the object 2700 includes three teeth 2710, 2720, and 2730 and a gingival portion 2740 adjacent to the three teeth.

  FIG. 27 is a viewpoint where a plurality of non-continuous surface portions can be seen. For example, from the viewpoint of FIG. 27, three non-continuous surface portions 2711 to 2713 are visible. Surface portion 2713 represents the side portion of tooth 2710. Surface portion 2711 represents a portion of the mating surface of tooth 2710 that is not continuous with surface portion 2713. Surface portion 2712 represents another portion of the mating surface of tooth 2710 that is not continuous with either portion 2711 or 2713. In a similar manner, tooth 2720 has four surface portions 2721 to 2724 and tooth 2730 has four surface portions 2731 to 2734.

FIG. 28 illustrates an object 2700 from a slightly different viewpoint (viewpoint in FIG. 28). The view point changes from FIG. 27 to FIG. 28 as a result of the viewer or scanner moving in a direction that allows a larger portion of the upper tooth surface to be seen.
The change in viewpoint results in a variation for multiple visible surface portions. For tooth 2710, tooth portion 2813 now represents a smaller 2D surface than its corresponding tooth portion 2713. On the other hand, tooth portions 2811 and 2812 are now seen as 2D surfaces that are larger than their corresponding portions 2711 and 2712 in FIG.

  For tooth 2720, surface 2824 is viewed as a 2D surface that is smaller than its corresponding tooth surface 2724 in FIG. For tooth 2720, surface 2821 represents a tooth surface that is viewed sequentially, including both surfaces 2721 and 2723 from the perspective of FIG.

  For teeth 2730, the 2D surfaces 2832 and 2835 seen each include a portion of the surface 2732 and a surface area not previously seen. This is a result of the topographic features of teeth 2730. As a result, surface 2732 is not continuously visible from the second frame viewpoint.

The relationship of the tooth portion of FIG. 27 to the tooth portion of FIG. 28 is from the same perspective as FIG. 28, with reference to the view with the seen surface portion of FIG. 27 shown as a shaded area. Well understood. For example, the surface portion 2711 in FIG. 27 is represented as a shaded portion in the surface portion 2811. As illustrated, due to the change in viewpoint between FIG. 27 and FIG. 28, the resulting surface portion 2811 includes a smaller viewed surface portion 2711.
Similarly, different surface portions are seen as a result of changes in viewpoint.

  The visible surface portion of FIG. 28 is superimposed on the view from directly above the teeth 2710 to 2730. Reference is now further made to the object 2700 illustrated in FIG. 27, which describes a particular embodiment of multi-frame reference independent scanning.

  FIG. 29 illustrates a method according to a particular embodiment of reference independent scanning. In step 3101, the object is scanned to obtain a 2D cloud of data. A 2D cloud of data has multiple frames. Each of the frames has a plurality of 2D points, which when viewed represent a 2D image.

  In step 3102, the first frame of the 2D cloud of data is converted to a 3D frame model. In one embodiment, the 3D frame model is a 3D point model, which includes a plurality of points in a three-dimensional space. The actual conversion to a 3D frame point model is performed on some or all of the 2D cloud of frames of data using conventional techniques for converting a scanned 2D cloud of data to a 3D point model . In certain embodiments that use coded features as disclosed herein, surfaces with non-consecutive viewed surfaces, such as teeth 2710 to 2730 in FIG. 27, are successfully scanned from frame to frame. Is possible.

  FIGS. 30 and 31 illustrate an object 2700 being scanned from the viewpoint of FIGS. 27 and 28, respectively. In FIG. 30, the scan pattern includes lines 3211 to 3223. Any scan line portion outside the frame boundary 3210 cannot be properly scanned. Within the boundary 3210, each scan line is converted into a plurality of 2D points (data clouds) as detected by the CCD (charge coupled diode) chip of the scanner. Some or all of the scan lines can be used in accordance with the present invention. For example, every other or every other point of the scan line can be used, depending on the desired resolution of the final 3D model. FIG. 30 illustrates four points (A to D) of each identified line. A 2D coordinate value, such as an XY coordinate, is determined for each of these points.

  In particular embodiments of scanning, a scanning rate of 1 to 20 frames per second is used. Faster scanning speeds can also be used. In certain embodiments, the scanning speed is chosen so that a three-dimensional image can be viewed in real time. The pulse time during which each frame is captured is a function of the speed at which the scanner is expected to be moving. For dentition structures, the maximum pulse width is determined to be approximately 140 microseconds, but a much faster pulse width, or 3 microseconds, may be used. In addition, in certain embodiments, the teeth 2710-2730 are coated with a material resulting in a more opaque surface than the teeth themselves.

  In certain embodiments, each point in the cloud of data is analyzed during various steps and functions described herein. In another embodiment, only a portion of the cloud of data is analyzed. For example, it may be determined that only every second or third point needs to be analyzed to meet the desired resolution. In another embodiment, a portion of the frame data is a frame of data so that only a particular spatial portion of the cloud of data is used, for example, only the central portion of the cloud of data is included in the bounding box. It can be a bounding box that is smaller than the whole. By using a subset of the cloud of data, it is possible to speed up the various routines described herein.

  FIG. 31 illustrates an object 2700 being scanned from the viewpoint of FIG. As such, the pattern seen, including lines 3321-3323, is positioned differently on teeth 2710-2730. In addition, the frame boundary 3310 moves to include most of the teeth 2720.

  FIG. 32 illustrates another embodiment of a 3D frame model referred to herein as a 3D initial model. The 3D initial model includes a plurality of initial shapes based on the 3D points of the frame. In the particular embodiment illustrated, the points adjacent to the 3D point model are selected to form triangles, including triangles PS1 to PS3 as initial shapes. Other implementations can use different or various initial shapes.

  Using the initial shape to perform the alignment can use a lower resolution model by using one initial surface representation of the point cloud, resulting in faster alignment. It is advantageous for registration techniques that attempt to obtain two point cloud points as close as possible to each other, since there are no desirable any error disadvantages. For example, if a 1 mm scan resolution is used for point-to-point alignment, the best guaranteed alignment between two frames is 0.5 mm. This is due to the fact that the portable scanner randomly captures which points on the surface are mapped. Point-to-surface use provides more accurate results because alignment can occur at any point on the surface, not just the vertices.

  In step 3103 of FIG. 29, a second 3D frame model is created from the second frame of cloud data. Depending on the particular implementation, the second 3D frame model may be a point model or an initial model.

  In step 3104, alignment is performed between the first frame model and the second frame model to create a cumulative model. “Alignment” means the process of aligning a first model with a second model and determining the best fit by using the portion of the second model that overlaps the first model. The portion of the second model that does not overlap the first model is the portion of the scanned object that has not yet been mapped and is added to the first model to create a cumulative model. The alignment is better understood with reference to the method of FIG.

  FIG. 33 includes an alignment method 3500, which in certain embodiments is invoked by one of the alignment steps of FIG. In step 3501 of FIG. 33, an item point for alignment is determined. The item point to alignment defines the initial guess for the alignment of the overlapping parts of the two models. The particular embodiment for selecting item points is discussed in more detail with reference to FIG.

  In step 3502, two shape registrations are attempted. Registration is successful if an overlap is detected that matches the defined approximation or quality of the fit. If the alignment is successful, the flow returns to the calling step of FIG. If the alignment is not successful, flow proceeds to step 3598 where it is determined whether to continue.

The decision to proceed can be made based on the number of factors. In one embodiment, the decision to proceed is made based on the number of alignment item points attempted. If the determination at step 3598 is to stop the alignment attempt, flow proceeds to step 3503 where alignment error handling occurs.
Otherwise, the flow continues at step 3501.

  FIG. 34 illustrates a particular method for selecting alignment item points. In step 3699, a determination is made whether this is the first item point for a particular alignment attempt for a new frame. If so, flow proceeds to step 3601, otherwise flow proceeds to step 3698.

  In step 3601, the X and Y components of the item point are determined based on a 2D analysis of the 2D cloud of data for each of the two frames. In certain embodiments, two-dimensional analysis performs cross-correlation of 2D images. These 2D images need not be from a 2D cloud of data; instead, data associated with plain video images of objects without patterns can be used for cross-correlation. In this way, a promising movement of the scanner can be determined. For example, cross-correlation is used to determine how the pixels have moved to determine how likely the scanner has moved.

  In another embodiment, rotation analysis is possible, but this is time consuming, so this is not done for a particular embodiment, and here it is by having the correct item points in the X and Y coordinates. The described alignment algorithm can handle rotation.

  In step 3602, a promising movement in the Z direction is determined.

  In one particular embodiment, the Z coordinate of the previous frame is used and any change in the Z direction is calculated as part of the alignment. In another embodiment, a promising Z coordinate is calculated as part of the item point. For example, the optical parameters of the system can “zoom” the second frame relative to the first frame until the best fit is received. The zoom factor used for this tells how far the two surfaces are in Z from each other. In certain embodiments, the X, Y, and Z coordinates can be aligned so that the Z coordinate is substantially parallel to the viewing axis.

  In step 3603, the item point value is returned.

  In step 3698, it is determined whether all item point variations have been attempted in registration steps 3601 and 3602. If not, the flow proceeds to step 3603; otherwise, the flow proceeds to step 3697.

  In step 3603, the next item point variation is selected. FIG. 35 illustrates a particular method for selecting registration item point variation. Specifically, FIG. 35 illustrates an initial item point E1 and subsequent item points E2 to E9. The item points E2 to E9 are sequentially selected in any predetermined order. The particular embodiment of FIG. 35 illustrates alignment item points E2 through E9 as various points of a circle 3720 having a radius 3710. According to certain embodiments, the range of item point variation is two-dimensional, for example, X and Y dimensions. In other embodiments, the item points can vary in three dimensions. Note that a variable number of item points, i.e. a subset of item points, can be used to speed up the registration process. For example, the single frame alignment used here may use fewer than the nine item points shown. Similarly, the cumulative registration described herein can benefit from using many numbers beyond the nine points illustrated.

  Returning to step 3698 of FIG. 34, once all variations of the first identified item point have been attempted, flow proceeds to step 3697. In step 3697, all item points associated with the first identified item point are tried to determine whether the second identified item point has been identified by step 3604. If not, flow proceeds to step 3604 where a second item point is defined. Specifically, in step 3604, scanner motion between two previous frame models is determined. Next, it is assumed that the scanner movement is constant for at least one additional frame. Using these assumptions, the item point at step 3604 is defined as the location plus the computed scanner movement for the previous frame. Flow proceeds to step 3606, which returns the item point to the calling step of FIG. In another embodiment, it can be assumed that the direction of movement of the scanner remains the same, but is accelerated at different speeds.

  If the second identified item point of step 3604 has been previously determined, the flow from step 3697 proceeds to step 3696. In step 3696, it is determined whether there is an additional registration item point variation for the second identified item point. If so, flow proceeds to step 3605; otherwise, flow returns to the calling step of FIG. 29 at step 3607, indicating that the selection of a new item point was unsuccessful. In step 3605, the next item point variation of the second identified item point is identified, and the flow returns to the calling step of FIG.

  Different item point routines can be used depending on the type of alignment being performed. For example, an alignment process that is not tolerant of frame data interruption requires more item points to be tried before abandoning a particular frame. For an alignment process that is tolerant of frame data interruption, simpler or fewer item points can be attempted, thereby speeding up the alignment process.

  Returning to FIG. 29, at step 3105, the next 3D model portion is created from the next frame of cloud data.

  In step 3106, alignment is performed between the next 3D model portion and the cumulative model to update the cumulative model. In a particular implementation, the cumulative model is updated by adding all new points from the frame to the existing cumulative model to arrive at the new cumulative model. In other implementations, a new surface based on the 3D points acquired so far can be stored, thereby reducing the amount of data stored.

  If all frames are aligned, method 3100 is complete, otherwise flow proceeds through step 3199 to step 3105 until the cloud of points for each frame is aligned. As a result of the registration process described in method 3100, it is possible to develop a model for object 2700 from multiple smaller frames, such as frames 3210 and 3310. By being able to align multiple frames, a highly accurate model of a large object can be obtained. For example, a model of the entire patient's dentition structure, including gingiva, teeth and orthodontic and prosthetic structures, can be obtained. In another embodiment, a model of the patient's face can be obtained.

  FIG. 36 illustrates a method 3800, which is an alternative method of aligning an object using multiple frames from a reference independent scanner. Specifically, in step 3801, the object is scanned and cloud data for the object is received. As described above, a cloud of data includes data from multiple frames, and each frame includes multiple points.

  In step 3802, a single frame alignment is performed. Single frame alignment performs alignment between adjacent frames of the scanned image without creating a cumulative model. Instead, in certain embodiments, a cumulative image of a single frame alignment process is displayed. An image formed by a single frame alignment process can be used to assist the scanning process. For example, the image displayed as a result of a single frame alignment is not as accurate as the cumulative model, but can be used by the scanner operator to determine the areas that require additional scanning.

  The single frame alignment process “extends” any error introduced between any two frames to all subsequent frames of the 3D model created using the single frame alignment. Is done. However, the level of accuracy is adequate to assist the operator during the scanning process. For example, the registration result describes the movement from one frame to another and can be used as an item point for cumulative registration processing. Single frame alignment is discussed in more detail with reference to FIG.

  In step 3803, cumulative alignment is performed. Cumulative registration creates a cumulative 3D model by registering each new frame with the cumulative model. For example, if 1000 individual frames representing 1000 reference independent 3D model parts (frames) were captured in step 3801, the cumulative registration step 3803 may represent 1000 reference independent 3D model parts representing a single object. Combine with the cumulative 3D model. For example, if each of the 1000 reference independent 3D model parts represents a portion of one or more teeth including the frames 3210 and 3310 of FIGS. To 2730 represent the entire set of teeth.

  In step 3804, the registration result is reported. This is discussed in further detail below.

  FIG. 37 describes a method 3900 that is specific to the single frame rendering implementation of step 3802 of FIG. In step 3903, the variable x is set equal to 2.

  In step 3904, alignment between the current frame (3DFx) and the previous or first adjacent frame (3DFx-1) is performed. The alignment between two frames is referred to as single frame alignment. Specific embodiments of alignment between two models are discussed in further detail with reference to the method illustrated in FIG.

  In step 3999, it is determined whether the single frame alignment of step 3904 was successful. In certain embodiments, an alignment method, such as the method of FIG. 38, provides a success indicator that is evaluated at step 3999. If the alignment is successful, the flow proceeds to step 3905; otherwise, the flow proceeds to step 3907.

  When step 3999 determines that the alignment is successful, flow proceeds to step 3905. In step 3905, the current 3D frame (3DFx) is added to the current set of 3D frames. Note that this set is generally a set of transformation matrices. The current frame set of 3D frames is a contiguous set of frames, and each frame of the sequence is likely to be successfully aligned with (both) its two adjacent frames. In addition, the newly aligned frame can be displayed relative to the previously displayed frame.

  In step 3998, it is determined whether the variable x has a value equal to n, where n is the total number of frames to be evaluated. If x equals n, single frame alignment is complete and flow can return to FIG. 36 at step 3910. If x is less than n, single frame alignment is continued at step 3906 and x is incremented before proceeding to step 3904.

  Returning to step 3999, if the alignment of step 3904 is not successful, the flow proceeds to step 3907. In step 3907, an alignment is attempted between the current frame (3DFx) and the previous adjacent frame (3DFx-2). If the alignment at step 3907 is successful, step 3997 directs the flow to step 3905. Otherwise, step 3997 directs the flow to step 3908, thereby indicating that the alignment of the current frame (3DFx) is unsuccessful.

  If the current frame cannot be aligned, step 3908 saves the current frame set or matrix set and a new current frame set is started. The flow from step 3908 proceeds to step 3905 where the current frame is added to the current frame set, which was newly created in step 3908. Thus, the single frame alignment step 3802 can identify multiple frame sets.

  Creating multiple frame sets during cumulative registration is undesirable due to the amount of intervention required to reconcile multiple cumulative models. However, interruption of single frame alignment is generally accepted because the purpose of single frame alignment is to assist the operator and define entry points for cumulative alignment. One way to deal with a break during single frame alignment is simply to display the first frame after the break in the same place as the last frame before the break, so that the operator views the image. Can continue.

  According to step 4001 of FIG. 38, the first model is a 3D initial shape model, while the second model is a 3D point model. For reference purposes, the initial shape of the first 3D model is referred to as S1... Sn, where n is the total number of shapes of the first model and the points of the second 3D model are referred to as P1. Where z is the total number of the second model.

  In step 4002, the individual points P1 ... Pz of the second model are analyzed to determine the shape closest to that location. In a particular embodiment, the point P1 has a shape S1 ... Sn closest to P1 having a surface location closest to P1 than any other surface location of any other shape. The shape closest to the point P1 is referred to as Sc1, while the shape closest to the point Pz is referred to as Scz.

In another embodiment, only points that are directly above or below the triangle are associated with the triangle, and points that are not directly above or below the surface of the triangle are lines formed between the two triangles, or , Associated with a point formed by a plurality of triangles.
Note that in a broad sense, the lines that form a triangle and the points that form the corners of a triangle can be considered as shapes.

  In step 4003, vectors D1... Dz are calculated for each of the points P1. In a particular implementation, each vector, eg, D1, has a magnitude and direction defined by the minimum distance from its corresponding point, eg, P1, to its closest shape, eg, the nearest point of Sc1. In general, only a part of the points P1... Pz overlaps the accumulated image. Non-overlapping points that do not need to be aligned may have an associated vector that is relatively larger than the overlapping points, or may not be directly above or below a particular triangle. Thus, in certain embodiments, only vectors having a magnitude smaller than a predetermined value (epsilon value) are used for further alignment.

  In addition to eliminating points that are not likely to overlap, epsilon values can be used to further reduce the risk of decoding errors. For example, if one of the pattern measurement lines is misinterpreted as a different line, a large error may occur in the Z direction as a result of the misinterpretation. When the general distance between adjacent pattern lines is approximately 0.3 mm and the triangulation angle is approximately 13 degrees, an error of 0.3 mm in the X direction results in approximately 1.. A 3 mm three-dimensional deformation error results (0.3 mm / tan 13 degrees). If the epsilon distance is kept below 0.5 mm, it is certain that there will be no influence of surface areas further away from each other by more than 0.5 mm. Note that in certain embodiments, the epsilon value is initially selected to be a value greater than 0.5 mm, for example 2.0 mm, and the value is reduced once a certain quality is reached.

  In step 4004, in a particular embodiment, the vectors D1 ... Dz are treated as spring forces to determine the motion of the second 3D model frame. In a particular embodiment, the second 3D model is moved in a linear direction defined by the sum of all force vectors D1 ... Dz divided by the number of vectors.

  In step 4005, the vectors D1 ... Dz are recalculated for each point of the second 3D model.

  In step 4006, the vectors D1 ... Dz are treated as spring forces to determine the movement of the second 3D model. For the particular embodiment of step 4004, the second 3D model frame rotates around the center of mass based on the vectors D1... Dz. For example, the second 3D model rotates around the center of mass until the spring force is minimized.

  In step 4007, the alignment quality for the current orientation of the second 3D model is determined. Those skilled in the art will appreciate that various methods can be used to define the quality of the alignment. For example, the standard deviation of vectors D1... Dz having a magnitude smaller than epsilon can be used. In another embodiment, the quality can be calculated using the following steps: That is, square the vector distance, sum the squared distances of all vectors within the epsilon distance, divide this sum by the number of vectors and take the square root. Note that one skilled in the art will recognize that the vector values D1... Dz need to be recalculated after the rotation step 4006. In addition, those skilled in the art will recognize that there are other statistical calculations that can be used to provide a quantitative value indicative of quality.

  In step 4009, it is determined whether or not the quality determined in step 4007 matches a desired quality level. If the quality is within the desired level, it indicates with a high degree of confidence that perfect alignment between the two frame models can be achieved. By ending the flow of method 4000 when the desired degree of quality is obtained, it is possible to sort immediately across all pairs of frames to provide the user with images. By eliminating possible interruptions in the data at this point in the method, subsequent cumulative alignment is likely to create a single cumulative model rather than multiple segments of the cumulative model. If the current quality level matches the desired level, the flow indicates success and returns to the appropriate call step. If the current quality level does not match the desired level, the flow proceeds to step 4098.

  In step 4098, it is determined whether the current quality of alignment has improved. In certain embodiments, this is determined by comparing the quality of the previous pass through the loop including step 4003 with the current quality. If the quality has not improved, the flow indicates that the alignment was not successful and returns to the calling step. If so, flow proceeds to step 4003.

  Returning to step 4003, another alignment iteration occurs using the new frame location. Note that once the frame data has been scanned and stored, there is no need to align exactly in the scan order. The alignment can be started in reverse, or any other order that makes sense may be used. In particular, when a scan results in multiple passes, it is already known where the frame belongs. Therefore, alignment of adjacent frames can be performed regardless of the order of image formation.

  FIG. 39 illustrates a particular embodiment of the method 4100 for FIG. Specifically, the method 4100 discloses cumulative alignment that attempts to combine all individual 3D frame models into a single cumulative 3D model.

  Steps 4101 to 4103 are setup steps. In step 4101, the variable x is set to be equal to 1, and the variable x_last defines the total number of 3D model sets. Note that the number of 3D model sets is based on step 3908 of FIG.

  In step 4102, the 3D cumulative model (3Dc) is initially set equal to the first 3D frame of the current set of frames. The 3D cumulative model is modified to include that information from subsequent frame models that are not already represented by the 3D cumulative model.

  In step 4103, Y is set equal to 2 and the variable Y_last is defined to indicate the total number of frames (3DF) or frame models in the set Sx, where Sx is the current frame model being aligned. Represents a set of

  In step 4104, the 3D cumulative model (3Dc) includes additional information based on the alignment between the current 3D frame model being aligned (Sx (3DFy)) and the 3D cumulative model (3DC). Will be corrected. Note that in FIG. 39, the current 3D frame model is referred to as Sx (3Dy), where 3Dy indicates the frame model and Sx indicates the frame set. Particular embodiments for performing the alignment of step 4104 are further described by the method illustrated in FIGS.

  In step 4199, it is determined whether the current 3D frame model is the last 3D frame model of the current step. According to the particular embodiment of FIG. 39, this can be achieved by determining whether the variable Y is equal to the value Y_last. When Y is equal to Y_last, the flow proceeds to step 4198. Otherwise, the flow proceeds to step 4106 where Y is incremented before returning to step 4104 for further alignment of the 3D frame model associated with the current set Sy.

  In step 4198, it is determined whether the current set of frames is the last set of frames. According to the particular embodiment of FIG. 39, this can be achieved by determining whether the variable x is equal to the value x_last. When x is equal to x_last, the flow proceeds to step 4105. Otherwise, flow proceeds to step 4107 where x is incremented before returning to step 4103 for further alignment using the next set.

  When the flow reaches step 4105, all frames in all sets are aligned. Step 4105 reports the result of the alignment of method 4100 and any other cleanup operations. For example, ideally method 4100 results in a single 3D cumulative model, but in practice, multiple cumulative models may be created (see the discussion of step 4207 in FIG. 41). When this occurs, step 4105 can report the resulting number of 3D cumulative models to the user or to the next routine for handling. As part of step 4105, the user may have an option to help align multiple 3D models with each other. For example, once two 3D cumulative models have been created, the user can graphically manipulate the 3D cumulative model to help identify item points, which can be used between the two 3D cumulative models. Registration can be performed.

  In accordance with another embodiment of the present invention, the second cumulative alignment process can be performed using the resulting matrix from the first cumulative alignment as a new calculation item point. In one embodiment, when processing encounters a point where one or more frames cannot be successfully aligned on the first attempt, a larger number of item points can be used, or Higher percentages can be used.

  40-42 disclose a specific embodiment of the alignment associated with step 4104 of FIG.

  Step 4201 is similar to step 4002 of FIG. 38, where each point (P1... Pm) of the current frame Sx (3Dy) is analyzed to determine the shape of the cumulative model that is the closest shape.

  Step 4202 defines a vector of points in the current frame in a manner similar to that described above with reference to step 4003 of FIG.

  Steps 4203 to 4206 move the current 3D frame model in the manner described in steps 4004 to 4006 of FIG. 38, where the first model of method 4000 is a cumulative model and the second model of method 4000 is the current model. It is a frame.

  In step 4299, it is determined whether the current pass through registration steps 4202 to 4206 resulted in an improved alignment between the cumulative model and the current frame model. One way to determine quality improvement is to compare a quality value based on the current position of the model to a quality value based on the previous position of the model. As discussed above with reference to FIG. 38, the quality value can be determined using standard deviation or other quality calculations based on the D vector. Note that by default, the first pass through steps 4202 to 4206 for each model 3Dy results in an improved alignment. If an improved alignment occurs, flow returns to step 4202, otherwise flow proceeds to step 4298 of FIG.

  Note that the flow control for the cumulative alignment method of FIG. 40 is different from the flow control for the single frame alignment method of FIG. Specifically, the cumulative flow continues until no quality improvement is realized, but the single frame flow stops once it reaches a certain quality. Other embodiments for controlling flow within the alignment routine are envisioned.

  In an alternative flow control embodiment, the registration iteration process continues as long as the convergence criteria are met. For example, the convergence criterion is considered met as long as a quality improvement greater than a fixed rate is achieved. Such a percentage can be in the range of 0.5 to 10%.

  In another embodiment, additional stationary iterations can be used once certain first criteria are met, such as once there is no improvement in convergence or quality. The stationary iteration passes the registration routine once the quality level stops improving or meets predetermined criteria. In certain implementations, the number of stationary iterations can be fixed. For example, 3 to 10 additional iterations can be identified.

  In step 4298, it is determined whether the current alignment is successful. In certain implementations, success is simply based on whether the calculated quality value of the current model configuration meets a predetermined criterion. If so, the alignment is successful and routine 4200 returns to the calling step. If the criteria are not met, flow proceeds to step 4207.

  In step 4207, it is determined that the current frame model cannot be successfully registered to the cumulative 3D model. Thus, the current cumulative 3D model is saved and a new cumulative 3D model with the current frame is started. As described above, since a new 3D cumulative model has been started, the current 3D frame model, which is a point model, is converted to the initial model before returning to the calling step.

  There are many other embodiments of the present invention. For example, the frame motion during steps 4004, 4006, 4203, and 4205 may include acceleration or overmotion components. For example, analysis can indicate that the motion in a particular direction needs to be 1 mm. However, the frame can be moved by 1.5 mm or some other scaled factor to compensate for the sample size being calculated or other factors. Subsequent movement of the frame can use similar or different acceleration factors. For example, a smaller acceleration value can be used for alignment progress. By using an acceleration factor, it helps to compensate for the local minima that occur when overlapping features that align are not generated. When this occurs, a small motion value can result in a lower quality level. However, it is likely that alignment errors can be overcome by using acceleration. In general, acceleration can be beneficial in overcoming feature “bumps”.

  It should be understood that the specific steps shown in the present method and / or the functions of the specific modules of the present application may generally be implemented in hardware and / or software. For example, certain steps or functions may be performed using software and / or firmware executed on one or more processing modules.

  Typically, a system for scanning and / or alignment of scanned data includes general or specific processing modules and memory. The processing module can be based on a single processing device or multiple processing devices. Such a processing device may be a microprocessor, microcontroller, digital processor, microcomputer, part of a central processing unit, state machine, logic circuit, and / or any device that manipulates signals.

The operation of these signals is generally based on an operation command represented in a memory. The memory may be a single memory device or a plurality of memory devices.
Such memory devices (machine-readable media) are read-only memory, random access memory, floppy disk memory, magnetic tape memory, erasable memory, part of system memory, operating instructions in digital format It may be any other device that stores. When a processing module performs one or more functions, it may be done where the memory storing the corresponding operating instructions is embedded in a circuit comprising a state machine and / or other logic circuits Please be careful.

  The invention has been described with reference to specific embodiments. In other embodiments, more than two alignment processes can be used. For example, if the cumulative registration process is interrupted, resulting in multiple cumulative models, a subsequent registration routine can be used to attempt registration between the multiple cumulative models.

  FIGS. 42 through 50 illustrate certain methods and apparatus using three-dimensional scanning data of anatomical structures that can be obtained in the particular manner described herein. The 3D scan data is transmitted to a remote facility for further use. For example, three-dimensional scanning data is designed for designing an anatomical device, manufacturing an anatomical device, monitoring structural changes in living tissue, or storing data belonging to an anatomical structure for a predetermined period of time. Perform a closed-loop iterative analysis of the anatomical structure, perform a repeated consultation of the structure, perform a structure-based simulation, perform a diagnosis on the anatomical structure, or The anatomical structure anatomy used to determine a treatment plan based on the anatomical structure can be represented.

  As used herein, anatomical devices are defined to include devices that actively or passively supplement or modify anatomical structures. Anatomical instruments include orthodontic instruments, which may be active or passive, and include items such as orthodontic bridges, retainers, brackets, wires and positioners However, it is not limited to this. Examples of other anatomical instruments include splints and stents. Examples of orthodontic and prosthetic anatomical devices include removable prosthetic devices, fixed prosthetic devices, and implantable devices. Examples of removable prosthetic devices include dental structures such as dentures, partial dentures, and other body part prosthetic structures such as extremities, eyes, implants included in cosmetic surgery, hearing aids, And a prosthetic device that acts as an artificial body part including analogs such as a frame of glasses. Examples of fixed prosthetic device anatomical devices include caps, crowns, and other non-dental anatomical replacement structures. Examples of implantable prosthetic devices include intraosseous and orthodontic implants and fixation devices such as plates used to hold or reduce breakage.

  FIG. 42 illustrates a flow according to the present invention. Specifically, FIG. 42 illustrates scanning an anatomical structure 4400 with a scanning device 4401 at a facility 4441. In accordance with one aspect of the present invention, any scanner type or method that can produce digital data for the purposes proposed herein can be used. Direct three-dimensional surface scanning indicates that some or all of the anatomical structures can be scanned directly. One embodiment for performing direct three-dimensional surface scanning has been described previously herein. In one embodiment, the scan is a surface scan, whereby the scanning device 4401 detects signals and / or patterns reflected therefrom at or near the surface of the structure 4400. Specific surface scanning methods and apparatus have been previously described herein. Other scanning methods can also be used.

  In general, a surface scan of an anatomical structure is a direct scan of the anatomical structure. Direct scanning means scanning the actual anatomy (in vivo). In an alternative embodiment, an indirect scan of the anatomy can be performed and can be integrated with the direct scan. Indirect scanning means scanning the display of the actual original anatomical structure (in vitro).

  Digital data 4405 is generated at a facility 4441 based on a direct scan of the anatomical structure 4400. In one embodiment, the digital data 4405 represents raw scan data, which is typically a two-dimensional cloud of points created by a scanning device 4401. In another embodiment, the digital data 4405 represents a three-dimensional point model, which is typically created based on a two-dimensional cloud of points. In yet another embodiment, the digital data 4405 represents a three-dimensional initial model. The digital data 4405 may be a composite of multiple independent scans, which may be performed at approximately the same or different points in time and may be performed at the same or different locations. Good.

  The actual data type of the digital data 4405 is determined by the amount of processing performed on the raw scan data at location 4441. In general, the data received directly from the scanner 4401 is a two-dimensional cloud of points. Thus, when no processing is performed at the facility 4441, the digital data 4405 is a two-dimensional cloud of points. The 3D point model and the 3D initial model are generally created by further processing the 2D point cloud.

  Facility 4441 represents the location where a physical scan of the anatomical structure occurs. In one embodiment, the facility 4441 is a dedicated or primarily dedicated location for scanning anatomical structures. In this embodiment, the facility is located where a large number of clients (patients) in need of scanning can easily access. For example, a shopping mall box or a small shopping center location may be dedicated to perform the scan. Such facilities may perform a wide variety of scans, or may be specialized for specific types of scans, such as scans of facial or dental structures. In an alternative embodiment, the scan can be performed by the user at home. For example, a user can be provided with a portable scanner to create scan data that can be used remotely to scan the anatomy and used to monitor the progress of the treatment plan. Or for diagnostic or monitoring or monitoring purposes.

  In another embodiment, the facility 4441 is a place to perform other value-added services related to scanning anatomical structures and creating digital data 4405. Examples of other value-added services include designing or partially designing an anatomical device based on scan data to produce digital data 4405, or such an anatomical device It is mentioned to install. In one embodiment, value added services beyond the creation of digital data 4405 are not performed at the facility 4441.

  Once digital data 4405 is created at facility 4441, the digital data can be provided to the client. Connection 4406 represents digital data being provided to a third party. This step of providing may be performed by the client, by the facility 4441, or by any other intermediate source. In general, the client identifies a third party to which data should be sent. The digital data 4405 can be provided to the facility 4442 physically, ie by mail or courier, or remotely, ie by communication. For example, the digital data 4405 can be physically provided on a non-volatile storage device, such as a portable magnetic medium, a read-only fuse device, or a programmable non-volatile device. In other embodiments, the digital data can be a direct connection, the Internet, a local area network, a wide area network, a wireless connection, and / or any digital information that can be transferred from one computing system to another. Depending on the device, it can be sent to a client or a third party. In certain embodiments, all or some of the digital data needs to be transmitted. For example, if the scan is a patient's teeth and related structures, such as gingiva, a portion of the tooth is transmitted.

  In the particular embodiment of FIG. 42, digital data 4405 received at facility 4442 (receiving facility) is used to design an anatomical device at step 4415. FIG. 43 illustrates a method having two alternative embodiments of step 4415. The first embodiment starts at step 4501 and designs an anatomical structure using a physical model, while the second embodiment starts at step 4511 and starts a virtual anatomical device. Design structures that use the model. A virtual model of an anatomical device is generally a virtual model created by a computer.

  In step 4501, digital data 4405 is used to generate a physical three-dimensional physical model of the anatomical structure. In certain embodiments, the physical model of the scanned object uses numerically controlled processing techniques such as three-dimensional printing, automated milling, laser sintering, stereolithography, injection molding and extrusion. Produced.

  In step 4502, an anatomical device is designed using a three-dimensional physical model. For example, using a physical model, a doctor creates an anatomical device for use by a client. In one embodiment, the anatomical device is custom designed based on a physical model. In another embodiment, a standard orthodontic device is selected based on a physical model of the anatomical structure. These standard devices may be modified as needed to form a semi-custom device.

  In step 4503 of FIG. 43, the manufacture of the anatomical device can be based on a physical model. If a physical model is used, the designing step 4502 and the manufacturing step 4503 are often steps in which the design and manufacturing processes occur simultaneously. In other embodiments, a molding or specification of the desired anatomical device is created and sent to a processing center for custom design and / or manufacturing.

  In an alternative embodiment starting at step 4511, the anatomical device is designed using a virtual three-dimensional model of the anatomical device. A virtual three-dimensional model refers to a model that is created by a numerically controlled device, such as a computer, that either contains digital data 4405 or is created based on digital data 4405. In one embodiment, the virtual 3D model is included as part of the digital data 4405 provided to the design center. In another embodiment, a three-dimensional model is created using digital data and received at step 4511. In another embodiment, an alternative 3D model is created at step 4511 based on the 3D model included as part of the digital data 4405. In another embodiment, multiple 3D models can be bound together from multiple scans. For example, data from multiple scan sessions can be used.

  Further, in step 4511, a virtual anatomical device is designed (modeled) using a virtual three-dimensional model. Virtual devices can be designed using standard or custom design software to identify virtual devices. Examples of such design software include commercially available products such as AutoCAD, Alias, Inc, and ProEngineer. For example, design software can be used to design a virtual crown using a three-dimensional virtual model of an anatomical structure, or from a library of devices that represent real devices, close to custom or standard Or a virtual device can be selected. Following selection of a standard device, customization can be performed.

  In step 4512, the anatomical device can be manufactured directly based on the virtual specification of the device. For example, anatomical equipment uses numerically controlled processing techniques such as three-dimensional printing, automated milling, or laser sintering, stereolithography, and injection and extrusion, and casting techniques. And can be made. It will be appreciated that manufacturing an anatomical device includes partially manufacturing the device and manufacturing the device at multiple locations.

  In step 4426, the manufactured anatomical device is scanned. By creating a virtual model of the manufactured anatomical device, a simulation can be performed to verify the relationship between the manufactured anatomical device and the anatomical structure, thereby A closed loop is provided to ensure proper manufacture of the device.

  In step 4504, the completed anatomical device is sent to a specific location for installation. For example, returning to FIG. 42, the anatomical device is sent to the facility 4444 where installation occurs at step 4435. In one embodiment, the anatomical device is installed at step 4435 by a doctor, eg, a dentist, orthodontist, physician or therapist. In another embodiment, the patient can install several orthodontic devices, such as a retainer or similar positioning device.

  In accordance with certain embodiments of the present invention, the anatomical device is designed or manufactured at a location remote from where the digital data 4405 is received or created. In one embodiment, digital data is received at location 4441. Specifically, the digital data is received by scanning the anatomical structure 4400. Once received, the digital data is transmitted to location 4442, which is remote to location 4441, where the anatomical device is at least partially designed.

The remote location (facility) has just been separated from the other location in some way. For example, the remote location may be a location physically separated from other locations. For example, the scanning facility may be in a different room, building, city, state, country or other location. In another embodiment, the remote location may be a functionally independent location. For example, one location can be used to perform one particular function or set of functions, while another location can be used to perform different functions. Examples of different functions include scanning, design and manufacturing.
Remote locations are generally supported by separate structural infrastructure such as personnel and equipment.

  In another embodiment, the digital data 4405 at the facility 4441 includes a partially designed anatomical device. The anatomical device is further designed at a remote facility 4442. Facility 4442 can determine the final anatomical device, make a diagnosis, form a treatment plan, monitor progress, or at cost, expertise, ease of communication and required response time. Note that one or more remote facilities can be represented that can be used in parallel or sequentially for designing the devices based. An example of a parallel facility is further illustrated in FIG.

  FIG. 44 illustrates another embodiment of the present invention. The flow of FIG. 44 is similar to the flow of FIG. 42 with an additional intermediate step 4615. Intermediate step 4615 indicates that digital data 4405 need not be received directly from the facility 4441 from which the data was scanned. For example, the digital data 4405 can be generated at the first facility (sending facility) by scanning the digital data and providing it to a second facility 4641 (receiving facility) where the intermediate step 4615 occurs. . Once the intermediate step 4615 is complete, the digital data 4405 or the modified digital data that is a representation of the digital data is transferred to a third facility (remote facility) that is remote to at least one of the first and second facilities. You can make a call. During intermediate step 4615, other steps can modify the digital data 4405 before the data is sent to the third facility. For example, the scanned data can be processed to provide a three-dimensional virtual model of the anatomical structure, the data can be added to the digital data, and the scanned anatomical structure 4400 image data, color information Video and / or photo data including diagnostic information, treatment information, audio data, text data, X-ray data, anatomical device design information, and any other data belonging to the design or manufacture of anatomical devices Including. In an alternative embodiment, intermediate step 4615 need not change digital data 4405.

FIG. 45 illustrates an alternative embodiment of the invention in which digital data 4405 is received at facility 4742 for forensic evaluation at step 4741. An example of forensic assessment is victim identification based on scanned anatomical structures.
Such identification is generally based on matching a particular anatomical structure to an anatomical structure contained in the target database, which is a single structure, multiple structures. Can be included. In one embodiment, the target database may be a centrally located database that includes data stored for a predetermined period of time.

  FIG. 46 illustrates an embodiment of the invention in which digital data 4405 or its display is sent to one or more remote facilities 4843 for diagnostic purposes or treatment planning in steps 4844 and 4845. Being able to transmit data for diagnostic purposes can provide three-dimensional information of anatomical structures to other doctors such as specialists without the need for the patient to be physically present. This ability improves the overall speed and convenience of the treatment and the accuracy with which multiple diagnoses can be made in parallel. Multiple opinions can be obtained by allowing digital data 4405 or its display to be sent to multiple facilities for treatment planning and diagnostics. Once a particular treatment plan is selected, any one of the devices identified as part of the treatment plan can be selected for manufacturing.

In the particular implementation of FIG. 46, an estimated price can be obtained from each of the facilities.
The quoted price can be based on a specific treatment identified by the trading partner, where the treatment relates to the anatomical structure. Alternatively, the quoted price can be based on the desired result specified by the trading partner, and the treatment definition and its associated implementation costs are determined by the facility providing the quote. In this way, the patient or patient representative can get a competitive bid in an effective manner.

  FIG. 47 illustrates an alternative embodiment of the present invention, where digital data 4405 or its display is received at a facility 4942 so that the data can be used at step 4941 for educational purposes. Because of the deterministic nature of the embodiments described herein, educational techniques can be implemented in a standardized manner that is not possible using previous methods. Examples of educational objectives include being able to provide self-learning objectives, educational monitoring objectives, and standardized application tests that were not previously possible. In addition, a particular patient's case facts can be matched to other patients' previous or current symptom records, where the symptom records are stored or stored for a predetermined period of time.

  FIG. 48 illustrates an embodiment in which scan data can be stored for a predetermined period of time at step 5001, which occurs at location 5002 for easy retrieval by an authorized person. In certain embodiments, such pre-determined retention is provided as a service, whereby the data is maintained in common, thereby obtaining an independent “golden rule” copy of a common site of digital data. it can.

  FIG. 49 illustrates a particular embodiment of the present invention in which digital data obtained by scanning an anatomical structure is used in a closed loop iterative system. The flow of FIG. 49 is also interactive. In particular, changes in anatomical structures, whether intentional or unintentional, can be monitored and controlled as part of a closed loop system. A closed-loop system according to the present invention is deterministic because the scan data can be measured in three-dimensional space, whereby a standard reference in the form of a three-dimensional model can be used for analysis.

  In step 5101, three-dimensional scan data of the anatomical device is obtained.

  In step 5102, data or a display of data from the scan of step 5001 is transmitted to the remote facility.

  In step 5103, the transmitted data is designed / evaluated. For example, the design for the treatment plan, diagnosis and anatomical device is determined during the first pass through the loop including step 5103. During the next pass through step 5103, the status or progress of the treatment or device is monitored and changes are made as necessary. In one embodiment, monitoring is performed by comparing current scan data against simulated expected results or prior history or a matched symptom record.

  In step 5104, the device or treatment plan is implemented or properly installed. Any manufacturing of the device is performed as part of step 5104.

  It is determined in step 5105 whether an additional pass through the closed loop system of FIG. If necessary, flow proceeds to step 5101. If not necessary, the flow ends. As discussed with reference to FIG. 49, a closed loop feedback loop can exist during any of the steps illustrated in FIG.

The ability to use feedback to verify progress is an advantage over the prior art where doctors rely on one or more of text notes, visual observation of models, and other images. However, these observations were made without a fixed three-dimensional model that the doctor could assure that they were viewed from the same perspective.
By using the visual model described here, a fixed reference can be obtained. For example, one method of obtaining a fixed reference point for an orthodontic structure includes selecting an orientation reference point based on the physical characteristics of the orthodontic structure. The orientation reference point can then be used to map a digital image of the orthodontic structure to the three-dimensional coordinate system. For example, the zonule can be selected as one of the orientation reference points, and the eyelid can be selected as the other orientation reference point. A zonule is a fixed point in an orthodontic patient and does not change or is minimal when changing during treatment. The zonule is the triangular shaped tissue of the upper part of the upper arch gingiva. The heel is a cavity in the palate 68 of the upper arch. Acupuncture also does not change its physical position during treatment. As such, zonules and wrinkles are fixed points in orthodontic patients that do not change during treatment. As such, a 3D coordinate system can be mapped by using these as orientation reference points. Incisor papilla, cleft lip, mid-pupil midpoint, mid-interval midpoint (eg between lips), mid-wing midpoint (eg between sides of nose), nasal fin (eg nose tip), under nose (eg Other physical parts of orthodontic patients including nose and lip joints), midline points, points on bones, fixed bone markers such as implants (eg root canal treatment, screws from oral surgery) Note that attributes are also used as orientation reference points.

  FIG. 50 illustrates that iterative feedback steps can occur within and between any combination of the steps illustrated herein. For example, the interactive and / or interactive loop may be in a single step described between the manufacturing step 4425 and the design step 4415 or with reference to step 4426 of FIG.

  In addition to the many methods that use the scanned data derived in the present application step, many methods are possible to assist in the use of the data. For example, the fee for using such scan data 4405 is the use of the data, the cost of the service provided, the anatomical device being made, or the value added to the device or service based on the data Or may be a fixed or variable charge based on. In addition, it is clear that many other types of charges are imaginable.

  The steps and methods described herein may be performed by a processing module (not shown). The processing module may be a single processing device or a plurality of processing devices. Such processing modules may be microprocessors, microcomputers, digital signal processors, central processing units of computers or workstations, digital circuits, state machines, and / or signals based on operational commands (eg, analog and / or Any device that operates (digital) may be used. The operation of the processing module is generally controlled by data stored in memory. For example, if a microprocessor is used, the microprocessor bus is connected to the memory bus to access commands. Examples of memory include single or multiple memory devices, such as random access memory, read only memory, floppy disk memory, hard drive memory, expansion memory, magnetic tape memory, zip drive memory, and / or Any device that stores digital information can be mentioned. Such a memory device may be local (ie, directly connected to the processing device) or may be physically different (ie, a site connected to the Internet). When a processing module performs one or more functions via a state machine or logic circuit, the memory storing the corresponding operation command is embedded in the circuit comprising the state machine or logic circuit Please be careful.

  Those skilled in the art will appreciate that the specific embodiments described herein provide advantages over known techniques. For example, the anatomical structure being scanned may have one or more associated anatomical devices or instruments. In addition, the present invention provides a deterministic method for diagnosing, treating, monitoring, designing and manufacturing anatomical devices. In addition, the present embodiments can be used to provide an interactive method for communicating between various parties designing and manufacturing prostheses. Such interactive methods can be implemented in real time. The methods described herein allow data to be stored for a period of time in such a way that others can obtain actual information and knowledge derived from the experience of others. Multiple consultants can access the same deterministic copy of information regardless of location, and objective design, manufacturing and / or treatment monitoring, even when multiple independent sites are used Information can be obtained. This embodiment allows for the avoidance of conventional labs used to make anatomical devices. Specifically, the number of support facilities used to create anatomical devices is now countless and remote to the patient. This can reduce the overall costs for doctors and patients. Because the scanning location may be remote from other support locations, the patient does not have to go to the doctor to monitor the status or progress of the device. Overall, the digital data of this embodiment can be stored for a fixed period of time, so that the same replica model can be created at low cost from the golden rule model, resulting in lost or inaccurate data. Reduce the likelihood of being By reducing the amount of time and travel required for the patient to analyze a particular anatomical structure, treatment costs are reduced. Prior art methods increase the accessibility to multiple opinions (estimates, treatment plans, diagnoses, etc.) without further inconvenience to the patients involved. Competition estimates can also be easily obtained using the particular embodiment illustrated.

In the foregoing specification, the invention has been described with reference to specific embodiments.
However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present invention as set forth in the claims below. For example, the digital data may be a combination of a direct scan of the anatomical structure and an indirect scan of the structure. This may occur when a portion of the anatomical structure cannot be viewed by the scanner 4401, so that the type of at least that portion of the anatomical structure is not visible. The mold or model made from the mold is then scanned and “bound” directly into the scanned data to form a full scan. In other embodiments, the digital data 4405 can be used in combination with other conventional methods. In other embodiments, the digital data described herein may be compressed or secured using an encryption method. When encrypted, one or more of the patient, the scanning facility, and the facility or patient representative that can be stored for a period of time can have a cryptographic key for the digital data. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense, and all such modifications are intended to be included within the scope of the invention. In the claims, one or more means-plus-function terms, if any, cover the structures described herein that perform the cited function (s). The singular and plural means plus function terms also cover structural equivalents and equivalent structures that perform the singular and plural cited functions. Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, benefits, other benefits, solutions to problems, and any singular and plural elements that may generate or make clear any benefit or advantage or solution are included in any or all claims. It should not be construed as a critical necessary or essential feature or element.

  Those skilled in the art will recognize that the present invention is advantageous over the prior art because the reference independent scanner is disclosed to incorporate a variable identifier in a direction orthogonal to the projection / viewing plane in certain embodiments. By providing variables in a direction orthogonal to the projection / viewing plane, the distortion of these variables is less than that in a direction parallel to the projection / viewing plane and does not prohibit identification of specific shapes. As a result, greater accuracy of object mapping can be obtained.

FIG. 1 illustrates an object being scanned by a single line according to the prior art. FIG. 2 illustrates an object being scanned by multiple lines according to the prior art. FIG. 3 illustrates the projection and viewing axes associated with the lines of FIG. 2 according to the prior art. FIG. 4 illustrates the object of FIG. 3 from a reference point equal to the projection axis of FIG. FIG. 5 illustrates the object of FIG. 3 from the visual axis of FIG. FIG. 6 illustrates an object having a plurality of projected lines of varying thickness according to the prior art. FIG. 7 illustrates the object of FIG. 6 from a reference point equal to the visual axis shown in FIG. FIG. 8 illustrates an object with varying projected line thickness from the side according to the prior art. FIG. 9 illustrates the object of FIG. 8 from a reference point equal to the visual axis of FIG. FIG. 10 illustrates a system according to the present invention. FIG. 11 illustrates a portion of the system of FIG. 10 in accordance with the present invention. FIG. 12 illustrates the method according to the invention in the form of a flowchart. FIG. 13 illustrates the object of FIG. 3 from a reference point equal to the visual axis of FIG. 3 in accordance with the present invention. FIG. 14 illustrates the object of FIG. 3 from a reference point equal to the visual axis of FIG. 3 in accordance with the present invention. FIG. 15 illustrates an object having a projected pattern in accordance with the present invention. FIG. 16 illustrates a table identifying various types of pattern components in accordance with the present invention. FIG. 17 illustrates a set of unique identifiers in accordance with the present invention. FIG. 18 illustrates a set of repeated identifiers according to the present invention. FIG. 19 illustrates the method according to the invention in the form of a flowchart. FIG. 20 illustrates the method according to the invention in the form of a flowchart. FIG. 21 illustrates the method according to the invention in the form of a flowchart. FIG. 22 illustrates the method according to the invention in the form of a flowchart. FIG. 23 illustrates a series of images to be projected onto an object according to an embodiment of the present invention. FIG. 24 illustrates an image having varying features according to an embodiment of the present invention. FIG. 25 illustrates projected image features reflecting off the surface at different depths, in accordance with a preferred embodiment of the present invention. FIG. 26 illustrates the projected image of FIG. 25 viewed at different depths. FIG. 27 illustrates a dentition object from various viewpoints, in accordance with a preferred embodiment of the present invention. FIG. 28 illustrates a dentition object from various viewpoints in accordance with a preferred embodiment of the present invention. FIG. 29 illustrates a method according to a particular embodiment of the invention. FIG. 30 illustrates a dentition object being scanned from various viewpoints, in accordance with a preferred embodiment of the present invention. FIG. 31 illustrates a dentition object being scanned from various viewpoints, in accordance with a preferred embodiment of the present invention. FIG. 32 illustrates an initial shape for modeling a dentition object. FIG. 33 illustrates a method according to a particular embodiment of the invention. FIG. 34 illustrates a method according to a particular embodiment of the invention. FIG. 35 illustrates a graphical representation of a method for selecting various item points for alignment according to a preferred embodiment of the present invention. FIG. 36 illustrates a method according to a particular embodiment of the invention. FIG. 37 illustrates a method according to a particular embodiment of the invention. FIG. 38 illustrates a method according to a particular embodiment of the invention. FIG. 39 illustrates a method according to a particular embodiment of the invention. FIG. 40 illustrates a method according to a particular embodiment of the invention. FIG. 41 illustrates a method according to a particular embodiment of the invention. FIG. 42 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 43 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 44 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 45 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 46 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 47 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 48 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 49 illustrates a specific flow according to a specific embodiment of the present invention. FIG. 50 illustrates a specific flow according to a specific embodiment of the present invention.

Claims (48)

A method for creating a three-dimensional model of an object,
Receiving a 3D initial model of a first object portion, the 3D initial model including a plurality of initial shapes;
Receiving a 3D point model of a second model part, wherein the 3D point model includes a plurality of points;
Determining a first set of points of the 3D point model that are valid and are believed to overlap a portion of the 3D initial model of the first object portion;
Aligning the 3D point model with the 3D initial model based on the first set of points;
Including methods. The determining step includes
x = 1 to n, where n is the number of points in the 3D point model,
Determining a minimum distance from the point Px to the initial model;
A sub-step including a point Px in the first set of points if Dx is less than a predetermined value;
The method of claim 1 comprising: An apparatus for scanning the surface of an object,
A scanning device including a projection device and an observation device;
A system control device operatively coupled to the scanning device, the system comprising: a first control device for controlling the projection device; and a second control device for controlling the observation device A control device;
A device equipped with a combination. A method for creating a three-dimensional model of an object,
Projecting an image having at least one coded feature with at least one component that varies orthogonally to a plane formed by at least one point of the projection axis and the viewing axis;
Receiving a display of a surface image with an observation device;
Determining a position of a point on the surface of the object based on the at least one encoded feature;
Including methods. A method for creating a three-dimensional model of an object from a reference independent scanning device, comprising:
a) scanning and receiving first data representing two-dimensional object data, wherein the first data includes a plurality of frames;
b) creating a first three-dimensional model from a first frame of the first data;
c) creating a second three-dimensional model from the second frame of the first data;
d) performing a process of aligning and arranging the first three-dimensional model with the second three-dimensional model to create a cumulative model;
e) repeating steps b) to d) until the process of aligning and creating a cumulative model is complete for all frames of the first data;
Including methods. A method of aligning data,
Receiving reference independent data representing an object, wherein the reference independent data includes a plurality of frames F1 to Fn, where n is a positive integer greater than 1, and each of the frames F1 to Fn includes the object Representing the object part of
Performing an alignment between the object part during frame Fx and the object part during frame Fx + 1, where x is an integer between 1 and n−1;
Including methods.   The method according to claim 6, wherein a spatial position of the object part of the frame Fx is unknown and a spatial position of the object part of the frame Fx + 1 is unknown.   The method of claim 6, wherein the spatial position of the scanner is unknown when the frame Fx is captured, and the spatial position of the scanner is unknown when the frame Fx + 1 is captured.   The method according to claim 8, wherein a spatial position of the object part of the frame Fx is unknown and a spatial position of the object part of the frame Fx + 1 is unknown. A method of aligning data,
Receiving a first two-dimensional image representing a first portion of an object, wherein the first two-dimensional image is a reference independent image;
Receiving a second two-dimensional image representing a second part of the object, wherein the second two-dimensional image is a reference independent image;
Determining an original item point used to perform a three-dimensional registration of the second image with respect to the first image;
Including methods.   The method of claim 10, further comprising performing a three-dimensional alignment based on the original item points. Defining alternative item points based on the original item points;
Performing three-dimensional alignment based on the alternative item points;
The method of claim 11, further comprising: A method of aligning data,
Receiving reference independent data representing an object, wherein the reference independent data includes a plurality of frames F1 to Fn, where n is a positive integer greater than 1, and each of the frames F1 to Fn includes the object Representing the object part of
Performing a first alignment using the reference independent data to create a first three-dimensional model;
Performing a second alignment using the reference independent data to create a second three-dimensional model;
Including methods.   The method of claim 13, wherein the first alignment has a first characteristic and the second alignment has a second characteristic.   The method of claim 14, wherein the first and second characteristics are accuracy characteristics.   The method of claim 15, wherein the first characteristic results in a second three-dimensional model that is more accurate than the first three-dimensional model.   The method of claim 16, wherein the step of performing the first alignment takes less time than the step of performing the second alignment.   14. The method of claim 13, wherein the first alignment is a single frame alignment that attempts to align each frame Fx with an adjacent frame Fx-1 when x = 2 ... n. . Receiving digital data, said digital data being based on direct three-dimensional surface scanning of anatomical structures;
Transmitting the display of the digital data to a remote location for designing an anatomical device;
Including methods.   The method of claim 19, wherein the receiving step further comprises scanning the anatomy receiving the digital data directly.   The method of claim 19, wherein the receiving step further comprises receiving the digital data at a receiving facility from a transmitting facility, wherein the receiving facility, the transmitting facility, and the remote facility are different facilities.   20. The method of claim 19, wherein the step of transmitting to design an anatomical device comprises manufacturing the device.   20. The method of claim 19, wherein the receiving step includes the digital data based on reference independent scanning.   24. The method of claim 23, wherein the reference independent scan comprises a multi-frame reference independent 3D scan of an object.   20. The method of claim 19, wherein the receiving step includes receiving the digital data at a facility that does not design the anatomical device. Receiving digital data, said digital data being based on direct three-dimensional surface scanning of anatomical structures;
Transmitting the digital data display to a remote location for forensic use;
Including methods. A processing module having a bus;
A memory having a bus coupled to the bus of the processing module;
In the processing module,
Receiving digital data based on a three-dimensional surface scan of an anatomical structure; and
A memory for storing a command to design an anatomical device by transmitting a display of the digital data to a remote place;
A system comprising: A method for mapping a surface,
Projecting a first image along a projection axis onto an object to be mapped during a first time period, wherein the first image comprises a measurement feature;
Projecting a second image along the projection axis onto the surface during a second time period, wherein the second image identifies the measurement feature of the first image. And having a coded feature used in the method, wherein the first period and the second period are temporally adjacent and substantially mutually exclusive.   29. The encoded feature of claim 28, wherein the step of projecting the second image includes a coded feature that is at least partially substantially orthogonal to a plane formed by one point of the projection axis and the viewing axis. the method of. An apparatus for mapping an object,
Transmitting a first image having a measurement feature along a projection axis to an object to be mapped during a first period and having a feature encoded during a second period along the projection axis. A projector for transmitting two images, wherein the second image is at least partially orthogonal to a plane and is used to identify the measurement feature of the first image; An apparatus in which the period and the second period are adjacent in time and are substantially mutually exclusive. An apparatus for scanning an object,
A data processor that executes instructions;
Multiple directives,
An operation for specifying a first period during which a first image is to be projected along a projection axis onto an object to be mapped, wherein the first image is formed by one point of the projection axis and the visual axis An operation having a measurement feature that is detectable in a direction parallel to the measured plane;
Identifying a second time period during which a second image is to be projected onto the surface along the projection axis, wherein the second image is at least partially orthogonal to the plane and the first image An operation having a coded feature used to identify the measurement feature of an image;
A plurality of instructions that can be executed by the data processor to perform
And the first period and the second period are temporally adjacent and substantially mutually exclusive. A method of scanning,
Providing a patient; and
Providing a scanner using a pattern with coded features orthogonal to the plane of triangulation;
Scanning a portion of a patient using a multi-frame reference independent three-dimensional scan to receive scan data;
Including methods.   The method of claim 32, wherein the portion of the patient is a dentition structure.   34. The method of claim 33, wherein the portion of the patient includes a tooth portion.   33. The method of claim 32, wherein the portion of the patient is not fixed during the scanning step. A method of scanning a part of a patient's mouth,
Receiving scan data having an image with an undercut of a dentition structure, wherein there is no known spatial reference associated with the scan data reference point;
Creating a three-dimensional model of the dentition structure using the scan data;
Including methods. A method of mapping an object,
Projecting a first image along a projection axis onto a surface of the object, wherein the first image is a measurement feature for mapping a first set of surface features having a first characteristic A step including a part;
Projecting a second image along a projection axis onto the surface of the object, wherein the second image is a measurement for mapping a second set of surface features having a second characteristic A step including a feature;
Including methods.   38. The method of claim 37, wherein the step of projecting the second image further comprises projecting the second image simultaneously with the first image.   The first image further includes a first image having a plurality of parallel features, each of the plurality of parallel features is separated by a predetermined distance X, and the second image has a predetermined distance. 39. The method of claim 38, comprising at least one feature separated from any other feature by Y, wherein Y is greater than X.   Determining the second set of surface features based on projecting the first image feature, wherein the second set of surface features uses the first image feature. 38. The method of claim 37, further comprising the step of:   An apparatus comprising a projector used to project an image for mapping an object and projecting a first image and a second image along a projection axis onto a surface of the object, the first image Includes a measurement feature for mapping a first set of surface features having a first characteristic, projecting a second image along the projection axis onto the surface of the object, and The apparatus includes a measurement feature for mapping a second set of surface features having a second characteristic. A method for mapping a surface,
Projecting an image onto a surface along a projection axis to provide a surface image, wherein the image varies in a direction substantially orthogonal to a plane formed by one point of the projection axis and the viewing axis Including a variable feature to:
Receiving a display of the surface image along the visual axis;
Determining a location for a point on the surface based at least in part on the variable feature of the display of the surface image;
Including methods.   43. The method of claim 42, wherein the image is a predetermined pattern. The predetermined pattern is:
A first image portion having a first feature including at least a location at the point of the surface; and a second feature including at least a portion of the variable feature;
A second image portion having a third feature and a fourth feature;
Including
44. The method of claim 43, wherein the determining step further comprises identifying the first feature using the second feature. A method for mapping a surface,
Projecting a pattern onto a surface, the pattern having a plurality of first features and a plurality of second features, each of the plurality of second features being Steps that can be uniquely identified independently of the phase;
Determining a specific feature of the plurality of first features based on the plurality of second features;
Including methods.   46. The method of claim 45, further comprising calculating a relative positional relationship of a point on the surface based on the specific feature. A method of scanning an object,
Projecting a plurality of discontinuous shapes to scan an object, wherein the plurality of discontinuous shapes includes a first discontinuous shape and a second discontinuous shape; Projecting a first encoded feature against a discontinuous shape;
Including methods. An apparatus for scanning an object,
A data processor for executing the instructions;
Multiple directives,
An operation of projecting a plurality of discontinuous shapes to scan an object, wherein the plurality of discontinuous shapes includes a first discontinuous shape and a second discontinuous shape;
Projecting a first encoded feature on the first discontinuous shape;
A plurality of instructions that can be executed by the data processor to perform
Including the device.

Images