NL2017513B1 - Scanning system for creating 3D model - Google Patents

Abstract

A scanning system for creating a 3D model of a surface comprises a first light source configured and arranged to project a first pattern of lines onto the surface. An image capturing device captures an image of the surface with the projected pattern under an angle different from the angle under which the first light source emits light. A computer connected to the capturing device is configured to receive the captured image from the image capturing device and programmed to transform the captured image into a 3D model of the surface. The scanning system furthermore comprises a second light source projecting light of a different wavelength than the first light source. The second light source is configured and arranged to project a coding pattern onto the surface. The coding pattern is projected on the lines of the first line pattern so as to provide a coding to the lines of said line pattern whereby each projected line is identifiable in the captured image.

Description

Patent center

The Netherlands

Θ 2017513

BI PATENT (51) Int. CL:

A61B 5/107 (2016.01) A61B 5/00 (2016.01) G01B 11/25 (2017.01) (21) Application number: 2017513 © Application submitted: 22/09/2016

(Tl) Application registered: (73) Patent holder (s): 29/03/2018 C.C.M. Management B.V. to NUENEN. (43) Application published: - (72) Inventor (s): Franciscus Henricus Alphonsus Gerardus Fey (47) Patent granted: to NUENEN. 29/03/2018 Henricus Rudolphus Kuppens in NUENEN. Bas Johannes van der Linden in HELMOND. (45) Patent issued: Evgenia Balmashnova in EINDHOVEN. 03/04/2018 (74) Agent: ir. H.V. Mertens et al. In Rijswijk.

(54) Scanning system for creating a 3D model (57) A scanning system for creating a 3D model of a surface comprising a first light source configured and arranged for a first pattern of lines onto the surface pattern. An image capturing device captures an image of the surface with the projected pattern under an angle different from the angle under which the first light source emits light. A computer connected to the capturing device is configured to receive the captured image from the image capturing device and programmed to transform the captured image into a 3D model of the surface. The scanning system furthermore comprises a second light source projecting light or a different wavelength than the first light source. The second light source is configured and arranged to project a coding pattern onto the surface. The coding pattern is projected on the lines of the first line pattern so as to provide a coding to the lines of said line pattern each projected line is identifiable in the captured image.

NL BI 2017513

This patent has been granted regardless of the attached result of the research into the state of the art and written opinion. The patent corresponds to the documents originally submitted.

P32824NLOO / CHO

Title: Scanning system for creating a 3D model

The present invention relates to a scanning system for creating a 3D model or a surface, including the system:

- a first light source configured and arranged to project a first pattern of (straight) lines onto the surface;

- an image capturing device for capturing an image of the surface with the projected pattern under an angle different from the angle under which the first light source emits light;

- a computer connected to the capturing device, the computer being configured to receive the captured image from the image capturing device and being programmed to transform the captured image into a 3D model of the surface.

Such a scanning system is known from US 2009/02218784.

The operation of such a scanning system is based on a triangulation principle. A pattern of straight lines is projected from a certain angle on a surface, for example a teeth surface. The projection of the lines on the surface is deformed by the height or depth variation in the surface. The image of these lines on the surface is captured by an image capturing device, e.g. a camera, from another angle as the projection angle. The height or depth variations are visible in the image as a shifting or points of the line in the two dimensional image (X and Y direction). From the captured image or points on the lines a three-dimensional contour can be derived from the computer. This triangulation principle is known from the prior art.

The shifting of points on the lines in the image can lead to segmentation of the individual lines or to disappearing of lines in the image. For an accurate 3D reconstruction it is important that the individual lines of the pattern can be recognized and that the correct sequence of lines in the pattern can be retrieved. If a triangulation is calculated based on the wrong line, a significant error can occur in the 3D reconstruction.

Therefore it is known to provide the lines in the pattern with a coding. In US 2009/0221874 it is disclosed that each line is coded with a unique pattern along the line segment, consisting of a periodic change in the width of the line, a periodic change in color in the line, or a periodic change in greyscale in the line.

-2The present invention has an object to provide a scanning system or the above identified type with an alternative coding or lines in the line pattern.

This object is achieved by a scanning system according to the preamble of claim 1, the scanning system furthermore comprises a second light source projecting light or a different wavelength than the first light source, which second light source is configured and arranged to project a coding pattern onto the surface, the coding pattern is projected on the lines of the first line pattern so as to provide a coding to the lines of said line pattern each projected line is uniquely identifiable in the captured image.

Preferably the coding pattern is a pattern of dots.

With the system according to the invention a line in the pattern is coded by the number of dots, or other shapes projected on a different light source. Adjacent lines could be coded for example with different number of dots, but it is also possible that two adjacent lines have the same number of dots, as long as there is one line with different number of dots every few lines as a reference.

The use of a coding in a color different from the line pattern allows simultaneous generation of the coding and line patterns, which can be analyzed separately. The simultaneous use of coding and line patterns eliminates possible interpretation errors by movement between consecutive captured images. The use of color difference between coding and line patterns allows separation or coding and lines possible via different color channels or a color camera.

In a preferred embodiment the second light source projects the coding pattern on the surface under an angle different from the angle that the first light source projects the first pattern of lines under. In this embodiment the coding pattern is projected onto the surface by the second light source from a different position than the first light source. The second light source thus projects the coding pattern from a different angle onto the surface than the first light source does. The angle of the second light source is also different from the angle of the image capturing device. The projection of the coding pattern on the surface is deformed by the height or depth variation in the surface. This results in the coding pattern, in particular the dots of the coding pattern, shift on the lines of the line pattern. The position at which the dots of the coding pattern are found on the associated line of the line pattern provides specific information about the line allowing to identify the line within the pattern.

-3Thus the lines in the line pattern are identifiable by the number of dots projected on them in combination with the specific position of the dots on the associated lines of the line pattern in the captured image.

In a possible embodiment of the scanning system furthermore comprises a third light source configured and arranged to project a second pattern of lines onto the surface, the lines of the second pattern of lines extend in another direction, preferably perpendicular to the lines of the first pattern of lines. By projecting two line patterns from two triangulation directions, the surface can be more accurately reconstructed even if a surface contains little curvature information.

In the embodiment in which two line patterns are projected onto the surface, the coding pattern is projected onto the surface by one light source, ie the second light source, under an angle different from the angle that the first light source projects the first pattern of lines under and different from the angle that the third light source projects the second pattern of lines under.

In a further embodiment the first light source and third light source are configured and arranged for their respective intermittently pattern. In this way one coding pattern of dots originating from the second light source can be used to code both the lines of the first line pattern originating from the first light source and the lines of the second line pattern originating from the third light source. Hence, only one light source is necessary for coding.

Thus, in a further embodiment the same pattern of dots is projected on the first pattern of lines and on the second pattern of lines.

In a further embodiment the first light source and third light source project their respective line patterns from a different angle relative to the image capturing device. The projections are static from two light sources in two different positions relative to the image capturing device.

In a preferred embodiment the light sources comprise a laser.

In a possible further embodiment the second light source comprises a laser with an optical diffraction element to create the pattern of dots. Preferably the pattern of dots is point symmetrical.

-4 In a possible embodiment the image capturing device is a camera. In particular a color camera, more in particular a CMOS camera can be used to capture the images and be connected to the computer.

The scanning system according to the present invention is particularly suitable for use in an intra oral scanner which is used to make a 3D model of the teeth or a patient for dental restoration or orthodontics.

The invention will be elucidated in the following description with reference to the drawing, in which:

FIG. 1 illustrates schematically a surface scanning system based on triangulation;

FIG. 2A illustrates schematically the triangulation in two directions;

FIG. 2B illustrates schematically projection of line patterns and a coding pattern on a surface from different angles;

FIG. 3A shows a projection or a line pattern and a coding dot pattern on a flat surface; and

FIG. 3B shows a projection or a line pattern and a coding dot pattern on a teeth surface.

In FIG. 1 is shown schematically a surface scanning system including a projector 4 and a camera 5. The projector 4 comprises a light source, in practice for example a laser. The projector 4 projects a pattern of lines on a surface 3. Thereto the projector 4 may include a slide element that creates the line pattern. The slide element may comprise a diffractive element to create the line pattern.

The pattern of lines is captured by a camera 5, which is in particular a color camera, for example a CMOS color camera. The main projector direction and the camera direction are under an angle as seen in FIG. 1. By this arrangement the depth variations in a surface can be captured, because line portions of the pattern are hit on a surface portion further away, e.g. at the surface 3 ”shown in Figs. 1, will be imaged as an image or the line portion shifted upwards in FIG. 1. The image of a surface portion that is situated closer to the projector 1 and camera 2, e.g. at 3, will be seen as a line portion shifted downwards in FIG.

1.

-5From the image of shifted line portions, which leads to curved lines of the pattern of lines a 3D reconstruction can be made of the surface by means of the well-known triangulation method.

To make the 3D reconstruction of the scanned surface, the system includes a computer 6 which is connected to the camera 5. The computer 6 is configured to receive the captured image from the camera and is programmed to transform the captured image into a 3D model or the surface 3.

In order to make a better reconstruction of the scanned surface, even if the scanned surface has little curvature / depth information, it is preferable to use two projectors, each projecting a line pattern from a different angle onto the surface. In Figs 2A and 2B this is schematically shown. As an example a first pattern 1 with vertical parallel lines is projected from a first triangulation angle by a projector 4.1, and a second pattern 2 with horizontal parallel lines is projected from another angle by another projector 4.2.

In a preferred embodiment of the system the projector 4.1 projecting the first pattern 1 or lines and the projector 4.2. projection the second pattern 2 of lines project their respective patterns intermittently on the surface 3 to be scanned.

In the example shown in Figs 2A and 2B the first line pattern and the second line pattern extend perpendicular. However, it is also possible to have other mutual orientations.

The shifting of portions of the lines in the image can lead to segmentation of the individual lines or to disappearing of lines in the image. For an accurate 3D reconstruction it is important that the individual lines of the pattern can be recognized and that the correct sequence of lines in the pattern can be retrieved. If a triangulation is calculated based on the wrong line, a significant error can occur in the 3D reconstruction.

To obviate this problem, the system according to the invention includes another light source 9, which projects a coding pattern in light or another wavelength than the light sources 4,

4.1,4.2 projecting the line patterns. This light source 9 preferably includes a laser and an optical diffraction element that creates a dot pattern. The wavelengths of the light emitted by the line pattern generating laser (s) and the dot pattern generating laser are chosen such that the camera 5 can reliably distinguish the color difference between the lines and the dots.

-6The coding pattern is projected onto the surface by the light source 9 from a different position than the projectors 4.1 and 4.2 can be seen in Fig.2B. The second light source 9 projects the coding pattern from a different angle onto the surface than the projectors 4.1 and 4.2 do. The angle of the light source 9 with respect to the surface is also different from the angle of the camera 5 (cf. Fig.1). The projection of the coding pattern on the surface is deformed by the height or depth variation in the surface. This results in that the coding pattern, in particular the dots 7 or the coding pattern shift on the lines of the line patterns 1,

2. The position at which the dots 7 or the coding pattern are found on the associated line or the line pattern 1.2 provides specific information about the line allowing to identify the line within the pattern.

In the example shown in FIG. 3A the first pattern 1 or vertical lines is shown with a coding pattern 8 or dots 7 having another color than the lines. Each line of the line pattern 1 is coded by the number of dots 7 or the coding pattern 8 projected on it. By this coding each line is identifiable in the captured image or the pattern 1.

In FIG. 3B is shown how the same pattern 1 with the coding dot pattern 8 is projected on teeth, for example if the system is used in an intra oral scanner which is used to make a 3D model of the teeth or a patient. As can be seen the straight and parallel lines of the pattern 1 result in curved lines in the image. Also visible is that some lines partly disappear in the image.

The individual lines of the deformed line pattern of Fig. 3B can be recognized by the coding by means of the dots 7 or the dot pattern projected on the lines. The correct sequence of lines in the pattern 1 can be retrieved by this coding.

The dot pattern is such that it can be used as a coding for the vertical lines of pattern 1 and for the horizontal lines of pattern 2 which are intermittently projected on the surface to be scanned. In a particular embodiment, the pattern of dots is point symmetrical.

It should be noted that for the coding pattern a pattern or dots are described as a preferred embodiment. However, also a pattern or other shapes is possible to project as a coding pattern, e.g. a pattern of rectangles, triangles, ellipses etc. without departing from the invention.

Claims (13)

CONCLUSIONS A scanning system for creating a 3D model of a surface (3), the system comprising: - a first light source (4, 4.1) configured and arranged to project a first pattern (1) of (straight) lines onto the surface (3); - an image capturing device (5) for capturing an image of the surface with the projected pattern at an angle other than the angle at which the first light source emits light; - a computer (6) connected to the image capture device (5), which computer (6) is configured to receive the captured image from the image capture device (5) and is programmed to convert the captured image into a 3D model of the surface (3); characterized in that, the scanning system further comprises a second light source (9) which projects light of a different wavelength than the first light source, which second light source (9) is configured and arranged to project a coding pattern (8) on the surface, wherein the coding pattern (8) is projected on the lines of the first line pattern (1) to provide coding on the lines of the line pattern (1) whereby each projected line is identifiable in the captured image. The scanning system of claim 1, wherein the coding pattern (8) is a pattern of points (7). The scanning system according to claim 1 or 2, wherein the scanning system further comprises a third light source configured and arranged to project a second pattern (2) of lines onto the surface, the lines of the second pattern (2) of lines being in a other direction than the lines of the first pattern (1). The scanning system of claim 3, wherein the lines of the second pattern (2) of lines extend perpendicular to the lines of the first pattern (1) of lines. The scanning system according to claim 3 or 4, wherein the first light source (4) and the third light source are configured and arranged to intermittently emit respective pattern (1; 2). -86. A scanning system according to any of claims 3-5, wherein the first light source (4.1) and the third light source (4.2) project their respective line patterns (1; 2) from a different angle with respect to the image capturing device (5). The scanning system according to any of the preceding claims, wherein the second light source (9) projects the coding pattern onto the surface at an angle other than the angle at which the first light source (4, 4.1) the first pattern (1) of lines or the first and third light sources (4.1,4.2) project the first and second patterns of lines (1,2). A scanning system according to any of claims 3-7, wherein the same pattern (8) of points is projected on the first pattern (1) of lines and on the second pattern (2) of lines. The scanning system according to any of the preceding claims, wherein the first light source (4; 4.1) comprises a laser. The scanning system according to any of the preceding claims, wherein the second light source (9) comprises a laser. The scanning system according to any of claims 3-9, wherein the third light source (4.2) comprises a laser. The scanning system according to any of the preceding claims, wherein the second light source comprises an optical diffraction element to create the pattern (8) of points. Scanning system according to one of the preceding claims, wherein the image recording device (5) is a camera, in particular a color camera, more in particular a CMOS camera. A scanning system according to any one of the preceding claims, wherein the light sources (4, 4.1,4.2) and image recording device (5) are integrated in an intra-oral scanner for scanning a set of teeth. XX. <iX * - - ** ϊ * ^. ^ 5, Jf '* - ^ι Ί 7 .} 3 3 '*