WO2013019405A1

WO2013019405A1 - Adaptive illumination method and apparatus for dental shade matching

Info

Publication number: WO2013019405A1
Application number: PCT/US2012/047096
Authority: WO
Inventors: Rongguang Liang; James MILCH; Jean-Marc Inglese
Original assignee: Carestream Health, Inc.
Priority date: 2011-08-02
Filing date: 2012-07-18
Publication date: 2013-02-07
Also published as: US20130034823A1; EP2739206A1; JP2014521458A; EP2739206B1; JP6030651B2; EP2739206A4

Abstract

An intra-oral imaging apparatus for obtaining an image of a tooth has an image capture apparatus with an imaging sensor that is energizable to obtain image data and one or more optical elements for directing light from the tooth to the imaging sensor. An illumination apparatus has one or more light sources energizable to emit light and a spatial light modulator that is configurable to shape an illumination beam from the emitted light. One or more optical elements relay the shaped illumination beam toward the tooth surface. A control logic processor in signal communication with the imaging sensor obtains image data and in signal communication with the spatial light modulator shapes the illumination beam according to the obtained image data.

Description

ADAPTIVE ILLUMINATION METHOD AND APPARATUS FOR

DENTAL SHADE MATCHING

FIELD OF THE INVENTION

The invention generally relates to imaging of small objects and more particularly relates to an apparatus and method for shape and color shade imaging of the surface of teeth and other structures.

BACKGROUND OF THE INVENTION

Modern restorative dental procedures often require accurate color matching, such as for filling materials and for the fabrication of restorations such as crowns, implants, fixed partial dentures, and veneers. The materials used for these procedures, such as ceramics and other materials, can be skillfully formed and treated to closely match the shape, texture, color and translucency of natural teeth.

One technique for determining and communicating tooth color information is a process referred to as "shade matching" whereby the dentist or technician visually matches a patient's tooth to one of a number of reference shade samples or shade tabs within one or more sets of standardized shade guides. The practitioner who performs the match records the identification of the matching shade tab and conveys that information to the dental laboratory where the restoration or prosthesis is then fabricated. The laboratory then uses its own set of the same shade guides to perform visual color evaluations of the restoration or prosthesis throughout the fabrication process.

The visual shade matching process can be highly subjective and subject to a number of problems. The initial matching procedure is often difficult and tedious, and it is not unusual for the process to take twenty minutes or longer. In many cases, there is no shade tab that perfectly matches the patient's teeth. Tooth color itself results from a relatively complex interaction of reflection, transmission, refraction, fluorescence, and scattering by a variety of organic and inorganic components. It is influenced by variations in tooth pulp volume, dentin condition, enamel composition, and other variations in the composition, structure, and thickness of the dental tissues. One result of this complexity is that color appearance and color measurement are greatly influenced by lighting geometry, spectrum, surrounding colors, and other environmental factors.

As a further complication, color within a single tooth is generally not uniform. Color non-uniformities can result from spatial variations in composition, structure, thickness, internal and external stains, surface texture, fissures, cracks, and degree of wetness. As a result, measurements taken over relatively large areas produce averaged values that may not be representative of a tooth's dominant color. In addition, natural color variations and non-uniformities make it unlikely that a given tooth can be matched exactly by any single shade tab. People are generally particular about the appearance of their teeth.

Understandably, they are quite intolerant of restorations that appear inappropriate in color.

In cosmetic dentistry, the fabrication lab may require additional information in order to more accurately map tooth color in addition to simple shade matching. In practice, the dentist or technician may provide a photograph in addition to a shade tab, so that the fabrication lab can adjust color characteristics over different portions of the tooth. This helps to provide a type of color mapping for subjective use, with information that relates to the shade tab and shows how colors in other portions of the tooth vary from that of the shade tab.

Among problems inherent to dental imaging is the difficulty in providing sufficient light to the tooth of interest. Light-emitting diodes (LEDs) can be positioned near the tooth surface, such as the imaging camera described in U.S. Patent No. 5251025 entitled "Electronic Video Dental Camera" (Cooper).

Fiber optic light might be employed, such as described in U.S. Patent No. 5027138 entitled "Dental Camera System" (Gandrud).

While cameras might illuminate the tooth in some applications, there are drawbacks. One issue relates to the light distribution pattern itself. Referring to Figure 1, there is shown a conventional intra-oral imaging apparatus 10 that has a camera 12 and a light source 14. To obtain an image of a tooth 20, light source 14 is energized to direct light toward tooth 20 and surrounding structures. Light along ray Rl illuminates the object of interest, tooth 20, and is detected at camera 12. The spectral content of light from Rl correctly shows the color of tooth 20 in the image. However, stray light introduces other spectral content that confounds the color data that has been obtained. Light along ray R2, for example, reflects from another tooth or other surface inside the mouth and can introduce some unwanted spectral content. Ray R3 reflects from a gum surface 22 or other tissue surface. Light reflecting from the gum tissue can find its way back to camera 12 and affect the color of the image data. Light along ray R4 reflects from another tooth 21 , not the tooth of interest. Among issues with scattered light are variations in spectral content. Scattered light tends to contribute more in the red parts of the visible spectrum than in the blue, causing some unwanted color shift. It is not practical to mask the scattered light, preventing some portion of this light from reaching the tooth of interest. Still other issues with the conventional arrangement shown in Figure 1 relate to light from specular reflection from the tooth surface itself.

Unwanted spectral content from light that scatters or reflects from surrounding structures is unavoidable with conventional designs and makes it difficult to perform accurate shade matching with existing intra-oral cameras. There is a need for a color matching apparatus and method that provide an image of a tooth or other object, wherein the image data that is obtained more accurately represents the spectral content of a tooth.

SUMMARY OF THE INVENTION

It is an object of the present invention to advance the art of shade matching in intra-oral and other imaging applications. Embodiments of the present invention help to provide an image of a tooth or other feature with reduced spectral content from surrounding tissue or other features.

Illumination that is directed toward the tooth or other feature is cross- sectionally shaped according to segmentation data that is obtained from an image of the tooth and nearby structures. Advantageously, stray illumination is reduced, allowing a more accurate measure of the color of the tooth or other feature of interest to be obtained.

These objects are given only by way of illustrative example, and such objects may be exemplary of one or more embodiments of the invention. Other desirable objectives and advantages inherently achieved by the disclosed invention may occur or become apparent to those skilled in the art. The invention is defined by the appended claims.

According to one aspect of the invention, there is provided an intra-oral imaging apparatus for obtaining an image of a tooth,

comprising: an image capture apparatus comprising an imaging sensor that is energizable to obtain image data and one or more optical elements for directing light from the tooth to the imaging sensor; an illumination

apparatus comprising one or more light sources energizable to emit light and a spatial light modulator that is configurable to shape an illumination beam from the emitted light; one or more optical elements for relaying the shaped illumination beam toward the tooth surface; and a control logic processor in signal communication with the imaging sensor for obtaining image data and in signal communication with the spatial light modulator for shaping the illumination beam according to the obtained image data.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of the invention will be apparent from the following more particular description of the embodiments of the invention, as illustrated in the accompanying drawings. The elements of the drawings are not necessarily to scale relative to each other.

Figure 1 shows a perspective view of a conventional imaging apparatus for intra-oral imaging.

Figure 2 is a logic flow diagram showing a sequence for imaging a tooth for color shade matching according to an embodiment of the present invention. Figure 3 shows a sequence of images and illumination masking used for shade color matching.

Figure 4 is a logic flow diagram showing a sequence for imaging multiple teeth according to an embodiment of the present invention.

Figures 5A and 5B contrast results from conventional illumination against results using the masked illumination according to embodiments of the present invention.

Figure 6 is a schematic block diagram that shows components of an imaging apparatus that uses the conditioned illumination of the present invention.

Figure 7 is a schematic block diagram that shows components of an imaging apparatus that uses the conditioned illumination of the present invention in an alternate embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The following is a detailed description of the preferred embodiments of the invention, reference being made to the drawings in which the same reference numerals identify the same elements of structure in each of the several figures. Where they are used, the terms "first", "second", and so on, do not necessarily denote any ordinal or priority relation, but may be simply used to more clearly distinguish one element from another.

Figures provided herein are given in order to illustrate key principles of operation and component relationships along their respective optical paths according to the present invention and are not drawn with intent to show actual size or scale. Some exaggeration may be necessary in order to emphasize basic structural relationships or principles of operation. Some conventional components that would be needed for implementation of the described embodiments, such as support components used for providing power, for packaging, and for mounting and protecting system optics, for example, are not shown in the drawings in order to simplify description of the invention itself. In the drawings and text that follow, like components are designated with like reference numerals, and similar descriptions concerning components and arrangement or interaction of components already described are omitted.

Embodiments of the present invention address the problem of color shade identification. For this reason, unless otherwise specified, references to a tooth relate to visible surfaces of the tooth for which color match is a concern.

Embodiments of the present invention address the problem of stray light in dental imaging described previously with reference to Figure 1 and condition the illumination to direct light only to the surface of the tooth of interest. The logic flow diagram of Figure 2 shows an imaging sequence 50 of steps used to address this problem, consistent with an embodiment of the present invention. The plan view of Figure 3 then shows the sequence of image capture and image processing that is used for the process described in Figure 2.

In an optional specification step 38 in Figure 2, the technician or practitioner identifies a tooth 20b as the tooth of interest for which a color match is desired. In a broad area illumination step 40, tooth 20b and nearby teeth 20a and 20c and surrounding tissue are illuminated and an aperture definition image 60 is captured in an image capture step 42. As noted previously, due to stray light reflected and scattered from a number of surfaces, this image can be poorly suited for obtaining accurate information for a color match of tooth 20b, but is used instead for segmentation that defines the shape of an aperture in the illumination path. A segmentation step 44 segments tooth 20b within aperture definition image 60, defining an area within the image that then serves as a mask for conditioning the illumination for obtaining a second image. Tooth segmentation is known in the dental imaging arts, and any of a number of tooth segmentation algorithms can be used for this purpose. As an example of tooth segmentation processing, U.S. Patent Application Publication No. 2005/0244794 entitled "Computer- implemented system and method for automated and highly accurate plaque analysis, reporting, and visualization" (Kemp) describes variations on a watershed segmentation algorithm that is known to those skilled in the image processing arts.

Continuing with the sequence of Figure 2, segmentation data about tooth of interest 20b is then used to shape an aperture that provides a mask for conditioning the illumination beam. A variable aperture in the illumination path cross-sectionally shapes the beam so that it illuminates only the surface of the tooth of interest in a masked illumination step 46. This use of masked illumination is shown as masked illumination 62 in Figure 3. A second image capture step 48 then obtains a color shade detection image 64, with the cross- sectionally shaped illumination beam directed only toward the tooth of interest as shown in Figure 3. In one embodiment, color shade detection image 64 is obtained under polychromatic illumination, in one image capture operation. In an alternate embodiment, color shade detection image 64 is obtained as a composite of individual images taken in close sequence, such as a first image obtained using Red illumination, a second image obtained using Green illumination, and a third image obtained using Blue illumination, for example. A color matching step 68 then performs the color matching algorithms for obtaining a suitable color match for the tooth of interest and reports and stores results in an electronic memory.

Optional identification step 38 for identifying the tooth of interest can be executed in a number of ways. Consistent with one embodiment, a laser pointing beam is provided to identify the tooth of interest and to center this tooth in the image that is obtained in image capture step 42. In an alternate

embodiment, aperture definition image 60 displays on a display monitor, allowing operator selection of a particular tooth within aperture definition image 60 as the tooth of interest. A touch screen or other operator interface is provided for identifying the tooth of interest.

It is noted that specification step 38 of Figure 2 can be executed automatically, in part of a looping operation that successively identifies each tooth in aperture definition image 60. Referring to the logic flow diagram of Figure 4, there is shown an automated imaging sequence 90 that automates the

segmentation, masked illumination, and imaging processes in an alternate embodiment, so that each tooth in aperture definition image 60 (Figure 3) is successively selected and imaged without further operator interaction.

Illumination step 40 executes as described with reference to Figure 2, and is followed by first image capture step 42 that obtains an aperture definition image for segmentation analysis in an identification step 70. As part of step 70, multiple tooth surfaces are identified, such as using a process that performs segmentation within multiple portions of the image area. In a selection step 72, a first tooth surface is identified as tooth n = 1. An illumination step 74 shapes the aperture in the illumination path and illuminates tooth n using the shaped illumination beam. An image capture step 76 then captures a color shade detection image of tooth n for use in color shade analysis. A decision step 80 then determines if all teeth in identification step 70 have been imaged. If not, the process indexes to the next tooth, n = 2 in a next tooth selection step 84 and steps 74, 76, and 80 repeat for the next and any subsequent teeth until all of the tooth surfaces identified in identification step 70 have been imaged. Using this sequence, an imaging apparatus can be kept in the same position for obtaining color shade information from multiple teeth in one part of the patient's mouth.

Images in Figures 5A and 5B contrast conventional tooth imaging results with results obtained using the imaging sequences described with reference to Figures 2, 3, and 4. Figure 5 A shows an image of tooth 20 with unwanted specular reflection, as shown by circled areas SI, S2, S3, and S4. This reflected light can scatter and complicate color shade analysis for tooth 20, as described earlier with reference to Figure 1. Figure 5B, on the other hand, shows results using segmentation and masked illumination according to embodiments of the present invention. Tooth 20 is illuminated, and surrounding gums and nearby teeth are not illuminated, to obtain the image for color shade matching.

In addition to masking illumination from other teeth and other specular regions, embodiments of the present invention also detect and respond to highly specular reflection from areas on the tooth of interest. As shown as area S4 in Figure 5A, there is a region of tooth 20, area S4, that exhibits specular reflection. Embodiments of the present invention can handle this type of condition by masking some or all illumination from the area, yielding the results shown in Figure 5B. According to one embodiment of the present invention, light that is directed to this area is reduced, without reducing the light intensity to surrounding portions of tooth 20. In an alternate embodiment of the present invention, that portion of the image that shows specular reflection is simply not used in color shade calculation. It is then assumed that surrounding areas accurately provide color shade information for tooth 20.

Imaging Apparatus

The schematic block diagram of Figure 6 shows an imaging apparatus 100 that is used to obtain aperture definition image 60 and to provide the masked illumination 62 and second image 64 of Figure 3 consistent with an embodiment of the present invention. An image capture apparatus 104 has an imaging sensor 108 that is energizable to obtain image data from a tooth or other object, along with suitable optics for directing light from the tooth, shown as a lens 110 but possibly including a number of refractive or reflective optical elements. An illumination apparatus 120 has one or more light sources 122r, 122g, and 122b, which maybe Red, Green and Blue light emitting diodes (LEDs) or other sources that cooperate to emit, along an illumination path P,

polychromatic light suitable for color matching. A single polychromatic light source, termed a white light source, can alternately be used. A spatial light modulator 124 is configurable as a variable aperture in the illumination path, to provide the light masking function of masked illumination 62 (Figure 3) that shapes an illumination beam for obtaining the second image 64. One or more lenses 128 are provided as optical elements for relaying the shaped illumination beam toward the surface of the tooth or other object of interest. A control logic processor 140, such as a dedicated microprocessor or other programmable logic device, is in signal communication with imaging sensor 108 and with spatial light modulator 124, as well as with light sources 122r, 122g, and 122b. Executing stored program instructions, control logic processor 140 obtains the aperture definition image 60, performs the needed segmentation sequence, and controls spatial light modulator 124 and light sources 122r, 122g, and 122b for obtaining first and second images according to the appropriate sequence described earlier with reference to Figures 2 through 4. An optional display 132 allows tooth selection and other operator control functions. An electronic memory 142, in signal communication with control logic processor 140, is used for storage of the obtained images.

Spatial light modulator 124 that provides an aperture in the illumination path can be any of a number of types of light conditioning element that is capable of forming a mask to shape the illumination beam. In one embodiment, spatial light modulator 124 is a liquid crystal diode (LCD) array that is energizable for selectively transmitting light according to a two-dimensional pattern. Alternate devices available include other types of light valves or light modulator arrays.

Figure 6 also shows an optional laser pointer 144 that is used as a light pointer to direct a laser beam 146 for pointing to the tooth or other object of interest for imaging. The location of laser beam 146 is then detected and used by control logic processor 140 to help isolate the tooth or other object of interest in the field of view for segmentation. For example, the tooth area pinpointed by the laser beam can be used to provide a seed for a segmentation algorithm.

Consistent with an embodiment of the present invention, the function of control logic processor 140 is executed by a dedicated microprocessor that is part of imaging apparatus 100. In an alternate embodiment, control logic processor 140 functions are executed, at least in part, on an external computer, such as on a networked computer or on some other logic processing apparatus.

The schematic block diagram of Figure 7 shows imaging apparatus 100 in an alternate embodiment, in which spatial light modulator 124 is a digital micromirror device or other multi-element reflector array. Control logic processor 140 continues to be in signal communication with imaging sensor 108 for obtaining image data and with reflective spatial light modulator 124 serving as a variable aperture for beam-shaping along the illumination beam path P according to the image that is obtained.

The light that is provided for obtaining the second image that is used for color shade detection is polychromatic light, such as light from red, green, and blue LEDs or light from a single white light source. When multiple light sources are used, segmentation can be performed using an image obtained using monochromatic light, such as by energizing only the green LED in step 40 of Figure 2 to obtain the aperture definition image for segmentation, for example.

Imaging sensor 108 can be any of a number of types of imaging array, such as a complementary metal-oxide semiconductor (CMOS) array or a charge-coupled device (CCD).

By shaping the illumination beam in a cross-sectional manner, embodiments of the present invention direct light only to surfaces of the tooth of interest, reducing the amount of stray light that can reflect from nearby teeth and tissue structures and can adversely impact color shade matching.

The invention has been described in detail with particular reference to a presently preferred embodiment, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention. While embodiments described herein are directed primarily to intra-oral imaging, the apparatus and methods of the present invention can alternately be used for imaging other small objects where obtaining a color match or other detailed data on the objects is of value.

Unless specifically stated otherwise as apparent from the preceding discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" or "determining" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage, transmission or display devices.

Certain aspects of the present invention include process steps and instructions described herein in the form of algorithms or image processing utilities. It should be noted that the process steps and instructions of the present invention could be embodied in software, firmware or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by a variety of operating systems. It should be noted that the term "memory", in the context of the present disclosure, can refer to any type of temporary or more enduring data storage workspace used for storing and operating upon image data in a system using a computer or other logic processor. The memory could be, for example, a long-term storage medium such as magnetic or optical storage. Alternately, the memory could be an electronic circuit, such as random-access memory (RAM) that is used as a temporary buffer or workspace by a microprocessor or other control logic processor device. Display data, for example, is typically stored in a temporary buffer and refreshed as needed in order to provide displayed data. This temporary storage buffer can also be considered to be a memory. Memory can be volatile, non-volatile, or a hybrid combination of volatile and non-volatile types.

Claims

CLAIMS:

1. An intra-oral imaging apparatus for obtaining an image of a tooth, comprising:

an image capture apparatus comprising an imaging sensor energizable to obtain image data and one or more optical elements for directing light from the tooth to the imaging sensor;

an illumination apparatus comprising one or more light sources

energizable to emit light and a spatial light modulator that is configurable to shape an illumination beam from the emitted light;

one or more optical elements for relaying the shaped illumination beam toward the tooth surface; and

a control logic processor in signal communication with the imaging sensor for obtaining image data and in signal communication with the spatial light modulator for shaping the illumination beam according to the obtained image data.

2. The imaging apparatus of claim 1 wherein the imaging sensor is taken from the group consisting of a complementary metal-oxide semiconductor array and a charge-coupled device.

3. The imaging apparatus of claim 1 wherein the spatial light modulator is taken from the group consisting of a digital micromirror device and a liquid crystal device.

4. The imaging apparatus of claim 1 wherein the one or more light sources comprise light emitting diodes of different wavelengths.

5. The imaging apparatus of claim 1 further comprising a light pointer for specifying a tooth of interest.

6. The imaging apparatus of claim 1 further comprising a display that is configured to accept operator commands for segmentation of an image.

7. A method for identi fying a color shade of an obj ect, comprising:

obtaining an aperture definition image that includes the object;

segmenting the object in the aperture definition image to obtain a segmented object;

shaping an aperture in an illumination path according to the segmented object;

for each illumination source in a set of one or more illumination sources, executing the steps of:

(i) energizing the illumination source;

(ii) directing light from the energized illumination source through the aperture and toward the object; and

(iii) capturing a color shade detection image of the object; and determining the color shade according to the one or more color shade detection images of the object.

8. The method of claim 7 wherein obtaining the aperture definition image of the object comprises illuminating the object with polychromatic light.

9. The method of claim 7 wherein the one or more illumination sources are of different respective wavelengths.

10. A method for obtaining an image of an object, the method comprising:

obtaining a first image of the object;

segmenting the object from within the first image to obtain a segmented object;

generating an illumination beam and cross-sectionally shaping the illumination beam in conformance with the segmented object and directing the shaped illumination beam toward the object;

obtaining a second image of the object; and

determining a color shade of the object from the second image.

11. The method of claim 10 wherein the object is a tooth.

12. The method of claim 10 wherein obtaining the first image of the object is performed using monochrome light and wherein obtaining the second image of the object is performed using polychromatic light.

13. The method of claim 10 wherein cross-sectionally shaping the illumination beam comprises directing the illumination beam toward a spatial light modulator.

14. The method of claim 10 wherein the object is a first object and further comprising:

segmenting a second object from within the first image to obtain a segmented second object;

generating the illumination beam and cross-sectionally shaping the illumination beam in conformance with the segmented second object and directing the shaped illumination beam toward the second object;

obtaining a third image that comprises the second object; and

determining the color shade of the second object from the third image.