KR20210017096A - ToolKit for Matching natural teeth color - Google Patents

Abstract

The present invention relates to a tone guide toolkit for natural teeth color matching. More specifically, the tone guide toolkit includes generation of a transformation matrix according to a spectrum reflection data set for statistically effective tooth sampling. Illumination is directed toward a tooth-related object by one wavelength band at a time over at least a first wavelength band, a second wavelength band, a the third wavelength band. Image data values corresponding to at least the first wavelength band, the second wavelength band, and the third wavelength band are obtained for each of a plurality of pixels in an imaging array. The transformation matrix is adapted to form color mapping by generating a set of visual color values for each of the plurality of pixels in accordance with the obtained image data values and in accordance with image data values obtained from a reference object in at least the first wavelength band, the second wavelength band, and the third wavelength band. The color mapping may be stored in an electronic memory.

Description

ToolKit for Matching natural teeth color

The present invention relates to a color guide tool kit for natural tooth color.

Modern restoration dental treatments frequently require accurate color matching for fillings and for fabrication of restorations such as crowns, implants, fixed partial dentures and veneers.

Materials used for this treatment, such as ceramics and other materials, can be skillfully formed and processed to closely match the shape, texture, color, and translucency of natural teeth.

A technique widely used to determine and communicate tooth-tone information is a process called "shade matching", whereby a dentist or technician can refer to multiple reference 　 hue samples or 　 tint tabs within one or more sets of standard 　 hue guides. shade tab) and the patient's teeth can be visually matched.

The practitioner who performs such 　matching records the identification of the 　matched 　color tab and transfers the information to the dental laboratory, where restorations or prostheses are manufactured.

At this time, the dental laboratory uses the same color guide set in the laboratory to visually evaluate the color of the restoration or prosthesis throughout the manufacturing process.

The visual 　 hue 　 matching 　 process is very subjective and can have many problems.

The initial 　matching 　 procedure is often difficult and tedious, and it is not uncommon for this process to take more than 20 minutes. In most cases, there is no color tab that perfectly matches the patient's teeth.

The problem in accurately modeling the 　 color of a tooth is more complicated than obtaining a close 　 color 　 matching using a 　 hue tab. The common intrinsic drawbacks and limitations of apparatus-dependent and visually-dependent 　 tonal 　 matching systems can be more fully understood by considering the difficulties involved in matching the appearance of a human 　 tooth.

Tooth 　 color 　 itself is due to the relatively complex interaction of reflection, transmission, refraction, fluorescence and scattering by various organic and inorganic components.

This is affected by changes in tooth pulp volume, dentin state, enamel composition, and other 　tooth 　 composition, structure and thickness.

One consequence of this complexity is that 　 color 　 appearance and 　 color 　 measurements are greatly influenced by lighting arrangement, spectrum, ambient color, and other environmental factors.

Another annoying problem is that the color of 　 within a single 　 tooth 　 is generally not uniform. Color unevenness can be caused by spatial changes in composition, structure, thickness, internal and external stains, surface texture, crevices, cracks and moisture levels. As a result, measurements taken over a relatively large area yield an average value that may not represent the dominant color of the tooth.

Other than that, it is unlikely that any 　tooth would be exactly matched with any single 　toned tap due to natural 　 color 　 changes and non-uniformities.

This means that we need a method to convey the 　 color distribution within 　 teeth, not just the average 　 color of 　 teeth.

Also, 　tooth 　 color is rarely uniform between 　tooth and 　tooth. Therefore, the ideal color of the restoration may not be visually matched with the color of the adjacent 　 tooth 　 or any other single 　 tooth in the patient's mouth.

In addition, people generally pay special attention to the appearance of their teeth. Not surprisingly, they cannot tolerate any restorations that appear inadequately colored.

In cosmetic dentistry, in addition to simple 　 tones 　 matching 　, manufacturing laboratories often need additional information in order to more accurately map 　 teeth 　 colors.

In fact, dentists or technicians can provide photos in addition to the hue tab so that the fabrication lab can adjust the color characteristics of various parts of the tooth.

This helps to provide some type of personal use 　 color 　 mapping with information related to the hue tab and showing how the color of the other parts of the tooth differs from the hue tab's color.

It is often difficult to determine which tabs are most closely matched (or vice versa) and to provide accurate information of the color changes to the tooth surface. Often, the practitioner decides that the patient's teeth are particularly difficult to match, so they ask the patient to go directly to the laboratory where the restoration will be made.

Here, experienced laboratory personnel can perform 　color 　 matching 　 and 　 color 　 mapping.

In most cases, the patient may even need to return to the dentist and laboratory twice, three times, or even more because the color of the prosthesis has been fine-tuned by sequentially adding ceramic or other colored materials.

In many cases, which are estimated to be nearly 10% in some dental prosthesis, the visual 　 color matching 　 processing still fails and the fabricated prosthesis is rejected by the dentist or patient because of color or visual coordination.

Considering the relatively difficult color/matching process and the complexity of color/mapping, the high failure rate is not surprising.

It is always difficult to visually evaluate the relatively small 　 color 　 difference, and situations in which 　tooth 　 color 　 evaluation should be performed include local chromatic adaptation, local brightness adaptation, and lateral-brightness adaptation. It is prone to many complex psychophysical effects.

In addition, the hue tab provides at best a conditional (i.e., non-spectrum) matching to the real tooth, so the matching is sensitive to light sources due to normal changes in human's color sense, for example observer metamerism. Easy to change

In accordance with the need for improved 　 color 　 matching 　 and 　 color 　 mapping in dental applications, many approaches have been tried. Some solutions to this problem are of the following types.

(i) RGB-based devices. With this approach, an image of the entire tooth is captured under white light illumination using a color sensor.

Using color correction conversion, a tristimulus value is calculated for the entire area of the 　 tooth 　 surface from the RGB (Red, Green, Blue) values of the sensor's 3-color channel.

　Color analysis by RGB-based devices strongly depends on the image quality of the captured image and requires robust correction, and may require the use of the same camera for 　color matching of teeth and prosthetic devices.

This requirement may be due to the calibration of the camera itself as well as the 　 color 　 pre-processing performed within the camera to provide RGB data, and the 　 color 　 pre-processing can vary considerably between cameras, even for cameras of the same manufacturer.

Maintaining precision tends to be difficult due to the conditional isochromicity in which the measured color is highly dependent on the light source.

This is particularly cumbersome because 　tooth measurement and imaging are generally carried out under conditions significantly different from natural lighting conditions.

In general, conventional methods of using a 　 color 　 filter at the light source end or the sensor end may not be so desirable, because they are subject to the limitations of the filter itself.

Therefore, there is a need for an improved measuring device that provides 　tooth 　 hue matching 　 and 　 mapping as a procedure to be performed directly with high precision without expensive or complicated parts.

It is an object of the present invention to improve the technique of 　 tones 　 mapping in dental applications.

With this object in mind, the present invention provides an apparatus and method for obtaining spectral reflection data from a multicolor image of a tooth without the complexity of using a spectrophotometer.

An advantage of the present invention is the use of an imaging array to obtain spectrophotometric measurements for the entire image. Accordingly, the embodiment of the present invention can be easily applied to an intraoral camera. In addition, the provided output spectral luminosity 　 color 　 data is not subject to conditional isochromaticity that affects the solution using colorimetric analysis and RGB 　 color matching. The approach used in the embodiment of the present invention enables accurate mapping of the color of the tooth, including the change of color and color for various parts of the tooth, by obtaining spectral reflection data for each pixel of the tooth image.

These objects are provided as illustrative examples only, and these objects may be examples of one or more embodiments of the present invention. Other desirable objects and advantages essentially achieved by the present disclosure may arise and will become apparent to those skilled in the art. The invention is defined by the appended claims.

According to an aspect of the present invention, there is provided a method for generating a 　 color 　 mapping 　 for a tooth-related object at least partially executed by a control logic processor, comprising: generating a transformation matrix according to a spectral reflection data set for statistically effective 　 tooth sampling; Irradiating illumination in one wavelength band at a time toward a tooth related object over at least a first wavelength band, a second wavelength band, and a third wavelength band; Obtaining image data corresponding to at least each of the first, second, and third wavelength bands for each of a plurality of pixels in the image array; By applying the conversion matrix, the visual color values for each of the plurality of pixels at least according to the image data obtained in the first, second and third wavelength bands and according to image data values obtained from a reference object Forming a 　 color 　 mapping by generating a set of; And storing the 　 color 　 mapping in an electronic memory accessible by a computer.

The present invention can improve the technique of tone mapping in dental applications.

The above and other objects, features, and advantages of the present invention will become apparent from the following more detailed description of the embodiments of the present invention as illustrated in the accompanying drawings. The elements in the drawings are not necessarily to scale with respect to each other.
1 is a schematic diagram showing the arrangement of parts in a conventional image-based device for obtaining color measurements for teeth.
2 is a schematic diagram showing a previous method of using color correction conversion to obtain a tooth color value.
3 is a schematic block diagram showing a color mapping device according to an embodiment of the present invention.
4A is a graph showing intensity measurements associated with specific wavelengths.
4B is a graph showing typical spectral characteristics for a light source used in an embodiment.
4C is a graph showing spectral response characteristics for a broadband sensor array in one embodiment.
5A is a schematic diagram showing tricolor stimulation value data obtained for each pixel using conventional color matching.
5B is a schematic diagram showing spectral reflection data obtained for each pixel using the apparatus and method of the present invention.
6A is a logic flow diagram of obtaining visual color mapping data from a tooth.
6B is a logic flow diagram for obtaining spectral reflection data from a tooth in an alternative embodiment.
7 is a logic flow diagram showing a process of forming a conversion matrix according to an embodiment of the present invention.
8 is a logic flow diagram showing a process of forming a transformation matrix according to an alternative embodiment of the present invention.
9 is a logic flow diagram showing a process of forming a transformation matrix according to another alternative embodiment of the present invention.
10 is a logic flow diagram showing a process of forming a transformation matrix according to another alternative embodiment of the present invention.

The following is a detailed description of a preferred embodiment of the present invention, with reference to the drawings, in which like reference numerals designate like structural elements in several drawings.

In the context of this application, the term “narrowband” is used to describe an LED or other illumination light source that emits most of the output light over a narrow range of wavelengths, such as 20-50 nm wide.

The term “broadband” is used to describe an optical sensor that exhibits high sensitivity to incident light over a wide wavelength range of at least about 400 nm to about 700 nm.

Because this type of sensor responds to light but does not differentiate between colors, it is often referred to as a "monochrome" sensor or slightly incorrectly a "black and white" sensor.

In the context of this application, the term "pixel" for "pixel element" has the usual meaning as the term is understood by a person skilled in the image processing technology.

The electronic image of the object is captured by an array of light-receiving elements, each of which provides a signal that forms one pixel of the image data.

The drawings shown and described herein are provided to illustrate the principle of operation according to the present invention and the relationship of the operation and component parts along each optical path, and are not drawn to show the actual size or scale.

Some exaggeration may be necessary to emphasize the basic structural relationships or operating principles.

In order to simplify the description of the invention itself, some conventional components that will be required to achieve the described embodiments, such as for example various types of optical tables, are not shown in the drawings.

In the following drawings and texts, similar components are designated by like reference numerals, and similar descriptions regarding the arrangement or interaction of the components and the previously described components are omitted.

When they are used, terms such as "first", "second", etc. do not necessarily denote any ordinal or ordinal relationship, but are simply used to more clearly distinguish one element from another.

The terms "color" and "waveband" are generally synonymous as used in connection with the present disclosure.

For example, a laser or other solid state light source does not mean the maximum output wavelength (635nm, etc.) or its wavelength band (630-640nm, etc.), but a general color such as red.

The term "set" as used herein refers to a set in which the concept of a group of elements or members of a set is not empty, as is widely understood in elementary mathematics.

Unless explicitly stated otherwise, the term "subset" as used herein refers to a non-empty, true subset, i.e. a subset of a large set of one or more members.

In the set S, the subset may include the complete set S. However, a "true subset" of set S is strictly included in set S and excludes at least one member of set S.

In the context of the present disclosure, the term "tooth-related material" refers to an object, material or other element for oral use or application, and includes teeth, crowns, dentures, orthodontic braces, and other supports. , Bridges, fillers, 　 color 　 matching 　 tabs, etc.

In contrast to some attempts to characterize the tooth color, the devices and methods of the present invention take into account a combination of factors that influence the color color measurement and complicate the task of accurately characterizing the color color.

Certain embodiments of the present invention specify and compensate for variable factors such as illumination wavelength and detector response characteristics to extract accurate color data.

To do this, the approach used in the embodiment of the present invention is substantially independent of the spectral response of the measuring device for each pixel in the image of a tooth or other ``tooth'' related object, and is applied to illumination over an arbitrary combination of wavelengths. Obtain spectral reflection data that can be used to provide a measure.

As a result, the data obtained for 　 tooth 　 color 　 mapping in the present invention can be used to reproduce the visual 　 color of an object when viewed under an arbitrary light source taken from a set of available light sources having a known spectral distribution.

And the 　color 　mapping generated for the 　tooth 　 related object can be used to generate the displayed image, or it can be used for comparison of the 　color 　mapping 　data for other objects or materials such as crowns or other dental prostheses or fillings.

The 　 color 　 mappings created for the tooth-related material can also be used, for example, to design and form dental prosthetic devices.

The color mapping for teeth or other teeth-related objects can consist of a significant amount of data and is typically stored as a data file in a memory accessible by a computer.

Referring to Fig. 1, a schematic block diagram of a conventional imaging apparatus 10 for obtaining 　tooth 　 color 　 data is shown. The light source 12 irradiates light onto the tooth 20.

In addition, the capture module 18 performs image capture and provides image data to the color reproduction module 22. The output is a set of visual and color values corresponding to points on the tooth surface.

The basic arrangement of Fig. 1 is used for each of the image-based color measurement approaches described earlier in the background section of this specification.

2 shows a schematic diagram for achieving the system described in the '830 patent by Giorgianni et al. cited above, for example.

In the imaging apparatus 30, the illumination light source 12 provides white light illumination to the tooth 20.

Lens 32 irradiates reflected light to sensor 34 providing the corresponding red, green, and blue color for each pixel.

And 　 color correction conversion 36 generates a visual 　 color value as an output for all image points.

In contrast to the approaches shown in Figs. 1 and 2, the apparatus and methods of the present invention provide a 　color 　 mapping apparatus using spectral reflection based 　 color 　 reproduction.

This approach is advantageous over RGB-based and colorimetric analysis devices by providing an intrinsically more accurate data set for the true color characteristics of the tooth.

Advantageously, this method is not conditional, otherwise the measurement result will depend on the light source of the measurement system. Unlike typical spectrophotometric measurements, the apparatus and method of the present invention obtains spectral data for each pixel in a tooth image. In addition, this information is obtained using an imaging array rather than a photodetector. Although the imaging array used in the embodiment of the present invention is a monochromatic sensor, a color sensor can be used as an alternative.

The apparatus and methods of the present invention not only obtain accurate 　 color 　 measurement for a single pixel or adjacent pixel group, but also provide information suitable for accurate 　 color 　 mapping for the entire image of 　 tooth related objects because an image pickup device is used. By obtaining spectrum measurement data using an imaging array, embodiments of the present invention obtain measurement results that cause spectral data to be generated for each pixel of a tooth image.

1,

2,

3,

4)를 갖는 칼라 LED로서 도시된 다수의 협대역 광원(14b, 14g, 14y, 14r)으로 구성된다. 도 3의 실시형태에서, 일 예로서 4개의 LED가 도시되어 있는데, 어떤 개수의 서로 다른　색도 될 수 있으며 각　색에 대하여 하나 보다 많은 LED 또는 그 외의 광원이 될 수 있다. 본 실시형태에서는 광대역 촬상 센서 어레이(44), CCD(전하결합소자) 또는 CMOS(상보형 금속산화막 반도체) 촬상 어레이는 각 협대역 광원으로부터의 반사광에 대응하는 각 픽셀에 대한 출력값의 집합을 제공한다. 참조 타겟(28)은 후에 설명하는 바와 같이 시스템에서의 강도 변동을 보정하는데 사용되는 참조강도 측정결과를 얻기 위해 선택에 따라서 제공된 물체이다. 일 실시형태에 있어서, 참조 타겟(28)은 공지의 스텍트럼 반사 특성을 갖는 그레이 패치다. 참조 타겟(28) 및　치아(20)는 둘 다 촬상 렌즈(32)의 시야 내에 있다. 대체 실시형태에 있어서, 참조 타겟(28)은 이미지 필드에서 나타나지 않는 물체이지만 참조 및 교정 데이터를 얻는데 사용되는 별개의 장치이다.Referring to the schematic diagram of FIG. 3, the color measurement and mapping device 40 uses an illumination device 24 capable of providing light of a separate color. In one embodiment, the lighting devices 24 each have a wavelength band (

One,

2,

3,

It consists of a number of narrowband light sources 14b, 14g, 14y, 14r, shown as color LEDs having 4). In the embodiment of Fig. 3, as an example, four LEDs are shown, which can be any number of different colors and can be more than one LED or other light source for each color. In this embodiment, the broadband imaging sensor array 44, a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor) imaging array provides a set of output values for each pixel corresponding to the reflected light from each narrowband light source. . The reference target 28 is an object optionally provided to obtain a reference intensity measurement result used to correct intensity fluctuations in the system, as described later. In one embodiment, the reference target 28 is a gray patch with known spectral reflective properties. Both the reference target 28 and the tooth 20 are within the field of view of the imaging lens 32. In an alternative embodiment, the reference target 28 is an object that does not appear in the image field, but is a separate device used to obtain reference and calibration data.

1,

2,

3,

4)의 각각에 대응하여 하나의 강도를 나타낸다. 이들 지점을 이용하여 완전한 스펙트럼 반사곡선을 생성하는데 통계 기반 변환 매트릭스(50)가 사용된다. 이 결과는　치아　이미지의 개개의 픽셀에 대한 스펙트럼 반사 곡선이다. 이들 데이터는 단지 직접 삼색자극치를 측정하려 하거나 또는 RGB로부터 예를 들어　색채-채도-휘도 값(hue-saturation-brightness value; HSV) 또는 Commission Internationale de L'Eclairage L*a*b* (CIELAB)　색공간 같은 표준　색공간으로의　색변환을 수행하는 다른 방법과 비교하여　색을 보다 정확히 나타내는　색조　맵핑을 제공한다. 또한 도 3에는 제어논리 프로세서(38)가 도시되어 있는데, 이는 제어논리 처리를 수행하며 그리고 이들 처리 기능을 실행하고 중간 및 최종 결과를 저장하도록 지원 컴퓨터가 억세스할 수 있는 전자 메모리 부품을 갖고 있다. 이런 부품들은 촬상 기술에서의 당업자에게 잘 알려진 것이다.In order to obtain spectral data using the device shown in FIG. 3, LEDs or other light sources can be continuously operated one color group at a time according to the color group, and the corresponding measurement result of the reflected light is obtained by the imaging sensor array 44. Lose. As shown in the form of a thumbnail in FIG. 3, a plurality of points are effectively provided on the graph for each pixel of the tooth image. In this example, the wavelength band (

One,

2,

3,

4) represents one intensity corresponding to each of. A statistics-based transform matrix 50 is used to generate a complete spectral reflection curve using these points. This result is a spectral reflection curve for individual pixels of the tooth image. These data are either intended to measure trichromatic values directly or from RGB, for example hue-saturation-brightness values (HSV) or Commission Internationale de L'Eclairage L*a*b* (CIELAB) colors. Compared to other methods of performing color conversion to a standard color space such as space, it provides a color tone mapping that more accurately represents colors. Also shown in Fig. 3 is a control logic processor 38, which has an electronic memory component accessible by a supporting computer to perform control logic processing and to execute these processing functions and store intermediate and final results. These components are well known to those skilled in the imaging technology.

The graph of FIG. 4B shows an example in which four individual intensity measurement results were obtained for one pixel, one for each light source 14b, 14g, 14y, and 14r corresponding to the wavelengths λ1, λ2, λ3, and λ4, respectively. This yields four points on the graph of intensity versus wavelength as shown.

Making accurate measurements requires knowledge of the various energy levels and measurement sensitivity in the system. As shown in Fig. 4B, each narrowband light source, in this embodiment, an LED provides an output intensity over a limited range. The wavelength bands λ1, λ2, λ3, and λ4 are identified by the nominal wavelength at which these intensity curves are the highest.

The wideband sensor array 44 can be any type of sensor array that measures the amount of reflected light corresponding to a component of the illumination device 24 that operates to provide measurements for each pixel. In one embodiment, the broadband sensor array 44 has a wide spectral sensitivity characteristic over a visible wavelength range. This is clearly different from the "color matching function" of the eye or the spectral response of a color sensor array, both of which have important values only in the band of isolated visible wavelengths. For example, a monochromatic CCD imaging array is shown in Figure 4c. It can have a typical quantum efficiency curve as shown in the example of. This device measures the amount of reflected light corresponding to each light source (14b, 14g, 14y, 14r) when illuminated, and an output indicating the received detected light intensity. Since the peak wavelength and bandwidth of the LED light source is known, the measurements can be easily related to the wavelength without the need for a color filter array (VFA) or other filtering component.

In contrast to some color matching solutions, the method and apparatus of the present invention efficiently generate an expected spectral reflection data value R _{(tooth) that} can be used to accurately profile the color of the tooth surface for each pixel of the tooth image. It provides storage spectrometry or spectral reflection color mapping. Compared to conventional methods that rely on RGB calibration conversion or colorimetric measurements, the methods of the present invention can create a mapping containing a significant amount of data for each pixel of a tooth image. This is schematically shown in FIGS. 5A and 5B. 5A shows color data collected for a single pixel P using a conventional color measurement approach. In the illustrated example, a single data value is provided for each of the red, green and blue color planes. Meanwhile, FIG. 5B shows the characteristics of the spectral reflection mapping data obtained for each pixel using the method of the present invention. Here, each pixel P effectively has an associated spectral reflection curve that provides a significant amount of information to the actual color content from which the tricolor stimulus values (X, Y, Z) or other color data is extracted. In fact, after this processing, only a small amount of data for each pixel P can actually be stored in the memory, and thus the spectral reflection curve itself can be reconstructed using the stored coefficient matrix obtained as described above. And in the memory storage device, each pixel may be associated with a set of spectral reflection values, optionally also having a link to a matrix or other identifier capable of regenerating a full spectral reflection curve for that pixel. As another method, a tricolor stimulus value, a CIELAB value, or other values in a standard color space calculated for each pixel P may be stored.

One beneficial result of having spectral reflection data values is an improved 　 color 　 match between 　 teeth and related objects, such as between teeth and prosthetic devices or materials. Using the data collection and processing sequence of the present invention, color matching can be achieved using any of a number of mathematical techniques. In one embodiment, the spectral reflection curve for 　 tooth 　 relation (A) is compared with the spectral reflection curve for 　 tooth 　 relation (B), and the difference between the two curves is a fit or an approximation of other metric It is evaluated for sex. For example, the area of overlap between two curves over different wavelength ranges can be evaluated. In another embodiment, a tricolor stimulation data value or a CIELAB color value is obtained and compared for each tooth-related object.

The general theory and principle of obtaining spectral reflection data for pixels of an image using multiple narrowband measurements by a multi-wavelength camera is described in "Analysis Multispectral Image Capture" (Fourth Color Imaging Conference: Color Science Systems and Applications, 1996, pp. 19-22), validated and described by Peter D. Burns and Roy S. Berns. These researchers obtained image data from a set of color reference samples at different wavelengths using a white light source with seven interference spectral filters continuously switched to predetermined positions. And by applying Principal Component Analysis (PCA), a set of scalar values that can be used to reconstruct spectral reflection data for unknown 　 color 　 patches is provided using a series of similarly obtained 　 color 　 images. . The work of these researchers has shown the possibility that a small number of measurements can be used to obtain accurate spectral reflection data, but the system used will not be practical for 　tooth 　color 　mapping 　 and other 　tooth objects and materials. The design of a multispectral camera using a plurality of switchable interference filters is not compatible with the size and access restrictions of the tooth image.

The logic flow diagram of FIG. 6A shows the steps executed by the control logic processing parts in the 　 color 　 measurement and 　 mapping apparatus 40 of FIG. 3 to generate a visual 　 color value for a 　 tooth in an embodiment of the present invention. The transformation matrix generation step S100 is initially performed using spectral reflection data from statistically effective sampling of the tooth as described later. The looping procedure is performed once per source color, in this case for each LED 　 color 　 or other narrow-band illumination source. In the illumination step (S110), an LED or other narrow-band illumination light source operates. The image capturing step S120 obtains an image of the reference target 28 at the given illumination and optionally. The resulting image is composed of light reflected from the tooth captured for each pixel in the image capturing step S120. In the value acquisition step S130, an array of image values measured from the broadband sensor array is acquired and stored. The loop consisting of step S110, step S120, and step S130 is repeated until each color group of LEDs or other narrowband light sources operates. The final result is a set of image values for each pixel of the image for 　tooth 　 or other 　tooth related objects.

에 근거하여 추가의 신호값을 포함할 수 있는데, 여기서 e 는 실수이다. 다른 방법으로서, 이미지 샘플 측정결과의 변환된 값은 지수가 [0.3-0.4]의 범위인 간단한 지수법칙(power law)을 이용하여 계산할 수 있다.As a result of the value acquisition step 130, a set of N signals for N image wavelength values, s={s ₁ , s ₂ , ..., s _k }. As another method, step (S130) is a conversion of the signal value, for example, polynomial conversion,

May include additional signal values based on, where e is a real number. Alternatively, the converted value of the measurement result of the image sample can be calculated using a simple power law with an exponent in the range [0.3-0.4].

Continuing the logic flow of FIG. 6A, the matrix applying step S160 applies the transform matrix to the measured image values. In step S174 of obtaining a visual 　 color value, a visual 　 color value is generated from image data according to the transformation matrix. And the generated 　 color 　 mapping is stored in the electronic memory by the control logic processor 38 in the storage step (S190). The memory itself may be a random access storage device for short-term storage, or an optical or magnetic storage device for long-term storage. If necessary, the memory device as shown in Fig. 6A becomes a temporary "workspace" and the control logic processor 38 is used only during the period during which color matching or comparison is performed, or, for example, during image display. Used by Alternatively, in the storage step S190, for a long period of time, a part of the database of 　 color 　 mapping indexed according to the lighting type, for example, may be stored for a long time. Once the 　 color 　 mapping is formed and available in a memory accessible to the computer, it can be used, for example, for comparison with the 　 color 　 mapping data for prosthetic devices.

The logic flow diagram of FIG. 6B shows the expansion steps for creating a color-to-color mapping in one embodiment. Steps S100 to S160 are similar to those described with reference to FIG. 6A. After applying the transform matrix in step 160, step S170 generates spectral reflection data or curves using the pre-calculated principal components and the blending coefficient obtained in step S160. Then, in step S180 of extracting 　 color values, visual 　 color values for 　 color 　 mapping are extracted based on the spectral distribution of the desired light source.

Advantageously, the spectral reflection for teeth and other dental materials works normally and follows a characteristic pattern. Spectral reflection curves for teeth and dental materials show a smooth and consistent shape with a change only within a relatively limited range. This feature makes it possible to use a statistical technique that provides the tools necessary for accurate 　 color 　 matching than was possible using a conventional approach.

As shown in FIG. 3, the transformation matrix 50 is used to provide visual 　 color 　 data based on measured values for a relatively small number of teeth. Embodiments of the present invention generate a transform matrix 50 by statistically effective sampling of spectral reflection data for a plurality of teeth. The number of teeth in the "statistically valid sampling" has enough members to represent a larger 　 teeth 　 population. Increasing the sample size beyond the statistically sufficient or statistically valid number of samples tends to have no significant effect on the resulting data obtained from the sampling distribution.

Referring to FIG. 7, a process of forming the conversion matrix 50 in the conversion matrix generation step S100 is shown. Initially, a sample set of spectral reflection curve 52 is obtained using a spectrophotometer. The sample set has a spectral reflection curve 52 for a sufficient number of teeth (M) (eg M=100 in one embodiment) to be used as a statistically valid sample. This data is then analyzed using the principal component analysis (PCA) shown at 54 to specify the principal component 56 and the corresponding blending coefficient 60 having the highest statistical significance. The experimental data showed that the number of important principal components for the spectral reflection of 　tooth　 and other 　tooth-related objects was about 4.

Principal component analysis (PCA) is a well-known vector space transformation method used in statistics to reduce the total number of dimensions for a multidimensional data set to better show the relationship in a multidimensional data set. In multivariate data such as spectral reflection data, PCA analyzes the covariance of data variables to help reveal trends in the data that can be used to better understand the data and to simplify its use. PCA entails decomposition of data by generating an eigenvector (*?* ₂ ) that is orthogonal to each other on the basis of an alternative reference or coordinate system for the data. PCA determines an ordered set of ordered eigenvectors based on the reduction of eigenvalues associated with each eigenvector. Since the eigenvectors can be increased or decreased to a unit vector and are orthogonal, an orthogonal linear transformation of the eigenvectors from the original coordinate system to the replacement coordinate system is obtained. Due to this type of data decomposition, the lower components (i.e. the first, second, third and subsequent principal components in the sequence) represent the most important aspects of the data, and the higher components represent increasingly less information about changes in the data. This makes it possible to statistically characterize data in a simpler form (ie, smaller dimensions) than the originally acquired data.

Returning to FIG. 7, a set of lower principal components at 56 is generated together with a corresponding set of blending coefficients 60. The blending coefficients are used to increase or decrease the principal component eigenvectors to generate each set of 　tooth spectral reflection curves for the sample set of reflection curve 52. Simply put, the blending coefficients form a vector in the vector space determined by the PCA eigenvector.

Referring also to the logic flow of FIG. 7, in one embodiment, an image of the M teeth of the sample set using the 　 color 　 measurement and 　 mapping device 40 (FIG. 3) at the same location where the spectral reflection curve was previously recorded. By taking value measurements 62, a transformation matrix 50 is created. Four measurement values for each of the M teeth are shown as measurement results 62 in FIG. 7, and the number of measurements obtained for each teeth corresponds to the number of light sources (LEDs) in the lighting device 24. Then, a least-squares fit procedure 78 is performed between the set of blending coefficients 60 and the measured image data, that is, the measured values 62 to generate the transform matrix 50, which is the measured image data ( 62) to a blending factor (60).

In the step of generating the transform matrix 50 according to the embodiment described in FIG. 7, a transform matrix for future use in? Color? Mapping is generated. Then, the matrix and the set of the main components 56 are stored in the control logic processor 38 of the 　 color 　 measurement and 　 mapping apparatus 40. And, whenever the 　 color measurement and 　 mapping 　 device 40 is used in the unknown 　 tooth, the stored transformation matrix 50 is applied to the image values obtained from the pixels of the 　 tooth to calculate a set of blending coefficients (step S160), which is stored It is used together with the main component 56 to generate spectral reflection data (step S170 in Fig. 6B). And the generated spectral reflection curve is used together with the spectral profile of an arbitrary observation light source and the visual 　 color 　 matching 　 function to calculate the tricolor stimulation value (XYZ) of the teeth (step S180 in Fig. 6B).

In an alternative embodiment of generating the transform matrix in step S100 as shown in the logic flow diagram of FIG. 8, the measured image data 62 for a statistically valid sample set instead of the spectral reflection data 52 from the sample set ), PCA 64 is performed. The value of the measured image data 62 is obtained using the 　mapping apparatus 40 (Fig. 3). The result is a set of important principal components 66 and corresponding blending coefficients 70 for the measured image value data 62. XYZ? color values 63 are obtained directly from each curve of a sample set of spectral reflection data measured using a spectrophotometer as described above. A least squares procedure 78 is performed between the obtained XYZ values 63 and the blending coefficients 70 to generate values for the transform matrix 72. In this embodiment, the transform matrix 72 is a best fit matrix for transforming the blending coefficient 70 corresponding to the main component of the measured image data 62 into a tricolor stimulus value. This embodiment does not provide full spectrum reflection information because it directly calculates the visual color of a given observation light source.

9 illustrates another alternative embodiment of step S100 for generating a transform matrix without performing PCA. The XYZ? color value 63 is first obtained from each curve of a sample set of spectral reflection data recorded using a spectrophotometer as described above. The value of the measured image data 62 is obtained using the ?mapping device 40 (Fig. 3). Then, the least squares method is performed between the obtained XYZ value 63 and the measured image value 62 to generate a value of the transformation matrix 74. In this embodiment, the transformation matrix is a best suited matrix for directly transforming the measured image value 62 into a tricolor stimulus value. This is accomplished more simply than the two previous embodiments described with reference to FIGS. 7 and 8. However, unlike the embodiment of Fig. 7 and similar to the embodiment of Fig. 8, only the XYZ? color values for a predetermined observation condition are calculated in this order.

을 포함하도록 확대되는데, 여기서 e는 전술한 바와 같이 실수다. 그리고 획득된 XYZ 값(63)과 측정된 이미지 값(210)의 확대 집합 사이에서 최소자승법이 수행되어 변환 매트릭스(76)의 값을 생성한다. 본 실시형태에서 변환 매트릭스(76)는 확대된 측정 이미지 값(210)을 직접 시각적　색값으로 변환하기 위한 최상적합 매트릭스이다. 도 7, 도 8 및 도 9의 실시형태와는 달리, 이 순서에 의해 소정의 관찰조건에 대한 시각적　색값만이 산출된다.10 illustrates another alternative embodiment of step S100 for generating a transform matrix without performing PCA. The XYZ values 63 are first obtained from each curve of a sample set of spectral reflection data recorded using a spectrophotometer as described above. A visual color value 200 is calculated for each set of XYZ values. These visual color values can be chosen to represent coordinates in a generally uniform visual color space such as CIELAB. The value of the measured image data 62 is obtained using the mapping device 40 (Fig. 3). However, in the embodiment of step S130, the set of measured image data is a polynomial transform value of each measured image value,

Is expanded to include, where e is a real number as described above. Then, the least squares method is performed between the obtained XYZ value 63 and the enlarged set of the measured image values 210 to generate the value of the transformation matrix 76. In this embodiment, the transformation matrix 76 is a best suited matrix for directly converting the enlarged measured image value 210 into a visual color value. Unlike the embodiments of Figs. 7, 8 and 9, only visual color values for a predetermined observation condition are calculated by this procedure.

)에 대하여 획득된 일반화 측정신호(S_i(tooth))는 "

" 같이 나타낼 수 있다.As mentioned earlier, the task of obtaining accurate color measurements using conventional methods is confused by changes in illumination and sensor response. These problems are addressed by the improved method of the present invention. Using the color measurement and mapping device 40, the specific wavelength (

The generalized measurement signal (S _i(tooth) ) obtained for) is "

"Can be expressed together.

)은 센서 응답이고, I_LEDi　는 i번째 LED 또는 그 외의 협대역 광원의 강도이다.Here, R ₀ is the actual reflectance of a tooth or other tooth-related object. This variable can also be expressed as R _tooth with reference to the tooth material, and D(

) Is the sensor response and I _LEDi is the intensity of the ith LED or other narrowband light source.

It can be seen that Equation (1) is generally well suited for tooth color measurements, whether RGB or white light sources are used, and whether the image sensor array is a CCD, CMOS, or other type of photodetector. In particular, it is advantageous to note that conventional tooth color measuring devices such as RGB-based color measuring devices and colorimetric analysis measuring devices generate tooth color values from the measured signals _Si(tooth) . However, as Equation (1) shows, this signal itself is the product of three variable factors: the actual tooth reflectance (R ₀ ), the light source and the sensor response. This is because this measurement depends on the light source and sensor response, so accurate color calculations cannot be obtained with conventional RGB-based or colorimetric measuring devices. However, unlike conventional solutions, the inventive approach isolates the tooth reflectance R ₀ from the other two factors in the measured signal. And the obtained reflectance R ₀ can be used to accurately characterize the color under arbitrary lighting conditions. In this way, the method of the present invention uses a spectrophotometric approach to color characterization. However, unlike conventional spectrophotometric instruments that acquire more than one measurement to extract data for a single photodetector at one location on an object, the apparatus and method of the present invention captures the mapping of spectrophotometric data to the entire object. This data is acquired using a well characterized narrowband light source with a monochromatic sensor array.

))으로부터 장치의 단기변화를 보상할 필요가 있다. 이 때문에, 촬상 공정의 일부로서, 참조 타겟 측정치(S_i(ref))가 교정기준으로서 사용하기 위해 선택에 따라서 획득된다.In the embodiment of the present invention, a plurality of signal values (S _{i (tooth)} ) are obtained, one for each of the N LED color light sources to which the index i (i = 1, 2, ..., N) is attached. . As Equation (1) shows, in order to provide accurate and consistent measurements using S _i(tooth) , for example, variations in light source intensity and/or sensor response (D(

It is necessary to compensate for short-term changes in the device from )). For this reason, as part of the imaging process, a reference target measurement _{value Si(ref} ) is obtained according to selection for use as a calibration criterion.

Here, R _ref is the reflectance of the reference target 28. This value is obtained in one embodiment by reading an image from a reference target 28 (FIG. 3) located within the same field of view as the tooth. In an alternative embodiment, the image reading from the reference target 28 is obtained separately, for example at the beginning of the imaging period.

Given the values _Si(tooth) and _{Si (ref)} , a more useful quantity to implement the method of the invention is a calibrated measure given as follows.

_i)는 LEDi의 피크값에서 참조 타겟(28)의 공지 반사율이다. 교정된 측정값(

)은 측정가변성의 1차 근원을 제거하고 강도 크기 또는 장치 응답에 대한 의존성을 제거한다. 교정된 측정값(

)으로부터 본 발명의 방법은 매우 정확한　색맵핑을 제공하기 위해 시각적　색값을 획득하는데 사용될 수 있다.Where R _ref(

_i) is the known reflectance of the reference target 28 at the peak value of LEDi. Calibrated measurement value (

) Eliminates the primary source of measurement variability and eliminates dependence on intensity magnitude or device response. Calibrated measurement value (

), the method of the present invention can be used to obtain visual color values to provide very accurate color mapping.

The tricolor stimulus values for each pixel X, Y, Z (hereafter X _q (q = 1, 2, 3) are represented, where X ₁ =X, X ₂ =Y, X ₃ =Z) are each in the following way. It can be calculated from tooth reflectance.

의 값은 표준관찰자의 대응하는 시각적　색　매칭　함수이며, I(

)는　치아가 관찰되는 광원의 스펙트럼 분포이다.here

The value of is the standard observer's corresponding visual color matching function, and I(

) Is the spectral distribution of the light source at which the teeth are observed.

₎)은　색　측정 및　맵핑　장치(40)로 직접 측정되지 않지만, 전술한 바와 같이 PCA를 사용하든가 해서 샘플 집합의 분석에서 추출된 혼합 계수를 사용하여 얻을 수 있다.Value (R _tooth(

₎ ) is not directly measured by the color measurement and mapping device 40, but can be obtained using a blending coefficient extracted from the analysis of a sample set by using PCA as described above.

According to Equation (4), the spectral reflection measurement results obtained from the 　 color 　 measurement and 　 mapping device 40 can be used to calculate the tricolor stimulus value of the 　 tooth under arbitrary lighting conditions having a known spectral energy distribution. Applying this procedure to both 　tooth and 　color 　matching 　 tabs will yield two sets of tricolor stimuli that can find the best 　 match. Alternatively, the tricolor stimulus can be transformed into another visual color space, such as CIELAB or HSV, to find the closest match between the tooth and the color match and the tap.

The above approach has been described for a monochromatic 　 wideband sensor array. The same method can also be used if a conventional RGB color sensor array is used to provide signals when each LED or other illumination light source is operating. For RGB color sensors, you can use the color channel with the highest signal level for a specific LED light. For each LED light, if three 　 channels of the sensor are used, the measured image data values will be obtained with the same combination of the weighted sum of the signals of the three 　 channels.

The use of an embodiment of the present invention simplifies the operation of obtaining the spectral photometric/color/mapping of the tooth surface, and the cost of providing this data is significantly reduced over conventional alternatives. As a result of applying the transformation matrix of the present invention, only a small number of measurements are required to obtain very accurate 　 color 　 data. It has been found that measurement from illumination using only 3 or 4 LEDs is sufficient for 　color 　mapping in many intraoral imaging applications.

It can be seen that the order of Figs. 6A and 6B enables 　 color 　 mapping under any condition in the set of observation light source conditions. The collected spectral reflection and mapping data can be used to calculate the color that best matches any observed light source, including incandescent, fluorescent, natural or daylight, or other light sources. Thus, for example, the spectral reflection 　 mapping 　 data obtained in a dentistry could be a prosthesis observed in natural light (daylight), or a prosthesis observed under stage lighting for a model or performer, or observed in a fluorescent lamp in an office environment or other lighting conditions. It can be used to determine the color that best matches the prosthesis. This is an advantage of the method of the present invention, which is better than the previous RGB and colorimetric analysis methods, which were limited to specific lighting conditions, where 　color 　 mapping 　 and 　 color 　 matching are used for color measurement, but may be very different from the lighting conditions most important to the patient.

The information obtained through the processes of FIGS. 6A and 6B can be stored and used in many ways. The generated visual 　 color value can be used directly to enable 　 mapping for display, for example, or can be encoded in some way and stored in a memory. Spectral reflection values can be correlated with other 　tooth 　 image data so that accurate 　color 　 characterization can be provided to a dental lab or other facility where useful.

It can be seen that the method described in FIGS. 6A and 6B obtains a plurality of measurements for each pixel of the 　 tooth 　 image in one embodiment. This is probably a significant amount of data, but it provides a complete evaluation of the spectral component of the image pixel by pixel. Since the data collected for the tooth 　 image is non-metameric, it is desirable for the color 　 mapping to use a conventional colorimetric analysis, RGB, or visually 　 matched system 　 color 　 lighting dependence that can damage data Eliminate the unsuccessful effects.

LEDs are advantageous as a light source for obtaining spectral reflection data according to the invention. In a preferred embodiment, the LED emits most of the light over a wavelength range of less than about 40 nm. In one embodiment, since the LED light sources do not substantially overlap for their respective wavelength bands, it is negligible that any light leaks into adjacent bands. Other types of narrowband light sources can alternatively be used to provide the necessary illumination for reflectance measurements. Other types of light sources include filtered light from broadband light sources such as lanterns, for example. In the example of FIG. 3, four LEDs are shown. In general, at least three different wavelength light sources should be used.

Since the apparatus and method of the present invention provides 　 mapping of spectral reflection values, more accurate information about the true color of teeth is formed than can be used when a conventional colorimetric analysis or RGB-based measurement method is used. Since spectral reflection contains complete 　 color information irrelevant to the light source, the acquired 　 color 　 data does not receive an error due to conditional isochromaticity. By using LEDs or other small light sources without the need for gratings or other devices, the device of the present invention can be packaged compactly at low cost. For example, the 　 color 　 measurement and 　 mapping 　 device 40 may be packaged as an intraoral camera.

The reference target 28 may be any suitable type of reference object, such as a patch, to provide a reference image necessary for 　 color 　 imaging. White or gray patches can be used as well as some other patches with uniform spectral components. The target 28 may be imaged separately, or may be positioned parallel to the tooth at approximately the same distance within the image field of view of the sensor array 44, as in a fixed position, or the actuator may be used for reference imaging as needed in the imaging cycle. It can pivot to a predetermined position by operating or the like.

For example, a number of calculation functions are used to obtain the tricolor stimulus values and to store and display the results. These functions may be provided by a control logic processor 38 that has or is configured to interact with the "color" matching" and "mapping" devices of the present invention. For example, the stored instructions constitute the processor logic circuit to execute the above-described 　 color 　 mapping 　 data access, calculation and output functions. For computational functions, any device of many types of control logic processor devices, such as dedicated computer workstations, personal computers, or embedded computing systems using dedicated data processing components (such as digital signal processing components) can be used. The control logic processing device accesses an electronic memory for data storage and recovery. In addition, an optional display device may be provided to display the result of 　 color 　 matching 　 and 　 color 　 mapping 　.

The method and apparatus of the present invention uses the sensor array 34 as a detector to obtain a 　 mapping with very accurate 　 color information. This part may contain an array of multiple CMOS or CCD sensors, which are typically assigned only one pixel. In one embodiment, a wideband monochromatic 　 sensor is used. However, it is also possible to use a sensor array configured for RGB? color detection or for some other? Color space characteristic. The method of obtaining spectral reflection data in the present invention can be similarly applied to such an apparatus as discussed above. It can be seen that the pixel spacing can be varied for 　 color 　 matching so that a number of sensor positions in the sensor array 44 are grouped together or grouped together, for example to obtain an average value. Since the 　 color measurement apparatus of the present invention uses an image sensor array, the same apparatus used for conventional intraoral 　 color imaging can be configured to suit the imaging and 　 color measurement modes of the work. For example, referring to the schematic block diagram of FIG. 3, 　 color imaging is performed by changing the illumination pattern of the illumination device 24, the resolution of the sensor array 34, and the 　 color processing provided from the control logic processor 38. I can. A mode control command or a mode switch (not shown) issued from an operator interface may be provided to set an operation mode of the apparatus 40 for conventional imaging or 　 tooth color 　 mapping as described herein.

The lighting device 24 of the present invention uses a plurality of color LEDs in one embodiment. However, other types of solid state light sources or other light sources capable of providing multiple color illumination, including more conventional lamps or lamps with a color filter, may alternatively be used.

In order to compensate for aging and drift of parts, initial and periodic calibration of the 　 color 　 measurement and 　 mapping device 40 is required, so that the profile of each LED in the illumination light source 12 can be maintained and regularly updated.

10: imaging device
12: illumination light source
14b, 14g, 14r, 14y: LED 18: capture device
20: teeth
22: color reproduction device
24: lighting device
28: reference target
30: imaging device
32: lens
34: sensor array
36: Color correction conversion
38: control logic processor 40: color measurement and mapping device
44: sensor array
50: statistics-based transformation matrix
52: reflection curve
54: principal component analysis
56: main component
60: mixing coefficient
62: measured image data 63: XYZ value
64: principal components analysis (PCA) 66: principal components
70: mixing coefficient
72: transformation matrix
74: transformation matrix
76: transformation matrix
78: least squares procedure 200: visual color values
210: measured image value S100: conversion matrix generation step
S110: lighting stage
S120: Image capture step
S130: Value acquisition step
S160: Step of applying a transformation matrix to the image value
S170: data generation step S174: visual color value acquisition step
S180: Value extraction step
S190: Save step
λ1, λ2, λ3, λ4: wavelength band P: pixel

Claims (1)

A method of generating color mapping for a dental object, at least partially executed by a control logic processor, comprising:
Generating a transformation matrix according to the spectral reflection data set for statistically valid tooth sampling,
Irradiating illumination to a tooth-related object one wavelength band at a time over at least a first wavelength band, a second wavelength band, and a third wavelength band, and
Obtaining image data values corresponding to at least each of the first wavelength band, the second wavelength band, and the third wavelength band for each of the plurality of pixels in the imaging array; and
Each of the plurality of pixels according to image data values obtained from a reference object in the first wavelength band, the second wavelength band and the third wavelength band, respectively, by applying the transformation matrix Forming a color mapping by creating a set of visual color values for,
A color guide toolkit for natural tooth color matching operated by a method comprising the step of storing the color mapping in an electronic memory accessible by a computer.

Images