JP2007190370A - Image composing device, and method and program thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image composing device, and a method and a program thereof, providing the image of an object in such a display mode that the tone of the object for measurement such as a vital tooth can be easily compared with the tone of an object of comparison to be compared with the object for measurement. <P>SOLUTION: The image composing device comprises an image extraction part 118 for extracting the image of the vital tooth (hereinafter called "the image of the object for measurement") from the colored image of the vital tooth and extracting the image of a shade guide (hereinafter called "the image of the object for comparison") from the colored image of the shade guide, a composite reference line setting part 119 for setting composite reference lines on the image of the object for measurement and the image of the object for comparison, a composing part 120 for creating the composite image by composing the image of the object for measurement and the image of the object for comparison, and a correcting part 121 for correcting at least either one of the image of the object for measurement or the image of the object for comparison if the contours of the images on the composite reference line do not fit each other in the composite image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image synthesizing apparatus that synthesizes a plurality of images, and more particularly to an image synthesizing apparatus, method, and program suitable for use in a colorimetric apparatus that performs colorimetry such as teeth and skin.

In recent years, interest in beauty and health has increased. For example, in beauty, whitening that suppresses melanin pigments in the skin is showing a trend as a field of aesthetic pursuit.
Conventional skin diagnosis uses a camera system for skin diagnosis that is configured so that the surface of the skin can be enlarged and observed on a monitor, for example, dermatology, esthetic salon, beauty counseling, etc. Used in. Of these, for example, in the case of dermatology, by observing images of skin grooves and dermis, the characteristics of the skin surface are captured and diagnosed, and counseling is performed.

  In the field of dentistry, treatment by the ceramic crown method or the like is performed as one means in consideration of aesthetics in dental treatment. This ceramic crown method is performed by producing a crown (ceramic crown prosthesis) having a color close to the color of the original tooth of the patient and covering the crown of the patient with the crown.

Conventionally, the crown is manufactured as follows.
First, in a dental clinic, imaging of the patient's oral cavity is performed by a dentist or the like. Specifically, a photograph of the entire oral cavity including a plurality of teeth, a tooth surface of a living tooth to be crowned, and the like are taken.
Subsequently, the dentist selects a shade guide having a color tone closest to the patient's vital tooth from a plurality of tooth samples (hereinafter referred to as “shade guides”) having different color tones (hereinafter referred to as “shade”). "Take"). The shade guide is obtained by processing different colored ceramics into a tooth shape, for example.
When the above operation is completed, the dentist sends the above-mentioned photographed photograph and the unique identification number assigned to the shade-taken shade guide to the crown laboratory. Thereby, in the technical laboratory, the crown is manufactured based on these pieces of information.

  However, since the above-mentioned shade take is performed by the dentist's subjectivity, it lacks quantitativeness. Furthermore, the appearance of the shade guide and the color of the patient's teeth changes depending on various factors such as the color of the gums, the surrounding environment, lighting conditions (for example, lighting conditions, lighting colors, etc.), and the fatigue level of the doctor. For this reason, it is very difficult to select an optimum shade guide, and there is a drawback that a burden on a doctor is great.

  Therefore, in order to reduce the above-mentioned burden on the doctor or the like, a work support device such as a shade take has been proposed which has a function of automatically selecting a shade guide whose color tone is most similar to a living tooth.

For example, in Japanese Patent No. 3710802 (hereinafter referred to as “Document 1”), identification information data of a plurality of tooth reference colors and L ^* a ^* b ^* color tone information data of the tooth reference colors are stored in advance. A corresponding data table is stored in advance in the computer, and image data obtained by simultaneously photographing a vital tooth and a reference tool (corresponding to the “shade guide” described above) having a tooth reference color tone is input and analyzed in the image data. Color tone information that substantially matches L ^* a ^* b ^* color system color tone information data of the tooth reference color of the reference tool and the tooth standard color color information data of the data table that matches the identification information data of the tooth reference color The correction value is calculated to correct the color tone of the vital tooth, and the tooth reference color identification information data of the color tone information data that matches or approximates the color tone information data of the corrected vital tooth color tone from the data table A technique for extracting and outputting is disclosed.
Japanese Patent No. 3710802 (FIGS. 5 and 6)

  However, in the invention disclosed in the above-mentioned Patent Document 1, since the comparison result of the two color tones and the like is expressed numerically, the user can sensuously know how much color difference actually exists. It was difficult.

  The present invention has been made in view of such circumstances, and provides a user with a display mode that makes it easy to compare the color tone of a measurement target such as a vital tooth and the color of a comparison target compared to the measurement target. An object of the present invention is to provide an image synthesizing apparatus, a method thereof, and a program.

In order to solve the above problems, the present invention employs the following means.
The present invention is an image composition device that synthesizes a first captured image in which a measurement target is photographed and a second captured image in which a comparison target to be compared with the measurement target is photographed to create a single image. And extracting a measurement target image from the first captured image and extracting a comparison target image from the second captured image; and a composite reference line on the measurement target image and the comparison target image Combining reference line setting means for setting each of the measurement target image and the comparison target image by combining the measurement target image and the comparison target image along the composite reference line, and the composite reference in the composite image Provided is an image synthesizing device including a correcting unit that corrects at least one of the measurement target image and the comparison target image so that the contours match when the contours in the line do not match.

According to such a configuration, for example, the measurement target image and the comparison target image are bonded and displayed as one image, so that the user can directly change the color tone of both by visually recognizing the boundary portion of the bonding. Can be compared. Thereby, the user can recognize a subtle color difference. In this case, even when the sizes of the measurement target image and the comparison target image are different and the contours do not match on the composite screen synthesized by the synthesis unit, the contour on the synthesis reference line is corrected by the correction unit. Are corrected so as to match, a natural composite image can be provided to the user.
The “extraction” in the extraction means includes not only the concept of extracting the measurement target image separately from the first captured image but also the concept of specifying the contour of the measurement target image in the first captured image. The same applies to the target image.
The “synthesis” technique is not particularly limited. For example, a method of pasting and synthesizing each image piece cut at the composite reference line, or overlapping each other so that the composite reference line set for each image matches, and using the composite reference line as a boundary, It is possible to employ a method of synthesizing images by changing the transmittance.

  The measurement object refers to an object for which a plurality of color samples expressed in different color tones are easily prepared in advance, and it is necessary to compare the color tones with these color samples. The measurement object is, for example, a part of the human body such as living teeth and skin, interiors such as curtains and carpets, leather products such as bags and sofas, electronic parts, painted cars, walls, and the like.

  In the image composition device, the composition reference line setting unit sets a line passing through the center of gravity of the measurement target image as the composition reference line on the measurement target image, and a line passing through the center of gravity of the comparison target image The reference line may be set on the comparison target image.

  According to such a configuration, in the measurement target image and the comparison target image, since the lines passing through the respective centroids are set as the composite reference lines, it is possible to keep the area ratio of both in the composite image substantially the same. A balanced composite image can be provided.

  In the image composition device, the composition reference line setting means places the composition reference line on the measurement target image along the vertical axis direction at a position where the length of the measurement target image in the vertical axis direction is maximum. It is good also as setting the said synthetic | combination reference line on the said comparison object image along the said vertical axis direction in the location where the length of the vertical axis direction of the said comparison object image becomes the maximum.

  According to such a configuration, in the measurement target image and the comparison target image, since the composite reference line is set along the vertical axis direction at the position where the length in the vertical axis direction is maximum, the composite standard described later Even when the composite reference line is changed by the line changing means, it is possible to easily follow this change, and it is possible to quickly create a composite image after the change.

  The image synthesizing apparatus may further include synthesis reference line changing means for changing the synthesis reference line from the outside in the synthesized image.

According to such a configuration, the user can move the composite reference line to a desired position, so that the area ratio between the measurement target image and the comparison target image in the composite image can be changed, and the color tone comparison can be performed. You can freely change the place to do.
Further, when the composite reference line is changed by the composite reference line changing means, the composite reference line after the change is not used as it is, but on the measurement target image and the comparison target based on the composite reference line moved by the user. By resetting the composite reference line on the image, it is possible to set the composite reference line at an appropriate location in consideration of the overall balance. As a result, it is possible to keep the area ratio of both of the composite images after the change at a suitable value and reduce the distortion of the contour.

  In the image composition apparatus, the correction unit may reduce or enlarge at least one of the measurement target image and the comparison target image.

  According to such a configuration, it is possible to match the contours in the composite image by an easy correction process.

  In the image composition device, for example, the measurement target is a vital tooth, and the comparison target is the shade guide.

The image composition device is particularly suitable for use in a dental colorimetry device.
Specifically, the present invention provides the image synthesizing apparatus, a colorimetric value calculation means for calculating a colorimetric value near the synthetic reference line in the measurement target image, and the synthetic reference line vicinity of the measurement target image. A difference calculating unit that calculates a difference between the colorimetric value of the comparison target image and the colorimetric value of the comparison target image in the vicinity of the composite reference line, the colorimetric value of the measurement target image in the vicinity of the composite reference line, and the comparison target image There is provided a dental colorimetric device comprising display control means for displaying at least one of the colorimetric value in the vicinity of the composite reference line and the difference between the colorimetric values on the same screen as the composite image. .

The present invention combines an imaging device that captures an intraoral area, a first captured image in which a measurement target is captured by the imaging device, and a second captured image in which a comparison target to be compared with the measurement target is captured. A dental colorimetric device having an image composition device for creating a single image, wherein the image composition device extracts a measurement target image from the first captured image and from the second captured image. Image extraction means for extracting a comparison target image; synthesis reference line setting means for setting a synthesis reference line on the measurement target image and the comparison target image; and the measurement target image and the comparison target image as the synthesis reference A synthesis unit that creates a synthesized image by synthesizing along the line; and in the synthesized image, when the contour in the synthesized reference line does not match, the measurement object image and the ratio To provide a dental colorimetry system comprising correction means for correcting at least one of the target image.

  The present invention is an image creation method for creating a single image by combining a first captured image in which a measurement target is captured and a second captured image in which a comparison target to be compared with the measurement target is captured. And extracting a measurement target image from the first captured image and extracting a comparison target image from the second captured image, and a composite reference line on the measurement target image and the comparison target image. A synthesis reference line setting process, a synthesis process for creating a synthesized image by synthesizing the measurement target image and the comparison target image along the synthesis reference line, and the synthesis reference in the synthesized image. Provided is an image composition method including a correction process for correcting at least one of the measurement target image and the comparison target image so that the contours match when the contours in the line do not match.

  The present invention relates to an image creation program for synthesizing a first captured image in which a measurement object is photographed and a second captured image in which a comparison object to be compared with the measurement object is photographed to create a single image. An image extraction process for extracting a measurement target image from the first captured image and extracting a comparison target image from the second captured image, and combining the image on the measurement target image and the comparison target image In the combined image, a combined reference line setting process for setting a reference line, a combined process for creating a combined image by combining the measurement target image and the comparison target image along the combined reference line, and the combined image, When the contours on the composite reference line do not match, the computer executes correction processing for correcting at least one of the measurement target image and the comparison target image so that the contours match To provide an image synthesizing program for causing.

  According to the present invention, it is possible to provide a user with a display mode in which the color tone of a measurement target such as a vital tooth and the color tone of a comparison target compared with the measurement target can be easily compared.

Hereinafter, an embodiment in which the image composition device of the present invention is applied to a dental color measurement system will be described with reference to the drawings.
As shown in FIGS. 1 and 4, the dental color measurement system according to an embodiment of the present invention includes an imaging device 1, a cradle 2, a dental color measurement device 3, and a display device 4. .

The imaging device 1 includes a light source 10, an imaging unit 20, an imaging control unit 30, a display unit 40, and an operation unit 50 as main components as shown in FIG.
The light source 10 is disposed in the vicinity of the tip of the photographing apparatus 1 and emits illumination light having four or more different wavelength bands for illuminating the subject. Specifically, the light source 10 includes seven light sources 10a to 10g that emit light in different wavelength bands. Each of the light sources 10a to 10g includes four LEDs. As shown in FIG. 2, the central wavelengths of the light sources 10a are about 450 nm, the light source 10b is about 465 nm, the light source 10c is about 505 nm, and the light source 10d is about 525 nm. The light source 10e is about 575 nm, the light source 10f is about 605 nm, and the light source 10g is about 630 nm. The emission spectrum information of the LED is stored in the LED memory 11 and used in the dental color measuring device 3 described later.

  These light sources 10a to 10g are arranged in a ring shape, for example. The arrangement is not particularly limited, and for example, four LEDs may be arranged in the order of shorter wavelengths, or may be arranged in reverse order or at random. Further, in addition to the arrangement in which all the LEDs form one ring, the LEDs may be divided into a plurality of groups, and these one group may be arranged to form one ring. The arrangement of the LEDs is not limited to the ring shape, and an appropriate arrangement such as a cross arrangement, a rectangular arrangement, a random arrangement, or the like may be adopted as long as it does not hinder imaging by the imaging unit 20 described later. Is possible. The light emitting element of the light source 10 is not limited to an LED, and for example, a semiconductor laser such as an LD (laser diode) or other light emitting elements can be used.

  In the imaging apparatus 1, an illumination optical system for irradiating illumination light from the light source 10 onto the subject surface substantially uniformly is provided on the subject side of the light source 10. A temperature sensor 13 for detecting the temperature of the LED is provided in the vicinity of the light source 10.

  The imaging unit 20 includes an imaging lens 21, an RGB color imaging device 22, a signal processing unit 23, and an AD converter 24. The imaging lens 21 forms a subject image irradiated with the light source 10. The RGB color image sensor 22 captures a subject image formed by the imaging lens 21 and outputs an image signal. The RGB color image pickup element 22 is constituted by, for example, a CCD, and the sensor sensitivity is such that it substantially covers the visible wide area. This CCD may be a monochrome type or a color type. The RGB color image sensor 22 is not limited to a CCD, and a CMOS type or other various image sensors can be widely used.

  The signal processing unit 23 performs gain correction, offset correction, and the like on the analog signal output from the RGB color image sensor 22. The A / D converter 24 converts the analog signal output from the signal processing unit 23 into a digital signal. A focus lever 25 for adjusting the focus is connected to the imaging lens 21. The focus lever 25 is for manually adjusting the focus, and a position detection unit 26 for detecting the position of the focus lever 25 is provided.

  The imaging control unit 30 includes a CPU 31, an LED driver 32, a data I / F 33, a communication I / F controller 34, an image memory 35, and an operation unit I / F 36. Each of these units is connected to the local bus 37 and is configured to be able to exchange data via the local bus 37.

  The CPU 31 controls the imaging unit 20, records the subject spectral image acquired and processed by the imaging unit 20 in the image memory 35 via the local bus 37, and outputs it to the LCD controller 41 described later. The LED driver 32 controls light emission of various LEDs included in the light source 10. The data I / F 33 is an interface for receiving the contents of the LED memory 11 provided in the light source 10 and the information of the temperature sensor 13. The communication I / F controller 34 is connected to a communication I / F contact 61 serving as a connection point with the outside, and has a function for realizing communication compliant with USB 2.0, for example. The operation unit I / F 36 is connected to various operation buttons provided in the operation unit 50 to be described later, and functions as an interface for transferring an input instruction input via the operation unit 50 to the CPU 31 via the local bus 37. The image memory 35 temporarily records image data captured by the imaging unit 20. In the present embodiment, the image memory 35 has a memory capacity capable of recording at least seven spectral images and one RGB color image.

  The display unit 40 includes an LCD controller 41 and an LCD 42. The LCD controller 41 causes an LCD (liquid crystal display) 42 to display an image based on the image signal transferred from the CPU 31, for example, an image currently captured by the imaging unit 20 or an already captured image. At this time, if necessary, the image pattern stored in the overlay memory 43 may be superimposed on the image acquired from the CPU 31 and displayed on the LCD 42. Here, the image pattern recorded in the overlay memory 43 is, for example, a horizontal line that captures the entire tooth horizontally, a cross line that intersects this, an imaging mode, an identification number of the tooth to be imaged, or the like. .

  The operation unit 50 includes various operation switches and operation buttons for a user to input an instruction to start a spectral image shooting operation and to input an instruction to start and end a moving image shooting operation. Specifically, a shooting mode changeover switch 51, a shutter button 52, a viewer control button 53, and the like are provided. The shooting mode switch 51 is a switch for switching between normal RGB shooting and multiband shooting. The viewer control button 53 is a switch for performing operations such as changing the image displayed on the LCD 42.

  Moreover, the imaging device 1 is equipped with a lithium battery 60 as a storage battery. The lithium battery 60 is used to supply power to each unit included in the imaging apparatus 1 and is connected to a connection contact 62 that is a contact for charging. Further, a battery LED 63 for notifying the charging state of the lithium battery is provided. In addition, the imaging apparatus 1 is provided with a power LED 64 for notifying the camera state and an alarm buzzer 65 for notifying danger during photographing.

The battery LED 63 includes, for example, three LEDs of red, yellow, and green, notifies that the charging capacity of the lithium battery 60 is sufficient by lighting in green, and in other words, indicates that the battery capacity is low by lighting in yellow. The fact that charging is necessary is notified, and the fact that the battery capacity is extremely low by lighting in red, in other words, the fact that urgent charging is required is notified.
The power LED 64 includes, for example, two LEDs of red and green, and notifies that the preparation for shooting is completed by lighting in green, and that the preparation for shooting (initial warming, etc.) is in progress by blinking green. Then, it is notified that the battery is being charged by lighting in red.
The alarm buzzer 65 notifies that the captured image data is invalid by warning.

  The cradle 2 that supports the imaging device 1 described above includes a color chart 100 for calibrating the imaging unit 20, a micro switch 101 for confirming whether the imaging device 1 is mounted at a normal position, The power supply 102 for turning on / off the power, the power lamp 103 which is turned on / off in conjunction with the on / off of the power switch 102, and the light on / off in conjunction with the microswitch 101, the imaging apparatus 1 A mounting lamp 104 for notifying whether or not the camera is mounted at the normal position is provided.

  For example, when the imaging apparatus 1 is mounted at a normal position, the mounting lamp 104 is lit green, and when it is not mounted, the mounting lamp 104 is lit red. In addition, the cradle 2 is provided with a power connection connector 105 to which an AC adapter 106 is connected. Then, when the charging capacity of the lithium battery 60 included in the imaging device 1 is reduced and the yellow or red of the battery LED 63 is lit, charging of the lithium battery is started when the imaging device 1 is mounted on the cradle. It is configured as follows.

  In the imaging device 1 of the dental colorimetry system configured as described above, illumination light in seven types of wavelength bands (seven primary color illumination lights) is sequentially irradiated onto the subject, and the seven subject spectral images are used as still images. In addition to multiband imaging, it is also possible to capture RGB images. As one of the methods for capturing RGB images, as in a normal digital camera, there is no illumination light of the seven primary colors, and an RGB color CCD having an object illuminated by natural light or room light is used. Imaging is performed. Also, each of one or more illumination lights from the seven primary colors can be selected as RGB three-color illumination lights, and these can be sequentially irradiated to capture a frame sequential still image. ing.

Among these imaging modes, RGB imaging is used when imaging a wide area, such as when imaging the entire patient's face or when imaging the entire jaw. On the other hand, multiband imaging is used when measuring the color of one or two teeth of a patient accurately, that is, when measuring the color of teeth.
Hereinafter, the color measurement processing of teeth by multiband imaging, which is a characteristic part of the present invention, will be described.

[Multiband shooting]
First, the imaging device is lifted from the cradle 2 by a doctor, and a contact cap is attached to an attachment port (not shown) provided on the projection port side of the housing of the imaging device. The contact cap is formed in a substantially cylindrical shape from a flexible material.

  Subsequently, when the photographing mode is set to the “colorimetry mode” by the doctor, the subject image is displayed on the LCD 42 as a moving image. While checking the image displayed on the LCD 42, the doctor positions the patient's vital teeth to be measured at an appropriate position in the imaging range, and performs focus adjustment using the focus lever 25. . At this time, since the contact cap is formed in a shape that guides the vital tooth to be measured to an appropriate photographing position, positioning can be easily performed.

  When the doctor presses the shutter button 52 in a state where the positioning and focus adjustment have been performed, a signal to that effect is transferred to the CPU 31 via the operation unit I / F 36, and multiband imaging is executed under the control of the CPU 31. Is done.

  In multi-band imaging, the LED driver 32 sequentially drives the light sources 10a to 10g, so that LED irradiation light having different wavelength bands is sequentially irradiated onto the subject. The reflected light of the subject is imaged on the surface of the RGB imaging element 22 of the imaging unit 20 and is captured as an RGB image. The captured RGB image is transferred to the signal processing unit 23. The signal processing unit 23 performs predetermined image processing on the input RGB image signal and selects predetermined one-color image data corresponding to the wavelength bands of the light sources 10a to 10g from the RGB image signal. Specifically, the signal processing unit 23 selects B image data from the image signals corresponding to the light sources 10a and 10b, selects G image data from the image signals corresponding to the light sources 10c to 10e, and selects the light sources 10f and 10f. R image data is selected from the image signal corresponding to 10g. Thus, the signal processing unit 23 selects image data having a wavelength that approximately matches the center wavelength of the irradiation light.

The image data selected by the signal processing unit 23 is transferred to the A / D converter 24 and recorded in the image memory 35 via the CPU 31. As a result, an image of a color selected from the RGB image corresponding to the center wavelength of the LED is recorded in the image memory 35 as a multiband image. When photographing, the CPU 31 controls the irradiation time of the LED, the irradiation intensity, the electronic shutter speed of the image sensor, etc. so that the photographing of each wavelength is properly exposed, and the temperature change during the photographing is severe. In this case, the alarm buzzer 65 sounds and issues a warning.
Note that an image of the vital tooth is further taken without irradiating the LED, and is recorded in the image memory 35 as an external light image.

Subsequently, when the imaging is finished and the imaging device 1 is placed on the cradle 2 by the doctor, the calibration image is measured.
The calibration image is measured by photographing the color chart 100 according to the same procedure as the multiband photographing described above. As a result, the multiband image of the color chart 100 is recorded in the image memory 35 as a color chart image.
Subsequently, the color chart 100 is shot in a state where the LEDs are not lit at all (under dark), and this image is recorded in the image memory 35 as a dark current image. Note that this dark current image may be captured a plurality of times and an image obtained by averaging these images may be used.

  Subsequently, signal correction using the external light image and dark current image recorded in the image memory 35 is performed on each of the multiband image and the color chart image. The signal correction for the multiband image is performed, for example, by subtracting the signal value of the external light image data for each pixel from the image data of the multiband image to remove the influence of the external light at the time of shooting. it can. Similarly, the signal correction for the color chart image is performed by subtracting the signal value of the dark current image data for each pixel from the image data of the color chart image, for example, to remove the dark current noise of the CCD that changes with temperature. It can be performed.

FIG. 3 shows an example of the signal correction result for the color chart image. In FIG. 3, the vertical axis indicates the sensor signal value, the horizontal axis indicates the input light intensity, the solid line indicates the original signal before correction, and the broken line indicates the signal after signal correction.
The multiband image and the color chart image after the signal correction is performed in this way are transmitted to the dental colorimetric device 3 via the local bus 37, the communication I / F controller 34, and the communication / I / F contact 61. And recorded in the multiband image memory 110 in the dental colorimetric device 3 shown in FIG.

  Note that the multiband image and the dark current image of the color chart 100 are not recorded in the image memory 35 in the imaging apparatus 1, and the local bus 37, the communication I / F controller 34, and the communication / I / F contact 61 are connected. Then, the configuration may be such that it is directly transmitted to the dental colorimetric device 3 and recorded in the multiband image memory 110 in the dental colorimetric device 3. In this case, the signal correction described above is performed in the dental color measuring device 3.

  The dental colorimetric device 3 is composed of, for example, a personal computer, receives a multiband image and a color chart image output via the communication I / F contact 61 of the imaging device 1, and variously displays the multiband image. By performing the above-described processing, an image in which a high-quality color reproduction of the subject tooth is created, a shade guide number suitable for the tooth is selected, and the information is displayed on the display device 4.

For example, as shown in FIG. 4, the dental colorimetric device 3 includes a chromaticity calculation unit 70, a shade guide number determination unit 80, a multiband image memory 110, an RGB image memory 111, a color image creation processing unit 112, and an image filing. Unit 113, shade guide chromaticity data storage unit 114, image display GUI unit (display control unit) 115, image synthesis unit (image synthesis device) 116, and difference calculation unit (difference calculation unit) 130. .
The chromaticity calculation unit 70 includes a spectrum estimation calculation unit 71, an observation spectrum calculation unit 72, and a chromaticity value calculation unit (colorimetric value calculation means) 73. The shade guide number determination unit 80 includes a determination calculation unit 81 and a shade guide reference image data storage unit 82. In the shade guide reference image data storage unit 82, for example, for each manufacturer that manufactures a shade guide in which color samples are arranged in a row, the image data of the shade guide that is manufactured by the manufacturer is associated with the shade guide number. In addition, a spectral reflectance spectrum of a predetermined area of the shade guide and a shade guide image with gingiva are stored.

  In the dental colorimetric device 3 having such a configuration, the multiband image and the color chart image transferred from the imaging device 1 are first recorded in the multiband image memory 110 and then transferred to the chromaticity calculation unit 70. Is done. In the chromaticity calculation unit 70, first, spectrum estimation calculation unit 71 performs spectrum (in this embodiment, spectrum of spectral reflectance) estimation processing and the like.

  As shown in FIG. 5, the spectrum estimation calculation unit 71 includes a conversion table creation unit 711, a conversion table 712, an input γ correction unit 713, a pixel interpolation calculation unit 714, an in-plane unevenness correction unit 715, a matrix calculation unit 716, and a spectrum. An estimation matrix creating unit 717 is provided. The input γ correction unit 713 and the pixel interpolation calculation unit 714 are individually provided for each of the multiband image and the color chart image. The input γ correction unit 713a and the pixel interpolation calculation unit 714a are provided for the multiband image. An input γ correction unit 713b and a pixel interpolation calculation unit 714b are provided for the color chart image.

  In the spectrum estimation calculation unit 71 having such a configuration, first, the multiband image and the color chart image are respectively transferred to the separately provided input γ correction units 713a and 713b, and input γ correction is performed. After that, image interpolation calculation processing is performed by the pixel interpolation calculation units 714a and 714b. The signals after these processes are transferred to the in-plane unevenness correction unit 715, where the in-plane unevenness correction processing of the multiband image using the color chart image is performed. Thereafter, the multiband image is transferred to the matrix calculation unit 716, and the spectral reflectance is calculated using the matrix created by the spectrum estimation matrix creation unit 717.

Hereinafter, each image processing performed in each unit will be specifically described.
First, a conversion table 712 is created by the conversion table creation unit 711 as a pre-stage of input γ correction. Specifically, the conversion table creation unit 711 has data in which the input light intensity is associated with the sensor signal value, and creates the conversion table 712 based on these data. The conversion table 712 is created from the relationship between the input light intensity and the output signal value. For example, as shown by the solid line in FIG. 6A, the input light intensity and the sensor signal value are approximately proportional to each other. Created to be

  Each input γ correction unit 713a and 713b performs input γ correction on the multiband image and the color chart image by referring to the conversion table 712. This conversion table is created so that an input light intensity D corresponding to the sensor value A at the present time is obtained, and an output sensor value B corresponding to the input light intensity D is output. As a result, the conversion table shown in FIG. It becomes like this. In this way, when the input γ correction is performed on the multiband image and the color chart image, the corrected image data is transferred to the pixel interpolation calculation units 714a and 714b, respectively.

  In the pixel interpolation calculation units 714a and 714b, a low-pass filter for pixel interpolation is applied to each of the multiband image data and color chart image data after the input γ correction. FIG. 7 shows an example of a low-pass filter applied to the R signal and the B signal. FIG. 8 shows a low-pass filter applied to the G signal. By multiplying each multiband image data by such a low-pass filter for pixel interpolation, for example, an image of 144 pixels × 144 pixels is changed to an image of 288 pixels × 288 pixels.

The image data g _k (x, y) on which the image interpolation calculation has been performed is transferred to the in-plane unevenness correction unit 715.
The in-plane unevenness correction unit 715 corrects the luminance at the center of the screen of the multiband image data using the following equation (1).

In the above equation (1), c _k (x, y) is color chart photographed image data, g _k (x, y) is multiband image data after input γ correction, and (x ₀ , y ₀ ) is the center pixel. The position, δ (= 5) is the area average size, and g ′ _k (x, y) is the image data after in-plane unevenness correction (where k = 1,..., N (number of bands)).
The in-plane unevenness correction is performed on each image data of the multiband image data.

The multiband image data g ′ _k (x, y) after the in-plane unevenness correction is transferred to the matrix calculation unit 716. The matrix calculation unit 716 performs spectrum (spectral reflectance) estimation processing using the multiband image data g ′ _k (x, y) from the in-plane unevenness correction unit 715. In this spectrum (spectral reflectance) estimation process, spectral reflectance is estimated at 1 nm intervals in a wavelength band from 380 nm to 780 nm. That is, in this embodiment, a 401-dimensional spectral reflectance is estimated.

In general, a heavy and expensive spectroscopic measuring instrument is used to obtain the spectral reflectance for each wavelength. However, in this embodiment, since the subject is limited to teeth, the subject has a certain amount. By utilizing the feature, the 401-dimensional spectral reflectance is estimated with a small number of bands.
Specifically, a 401-dimensional spectrum signal is calculated by performing a matrix operation using the multiband image data g ′ _k (x, y) and the spectrum estimation matrix Mspe.
The spectrum estimation matrix Mspe is created by the spectrum estimation matrix creation unit 717 based on the spectral sensitivity data of the camera, the LED spectrum data, and the statistical data of the subject (tooth). The creation of the spectrum estimation matrix is not particularly limited, and a well-known method can be used. For example, one example is SKPark and FOHuck
“Estimation of spectral
reflectance curves from multispectrum image data ”, Applied Optics,
16, pp 3107-3114 (1977).
Note that the spectral sensitivity data of the camera, the LED spectrum data, the subject (tooth) statistical data, and the like are stored in advance in the image filing unit 113 shown in FIG. When the spectral sensitivity of the camera changes depending on the position of the sensor, spectral sensitivity data may be acquired according to the position, or the center position may be appropriately corrected and used. .

  When the spectral reflectance is calculated by the spectrum estimation calculation unit 71, this calculation result is sent to the observation spectrum calculation unit 72 in the shade guide number determination unit 80 and the chromaticity calculation unit 70 shown in FIG. 4 together with the multiband image data. Transferred.

  Information from the spectrum estimation calculation unit 71 is transferred to the determination calculation unit 81 in the shade guide number determination unit 80. In the determination calculation unit 81, first, an area specifying process for specifying a tooth area to be measured is performed.

  Here, the multiband image data picked up by the image pickup apparatus 1 includes not only vital teeth to be measured but also information such as teeth adjacent to the vital teeth and gums. Therefore, in the area specifying process, a process of specifying a vital tooth area to be measured from the intraoral image data is performed.

  FIG. 9 shows an example of a tooth reflection spectrum (sample number n = 2), and FIG. 10 shows an example of a gum reflection spectrum (sample number n = 5). 9 and 10, the horizontal axis indicates the wavelength and the vertical axis indicates the reflectance. Since the teeth are generally white and the gums are red, the spectra of both are blue wavelength band (for example, 400 nm to 450 nm) and green wavelength band (for example, 530 nm), as can be seen from FIGS. To 580 nm). In this embodiment, paying attention to the fact that vital teeth have a specific reflection spectrum in this way, by extracting pixels having the reflection spectrum of vital teeth, which is a specific spectrum, from the image data, the region of vital teeth Is identified.

[Identification method of vital tooth area 1]
In this method, wavelength band characteristic values determined by signal values of n types of wavelength bands in a region (pixel or set of pixels) in an image indicated by captured multiband image data are represented in the n-order space. . Then, in the n-order space, a plane area representing the characteristics of the measurement target is defined, and when the wavelength band characteristic value represented in the n-order space is projected onto the plane area, the wavelength band characteristic value is By determining that the region in the image to be included in the vital tooth region to be measured, the region (contour) to be measured is specified.

  FIG. 11 illustrates a method for specifying a vital tooth region to be measured by this method. As shown in FIG. 11, a seven-dimensional space is formed by seven wavelengths λ1 to λ7. In the 7-dimensional space, a classification plane that best separates vital teeth to be measured is set. Specifically, classification spectra d1 (λ) and d2 (λ) for plane projection are obtained. First, a predetermined region is cut out from the captured multiband image data, and a feature amount represented in the 7-dimensional space is calculated as a wavelength band characteristic value. The feature amount is a group of seven signal values when each band is averaged in this region and converted into seven signal values. The size of the region to be cut out is, for example, 2 pixels × 2 pixels, but is not limited thereto, and may be 1 pixel × 1 pixel or 3 pixels × 3 pixels or more.

  The feature amount is represented by one point in the 7-dimensional space of FIG. One point in the 7-dimensional space represented by this feature amount is projected onto the classification plane to obtain one point on the classification plane. The coordinates of one point on the classification plane can be obtained from the inner product calculation with the classification spectra d1 (λ) and d2 (λ). If one point on the classification plane is included in the area T on the classification plane defined by the characteristic spectrum of the tooth, that is, the plane area representing the characteristics of the measurement object, the extracted area is included in the outline of the vital tooth It is determined that the area is On the other hand, if one point on the classification plane is included in the region G on the classification plane defined by the characteristic spectrum of the gums, it is determined that the extracted region is an area included in the outline of the gums.

  In this method, the vital tooth region is specified by sequentially making such a determination while changing the region to be cut out. In particular, the vital tooth region to be measured is usually located near the center of the image indicated by the captured multiband image data, so the region to be clipped is directed from near the center of the image to the periphery. The region of the vital tooth to be measured (that is, the contour of the vital tooth to be measured) is identified by sequentially making the above determination as to whether or not the region is included in the vital tooth region while changing. To do. In particular, in the present embodiment, since the feature amount is defined in a 7-dimensional space that is larger than the 3-dimensional space represented by normal RGB, it is preferable because the measurement target region (contour) can be specified more accurately. .

[Identification method of vital tooth area 2]
In addition to the above method for identifying regions based on the classification spectrum, in this method, for example, only specific signal values (spectrums) corresponding to the blue wavelength band and the green wavelength band are extracted, and these signal values are compared. Thus, a region having a signal value (spectrum) peculiar to vital teeth is specified as a vital tooth region. According to such a method, since the number of samples to be compared is reduced, it is possible to easily specify a region in a short time.

  Specifically, as in the case of region specification based on the classification spectrum, the position is measured by detecting the position of the inflection point where the spectral feature value changes suddenly from the center of the image toward the periphery. Determined as the outline of the subject's vital teeth. For example, compared with the subject (tooth) to be detected and the subject to be separated (for example, gums other than vital teeth), characteristic bands λ1 and λ2 are selected, and the ratio is set as the spectral feature value. Assuming that the subject wants to detect vital teeth, for example, a ratio of two points is calculated assuming that λ1 = 450 nm and λ2 = 550 nm, and an inflection point of the ratio is obtained. Thereby, the outline with the adjacent vital tooth is decided and the pixel of the vital tooth to be measured can be obtained. In addition to specifying for each pixel, an average of a pixel group composed of a plurality of pixels may be taken, and the specification may be made for each pixel group based on this average.

  In addition, in this embodiment, although region identification was performed for vital teeth in either of the above-described vital tooth region identification methods 1 and 2, gum region identification is also possible. In addition to vital teeth, the present invention can also be applied to the other side, such as specifying areas such as spots and freckles in dermatology and the like, and specifying areas of predetermined paint colors in multicolor pattern painting. In these cases, a spectrum peculiar to the region to be specified such as gums, stains, freckles, and paints may be used. For example, when specifying a spot or freckles area, a spectrum peculiar to benign spots or freckles or a spectrum peculiar to malignant spots or freckles is registered in advance, and an area that approximates these spectra is identified. Thus, it is possible to easily specify the region to be measured.

  In this way, when the pixel of the vital tooth to be measured is specified, subsequently, a measurement region setting process for setting the measurement region in the region of the vital tooth to be measured is performed. As shown in FIG. 12, this measurement region is set as a rectangular region at the upper, middle, and lower portions of the tooth surface. For example, a region having an area with a certain ratio with respect to the height of vital teeth is set. That is, the measurement region and its position are set at a constant ratio for both small and large vital teeth. Note that the shape of the measurement region is not limited to a rectangle as shown in FIG. 12, and may be, for example, a circle, an ellipse, or an asymmetric shape.

Subsequently, a shade guide selection process (sample selection process) for selecting the closest shade guide is performed for each measurement region set as described above. In this shade guide selection process, a comparison determination is made between the color tone of the vital tooth to be measured and the color tone of the shade guide. This comparison is performed for each measurement region set in advance, and each shade guide registered in advance in the spectrum (in this embodiment, the spectral reflectance) of the target measurement region and the shade guide reference image data storage unit 82. Are compared with each other (spectral reflectance in the present embodiment) and the difference between the two is determined to be the smallest.
For example, it is performed by obtaining a spectrum determination value Jvalue based on the following equation (2).

In the above equation (2), Jvalue is a spectrum judgment value, C is a normalization coefficient, n is a statistical number (number of λ used in calculation), λ is a wavelength, and f ₁ (λ) is a vital tooth to be judged. , F ₂ (λ) is a spectral sensitivity spectrum value of the shade guide, and E (λ) is a determination sensitivity correction value. In the present embodiment, weighting relating to spectral sensitivity according to λ is performed by E (λ).

In this way, the spectrum value of the shade guide of each company is substituted into f ₂ (λ) in the above equation (2), and each spectrum judgment value Jvalue is calculated. Then, it is determined that the shade guide having the smallest spectrum determination value Jvalue is the shade guide number that most closely approximates the tooth. In the present embodiment, a plurality of (for example, three) candidates are extracted in ascending order of the spectrum determination value Jvalue. Of course, one candidate can be extracted. In the above equation (2), the determination sensitivity correction value E (λ) may be variously weighted.

  When the shade guide numbers are selected in this way, these numbers are transferred from the determination calculation unit 81 to the shade guide chromaticity data storage unit 114 and the image display GUI unit 115. The chromaticity data corresponding to the selected shade guide number is transferred from the shade guide chromaticity data storage unit 114 to the image display GUI unit 115 and the difference calculation unit 130. Further, the spectrum determination value Jvalue for each shade guide number is transferred from the determination calculation unit 81 to the image display GUI unit 115.

  On the other hand, the observation spectrum calculation unit 72 of the chromaticity calculation unit 70 multiplies the vital tooth spectrum obtained by the spectrum estimation calculation unit 71 by the illumination light S (λ) to be observed under the illumination light to be observed. The spectrum G (x, y, λ) of the subject is obtained. This S (λ) is a spectrum of a light source that is desired to observe the color of the vital tooth, such as a D65, D55 light source, or a fluorescent light source, and this data is stored in advance in an input profile storage unit (not shown). The spectrum of the subject under the illumination light to be observed, which is obtained by the observation spectrum calculation unit 72, is transferred to the chromaticity value calculation unit 73 and the color image creation processing unit 112.

In the chromaticity value calculation unit 73, L ^* a ^* b ^{* which} is a chromaticity value is calculated for each pixel from the spectrum of the subject under illumination light to be observed, and the chromaticity value L ^* a ^* in a predetermined area is calculated ^. The average value of b ^* is calculated and sent to the image display GUI unit 115 and the difference calculation unit 130, respectively. This predetermined area is set at, for example, three positions of the upper, middle and lower parts of the vital tooth.
On the other hand, the spectrum G (x, y, λ) of the subject under illumination light to be observed is an RGB image that is sent from the observation spectrum calculation unit 72 to the color image creation processing unit 112 and displayed on the monitor. RGB2 (x, y) is created and sent to the image display GUI unit 115. Note that this RGB image may be subjected to edge enhancement or the like.

The difference calculation unit 130 averages the chromaticity values L ^* a ^* b ^* in a predetermined area sent from the chromaticity value calculation unit 73 and the chromaticity data sent from the shade guide chromaticity data storage unit 114. Based on the above, differences ΔE, ΔL ^* , Δa ^* , Δb ^* between the chromaticity of the tooth and the chromaticity of the shade guide are calculated. Further, the difference calculation unit 130 calculates the chromaticity differences ΔE, ΔL ^* , Δa ^* , and Δb ^* of the teeth and the shade guide for each pixel with reference to a predetermined position of the teeth. Here, the “predetermined position” can exemplify the intersection of the vertical scanning line Y and the horizontal scanning line X displayed in each of the tooth image and the shade guide image as described later. Difference Delta] E chromaticity values in the predetermined ^{area, ΔL *, Δa *, Δb} * and, teeth calculated for each pixel and the shade guide chromaticity difference ^{ΔE, ΔL *, Δa *,} Δb * , respectively It is transferred to the image display GUI unit 115.

When the image display GUI unit 115 receives the RGB image and the chromaticity value L ^* a ^* b ^{* of the} measurement region, the shade guide number, and the like obtained from each unit, the image display GUI unit 115 displays a screen as shown in FIG. On the display screen.
As shown in FIG. 13, the image display GUI unit 115 displays the color image A to be measured at the upper center of the display screen, and further, the chromaticity value L ^* a ^* b of the measurement target to the right of the color image A. ^* Distribution image B is displayed. Specifically, the image display GUI unit 115 displays a vertical scanning line Y movable up and down and a horizontal scanning line X movable right and left on the distribution image B, and an intersection of the scanning lines Y and X. The color difference distribution with reference to the chromaticity at the reference position designated in FIG.

Further, the image display GUI unit 115 displays the color difference change along the vertical scanning line Y or the horizontal scanning line X as a line graph C on the upper right side of the screen. Since the scanning lines Y and X shown on the distribution image B are arbitrarily movable by the user, the image display GUI unit 115 scans each time the scanning line Y or X is scanned. The positional information of the later scanning lines Y and X is transferred to the difference calculation unit 130, and the tooth chromaticity differences ΔE, ΔL ^* , Δa ^* , Δb ^* with respect to the intersection of the scanning lines Y and X are again set as pixels. Calculate every time. Then, the image display GUI unit 115 acquires the color difference distribution based on the reference position after scanning from the difference calculation unit 130, and promptly displays this information.

Further, the image display GUI unit 115 displays on the color image A a plurality of measurement regions set in the measurement region setting process of the determination calculation unit 81 (see FIG. 4). Here, when any one of the plurality of measurement regions Q is designated by a user such as a doctor, the color image D of the shade guide selected to approximate the color of the designated measurement region Q is displayed in the center of the screen. Further, the distribution image E of the chromaticity value L ^* a ^* b ^* is displayed on the right side of the color image D. The image display GUI unit 115 provides the difference calculation unit 130 with the shade guide chromaticity differences ΔE, ΔL ^* , Δ based on the chromaticity of the teeth at the intersection of the scanning lines Y and X shown in the distribution image B. Δa ^* and Δb ^* are calculated for each pixel, and the calculated color difference distribution of the shade guide is displayed on the distribution image E.
Also in this distribution image E, the vertical scanning line Y and the horizontal scanning line X are shown as in the distribution image B, and the image display GUI unit 115 displays the color difference along the vertical scanning line Y or the horizontal scanning line X. The change is displayed as a line graph F at the lower right of the screen.

In addition, the image display GUI unit 115 displays a list of information G regarding the shade guide selected corresponding to the measurement region Q in the lower left portion of the screen. Here, three shade guide numbers selected in the shade guide selection process of the determination calculation unit 81 shown in FIG. 4 are displayed in ascending order of the spectrum determination value Jvalue. Further, the image display GUI unit 115 displays the spectrum determination value Jvalue and the differences ΔE, ΔL ^* , Δa ^* , Δb ^* between the chromaticity of the tooth and the chromaticity of the shade guide for each shade guide number.

  Further, the image display GUI unit 115 displays a switching button H that enables switching of the display mode at the upper left portion of the screen. When these switching buttons H are selected, the image display GUI unit 115 switches the display screen to a screen as shown in FIG.

  As shown in FIG. 14, the image display GUI unit 115 displays a color image of the measurement target tooth near the upper left portion of the screen. Further, the image display GUI unit 115 displays a list of color images of the shade guide under a predetermined illumination light at the lower center of the screen, and displays the shade guide number below the list.

  In FIG. 14, “none”, “D65”, and “A” are selected as the illumination light. Here, “none” corresponds to displaying the spectral reflectance itself. This illumination light can be arbitrarily changed as a pull-down menu. Thereby, the user can easily select the desired illumination light, and can confirm the shade of the shade guide under the desired illumination light. Moreover, the shade guide number selected in the shade guide selection process of the shade guide number determination part 80 shown in FIG. 4 among each shade guide is displayed on the left part of the image of each shade guide as SG-No. Yes.

  In addition, the image display GUI unit 115 adjoins the measurement target vital tooth and a predetermined shade guide at the upper center of the screen so that the color tone of the measurement target vital tooth and the shade guide can be easily compared. The comparison area R for displaying the image is displayed. This is an area provided to eliminate such problems because it is difficult to make a detailed comparison of color tone if it is displayed at a distant place on the screen when comparing a vital tooth and a shade guide. .

  For example, when a shade guide to be compared is moved from the shade guides displayed on the screen to the comparison region R by an input operation such as drag and drop by the user, the image display GUI unit 115 moves to the comparison region R. The shade guide number thus transferred is transferred to the image composition unit 116. Thereby, the image composition processing by the image composition unit 116 is performed, and one composite image in which part of the vital tooth and part of the shade guide are displayed adjacent to each other is displayed in the comparison region R.

Hereinafter, the image composition unit 116 according to the present embodiment will be described in detail.
As shown in FIG. 15, the image synthesis unit 116 includes an image acquisition unit 117, an image extraction unit (image extraction unit) 118, a synthesis reference line setting unit (synthesis reference line setting unit) 119, a synthesis unit (synthesis unit) 120, And a correction unit (correction means) 121.
In the image composition unit 116 having such a configuration, the shade guide number from the image display GUI unit 115 is input to the image acquisition unit 117. The image acquisition unit 117 acquires a color image (second captured image) of the shade guide specified by the input shade guide number from the shade guide reference image data storage unit 82 in the shade guide number determination unit 80.

  Further, the image acquisition unit 117 acquires a color image (first captured image) of the vital tooth to be measured from the color image creation processing unit 112. The color image of the shade guide and the color image of the vital tooth that is the measurement target thus obtained are transferred to the image extraction unit 118. The image extraction unit 118 extracts a vital tooth region (hereinafter referred to as “measurement target image”) from the color image of the vital tooth to be measured, and at the same time, the shade guide portion, that is, the vital tooth, from the shade guide color image. Is extracted (hereinafter referred to as “comparison target image”).

  Here, with respect to the extraction method of the measurement target image and the comparison target image, the method employed in the determination calculation unit 81 described above, that is, “Determination of vital tooth region 1” or “Identification of vital tooth region”. For example, both images may be temporarily displayed on the monitor 4 and the user may specify the area. After extracting the measurement target image and the comparison target image, the image extraction unit 118 transfers these pieces of information to the composite reference line setting unit 119.

  The combination reference line setting unit 119 sets a combination reference line that is used as a reference when combining these images in the measurement target image and the comparison target image extracted by the extraction unit 118. The composite reference line setting unit 119 sets, for example, a line that passes through the center of gravity of the color measurement target image and is parallel to the longitudinal axis, in other words, in the longitudinal direction, as a composite reference line on the color measurement target image. Similarly, the synthesis reference line setting unit 119 sets a line passing through the center of gravity of the comparison target image and parallel to the vertical axis direction on the comparison target image as a synthesis reference line. Specifically, as shown in FIG. 16, a line Ymax having a maximum length in the vertical axis direction is obtained, and a line Xref passing through the midpoint of the line Ymax and perpendicular to Ymax is obtained. Further, the midpoint of this Xref is obtained, and a line passing through this midpoint and parallel to the line Ymax is set as a composite reference line.

When the composite reference line is set according to the above-described procedure, the composite reference line setting unit 119 transfers the measurement target image and the comparison target image after the composite reference line is set to the composite unit 120.
The combining unit 120 combines the measurement target image and the comparison target image along the combination reference line. Specifically, the measurement target image is divided into two along the synthesis reference line, and the comparison target image is divided into two along the synthesis reference line. Subsequently, a composite image is created by pasting the left side of the measurement target image after division and the right side of the comparison target image after division. At this time, it is preferable to paste the images so that the midpoints of the composite reference line coincide with each other.

  As a result, as shown in FIG. 17, a composite image in which the left half of the measurement target image and the right half of the comparison target image are bonded together is created. The combining unit 120 transfers this combined image to the correction unit 121. The correction unit 121 determines whether or not the contours on the composite reference line match in the composite image, and if they do not match, corrects at least one of the measurement target image and the comparison target image. Match the contours.

For example, the outlines are matched by enlarging or reducing either or both. At this time, the enlargement / reduction ratio is preferably the same in the vertical axis direction and the horizontal axis direction, but may be different. By performing such correction, as shown in FIG. 18, the contours on the composite reference line coincide with each other, and a natural composite image is created. The corrected composite image is transferred from the correction unit 121 to the image display GUI unit 115.
When receiving the composite image from the correction unit 121, the image display GUI unit 115 displays the composite image on the monitor 4. Specifically, the image display GUI unit 115 displays a composite image in the comparison region R shown in FIG.

  The image display GUI unit 115 displays a composite reference line (composite reference line changing unit) in the composite image displayed in the comparison region R. The composite reference line is configured to be freely adjustable with a mouse or the like. When there is an instruction to change the position of the combined reference line, the combined reference line setting unit 119 sets the changed combined reference line based on the instruction. The user can freely change the display ratio of the vital tooth and the shade guide by moving the composite reference line to the left and right.

Hereinafter, the update process of the composite image when the composite reference line is moved by the user will be described in detail. In the following description, a composite image already displayed in the comparison region R is defined as an initial composite image, and a composite reference line in the initial composite image is defined as an initial composite reference line.
For example, in the initial composite image displayed in the comparison region R in FIG. 14, when the composite reference line is moved to the right by the user as shown in FIG. 19, information on the amount of movement of the composite reference line is displayed on the image. The image is transferred from the GUI unit 115 to the image composition unit 116. As a result, the composite reference line setting unit 119 can grasp where the updated composite reference line is set in the initial composite image.

The composite reference line setting unit 119 first obtains a line Xmax that passes through the right end of the contour of the initial composite image and is parallel to the initial composite reference line, and then passes through the right end and passes through the initial composite reference line. A line Yref perpendicular to is obtained. Then, the synthesis reference line setting unit 119 obtains the division ratio a: b of the length of the line Yref between the initial synthesis reference line and the line Xmax based on the updated synthesis reference line.
In this way, when the synthesis reference setting unit 119 obtains the division ratio a: b, the measurement target image and the comparison target image acquired from the image extraction unit 118 in the first image synthesis process are set to the same division ratio as described above. A new composite reference line to be divided is set, and the measurement target image and the comparison target image in which the new composite reference line is set are output to the combining unit 120. Then, a new composite image is created by combining based on the new composite reference line by the combining unit 120, and the new combined image is corrected by the correcting unit 121. Then, the corrected composite image is transferred to the image display GUI unit 115. The image display GUI unit 115 displays the new composite image received from the correction unit 121 in the comparison area R of the monitor 4. As a result, a new composite image based on the updated composite reference line is presented to the user.

  As described above, according to the dental colorimetry system according to this embodiment, the divided measurement target image obtained by dividing the measurement target image along the composite reference line and the comparison target image divided along the composite reference line. A composite image is created by pasting the divided comparison target images together, and this composite image is provided to the user. Thereby, the user can compare the color tone of the vital tooth and the shade guide very easily.

Furthermore, since the composite reference line is configured to be movable by the user in the composite image, the user can change the area ratio between the measurement target image and the comparison target image in the composite image by moving the composite reference line. In addition, it is possible to freely change the location for color tone comparison.
Furthermore, since the shade guide used for color tone comparison is also configured to be changeable, the user can freely select a desired shade guide number as a comparison target. As a result, it is possible to display a composite image based on a desired shade guide and vital teeth, and it is possible to compare the color tone of various shade guides with the color tone of vital teeth.

In the above-described embodiment, the setting procedure of the composite reference line by the composite reference line setting unit 119 is an example, and the setting of the composite reference line is not limited to this example. For example, it is good also as setting a synthetic | combination reference line along the vertical axis direction in the location where the length of the vertical axis direction in the measurement target image and the comparison target image is maximum.
Further, the colorimetric values near the composite reference line may be calculated in the vital tooth and the shade guide, respectively, and these calculation results may be displayed as numerical data on the same screen. Specifically, as illustrated in FIG. 20, when the position of the composite reference line is changed in the composite reference line setting unit 119, the position information of the composite reference line after the change is changed from the composite reference line setting unit 119 to the color. It is transferred to the degree value calculation unit 73. As a result, the predetermined area set in advance is changed to the position corresponding to the changed composite reference line. Next, the chromaticity value calculation unit 73 again calculates the average value of the chromaticity values L ^* a ^* b ^* in the predetermined area after the change from the subject spectrum, and outputs the average value to the image display GUI unit 115 and the difference calculation unit 130. To do.
Furthermore, the difference calculation unit 130 also calculates a difference between the chromaticity value of the vital tooth and the chromaticity value of the shade guide based on the average value of the chromaticity values L ^* a ^* b ^* in the predetermined area after the design change. However, this difference may also be displayed.

  In the above embodiment, the case where the measurement target image and the comparison target image are divided based on the composite reference line has been described. However, the division may be performed on the entire image. For example, by extending the composite reference line set on the measurement target image, a composite reference line is set in the entire color image of the vital tooth in which the measurement target is photographed, and the vital tooth is aligned along the composite reference line. The color image may be divided into two image pieces. Similarly, with respect to a shade guide image in which the comparison target is photographed, the entire image is divided into two image pieces based on the composite reference line, and these image pieces are bonded to each other so that the measurement target vital tooth or comparison A composite image including not only the target shade guide but also background information such as gums may be created.

In FIG. 14, the case where one composite reference line is set along the vertical axis is described. However, the composite reference line may be set along the horizontal axis, and the number of the set reference lines is also set. It can be set arbitrarily in vertical and horizontal directions.
For example, as shown in FIG. 21, it is possible to create a composite image by combining four image pieces by setting the composite reference line to one vertical and one horizontal. Similarly, FIG. 22 shows a case where the composite reference line is two vertical lines and one horizontal line, FIG. 23 shows a case where the composite reference line is three vertical lines and one horizontal line, and FIG. , Each of the combined images in the case of one horizontal is shown.
Further, even in a composite image in which a plurality of composite reference lines are set, each composite reference line can be moved by the user.

  Furthermore, the number of shade guides that can be compared at one time is not limited to one, and a configuration may be used in which a plurality of shade guides and vital teeth are compared at a time, and a plurality of types of shade guides are adjacent to vital teeth. Can also be displayed. For example, in FIG. 16, among the four divided areas, a part of the shade guide is displayed at the upper left, a shade guide different from the upper left shade guide is displayed at the lower left, and vital teeth are displayed in the two areas on the right. It can also be made to do. Similarly, it is possible to make a configuration in which the shade guide and a plurality of vital teeth are compared at a time.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus and a cradle according to an embodiment of the present invention. It is a figure which shows the spectrum of the light source shown in FIG. It is a figure explaining signal correction. 1 is a block diagram illustrating a schematic configuration of a dental color measurement device according to an embodiment of the present invention. It is a figure which shows the schematic internal structure of the spectrum estimation calculating part shown in FIG. It is a figure explaining input gamma correction. It is a figure which shows an example of the low-pass filter applied to R signal and B signal in pixel interpolation calculation. It is a figure which shows an example of the low pass filter applied to G signal in pixel interpolation calculation. It is a figure which shows an example of the reflection spectrum (sample number n = 2) of a vital tooth. It is a figure which shows an example of the reflectance spectrum (sample number n = 5) of a gum. It is explanatory drawing explaining the method of specifying the area | region of the vital tooth which concerns on one Embodiment of this invention. It is a figure which shows an example of the measurement area | region set by a measurement area | region setting process. It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a figure which shows schematic structure of the image synthetic | combination part shown in FIG. It is explanatory drawing used for description about the setting procedure of a synthetic | combination reference line. It is a figure which shows an example of the synthesized image produced by the synthetic | combination part. It is a figure which shows an example of the synthesized image after the correction | amendment by the correction | amendment part. It is explanatory drawing used for description about the reset of a synthetic | combination reference line when a synthetic | combination reference line is moved by the user. It is explanatory drawing for demonstrating the effect | action of the synthetic | combination reference line setting part in FIG. It is the figure which showed the synthetic | combination image when a synthetic | combination reference line is 1 length and 1 width. It is the figure which showed the synthetic | combination image in case a synthetic | combination reference line is 2 vertical and 1 horizontal. It is the figure which showed the synthetic | combination image in case a synthetic | combination reference line is set to 3 length and 1 width. It is the figure which showed the synthetic | combination image when a synthetic | combination reference line is 4 length and 1 width.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Cradle 3 Dental colorimetric device 4 Display device 10 Light source 70 Chromaticity calculation part 71 Spectrum estimation calculation part 72 Observation spectrum calculation part 73 Chromaticity value calculation part 80 Shade guide number determination part 81 Determination calculation part 82 Shade guide Reference image data storage unit 114 Shade guide chromaticity data storage unit 115 Image display GUI unit 116 Image composition unit 117 Image acquisition unit 118 Image extraction unit 119 Composition reference line setting unit 120 Composition unit 121 Correction unit 130 Difference calculation unit

Claims (10)

An image composition device that synthesizes a first captured image in which a measurement target is captured and a second captured image in which a comparison target to be compared with the measurement target is captured to create a single image,
Image extraction means for extracting a measurement target image from the first captured image and extracting a comparison target image from the second captured image;
Combined reference line setting means for setting a combined reference line on the measurement target image and the comparison target image, respectively;
Synthesizing means for creating a synthesized image by synthesizing the measurement object image and the comparison object image along the synthesis reference line;
In the synthesized image, an image comprising correction means for correcting at least one of the measurement target image and the comparison target image so that the contours match when the contours on the composite reference line do not match. Synthesizer.   The composite reference line setting means sets a line passing through the center of gravity of the measurement target image as the composite reference line on the measurement target image, and a line passing through the center of gravity of the comparison target image as the composite reference line The image composition device according to claim 1, wherein the image composition device is set on an image.   The composite reference line setting means sets the composite reference line on the measurement target image along the vertical axis direction at a position where the length of the measurement target image is maximum in the vertical axis direction, and the comparison target The image synthesizing apparatus according to claim 1, wherein the synthesis reference line is set on the comparison target image along the vertical axis direction at a position where the length of the image in the vertical axis direction is maximum.   The image composition apparatus according to claim 1, further comprising a composition reference line changing unit configured to change the composition reference line from outside in the composition image.   The image synthesizing apparatus according to claim 1, wherein the correction unit reduces or enlarges at least one of the measurement target image and the comparison target image.   The image composition device according to claim 5, wherein the measurement object is a vital tooth, and the comparison object is the shade guide. An image composition apparatus according to claim 1;
A colorimetric value calculating means for calculating a colorimetric value near the composite reference line in the measurement target image;
Difference calculating means for calculating a difference between a colorimetric value near the composite reference line of the measurement target image and a colorimetric value near the composite reference line of the comparison target image;
At least one of the colorimetric value in the vicinity of the composite reference line of the measurement target image, the colorimetric value in the vicinity of the composite reference line of the comparison target image, and the difference between the colorimetric values is the composite image. A dental colorimetric device comprising display control means for displaying on the same screen. An imaging device for imaging the oral cavity;
An image synthesis device that creates a single image by synthesizing a first captured image in which a measurement target is captured by the imaging device and a second captured image in which a comparison target to be compared with the measurement target is captured; A dental colorimetric device,
The image composition device includes:
Image extraction means for extracting a measurement target image from the first captured image and extracting a comparison target image from the second captured image;
Combined reference line setting means for setting a combined reference line on the measurement target image and the comparison target image, respectively;
Synthesizing means for creating a synthesized image by synthesizing the measurement object image and the comparison object image along the synthesis reference line;
A correction unit that corrects at least one of the measurement target image and the comparison target image so that the contours match when the contours in the composite reference line do not match in the composite image; Color measuring system. An image creation method for creating a single image by combining a first captured image in which a measurement target is captured and a second captured image in which a comparison target to be compared with the measurement target is captured,
An image extraction process of extracting a measurement target image from the first captured image and extracting a comparison target image from the second captured image;
A combined reference line setting process for setting a combined reference line on the measurement target image and the comparison target image;
Combining the measurement object image and the comparison object image along the combining reference line to create a combined image;
An image comprising: a correction process for correcting at least one of the measurement target image and the comparison target image so that the contours match when the contours in the composite reference line do not match in the composite image Synthesis method. An image creation program for creating a single image by combining a first captured image in which a measurement target is captured and a second captured image in which a comparison target to be compared with the measurement target is captured,
An image extraction process for extracting a measurement target image from the first captured image and extracting a comparison target image from the second captured image;
A combined reference line setting process for setting a combined reference line on the measurement target image and the comparison target image;
A synthesis process for creating a synthesized image by synthesizing the measurement object image and the comparison object image along the synthesis reference line;
In the composite image, when the contours on the composite reference line do not match, the computer executes a correction process for correcting at least one of the measurement target image and the comparison target image so that the contours match. Image composition program to let you.

Images