TWI749909B - Image analysis system and image analysis method - Google Patents

Abstract

An image analysis system and an image analysis method are provided. An image analysis system includes a storage device, an image capture device and a processing circuit. The storage device stores a tooth-position identify module and a dental plaque identify module. The image capture module captures white light image and UV light image corresponding to teeth. The processing circuit inputs the white light image into the tooth-position identify module and the tooth-position identify module outputs a teeth image with multiple teeth areas corresponding to teeth of multiple teeth positions. The processing circuit inputs the UV light image into the dental plaque identify module and the dental plaque identify module outputs a dental plaque distribution image according the UV light image. The processing circuit analyses each dental plaque distribution of the teeth areas and creates a dental plaque value table.

Description

Image analysis system and image analysis method

The present invention relates to an image analysis, and particularly relates to an image analysis system and an image analysis method.

The number of dental plaque is a reference index to measure a person's dental health. Traditionally, the detection of dental plaque is performed by a physician comparing and examining each tooth with the naked eye, and manually registering the plaque distribution of each tooth on a form. However, this recording method is not only inefficient, but also consumes the time and energy of medical staff.

The invention provides an image analysis system and an image analysis method, which can automatically determine the tooth number and the distribution of dental plaque.

The image analysis system of the present invention includes a storage device, an image capturing device and a processing circuit. The storage device is used for storing the tooth position identification module and the dental plaque identification module. The image capturing device is used to obtain white light images and ultraviolet light images corresponding to the teeth. The processing circuit is coupled to the storage device and the image capturing device. In this embodiment, the processing circuit inputs the white light image to the tooth position recognition module, and the tooth position recognition module outputs tooth images defined with a plurality of tooth regions corresponding to a plurality of tooth positions of the tooth. In this embodiment, the processing circuit inputs the ultraviolet light image to the dental plaque recognition module, and the dental plaque recognition module outputs the plaque distribution image according to the ultraviolet light image. In this embodiment, the processing circuit compares the plaque distribution image and the tooth image with the tooth position recognition module to analyze the individual plaque distribution in the tooth region, and establish a plaque value table corresponding to the tooth .

The image analysis method of the present invention includes the following steps: obtaining white light images and ultraviolet light images corresponding to the teeth through the image capturing device; Tooth images of multiple tooth regions and multiple tooth positions of the tooth; input the ultraviolet light image to the plaque recognition module so that the plaque recognition module outputs the plaque distribution image according to the ultraviolet light image; and The tooth position recognition module compares the plaque distribution image and the tooth image to analyze the individual plaque distribution in some tooth regions, and establish a plaque value table corresponding to the teeth.

Based on the above, the image analysis system and the image analysis method of the present invention can automatically determine the tooth number and the distribution of dental plaque, effectively reducing the time required for the inspection of dental plaque.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

In order to make the content of the present invention more comprehensible, the following embodiments are specifically cited as examples on which the present invention can indeed be implemented. In addition, wherever possible, elements/components/steps with the same reference numbers in the drawings and embodiments represent the same or similar parts.

And, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meaning in the context of related technologies and the present invention, and will not be interpreted as idealized or excessive The formal meaning, unless explicitly defined as such in this article.

FIG. 1 is a schematic diagram of an image analysis system according to an embodiment of the invention. 1, in this embodiment, the image analysis system 100 includes a storage device 150, an image capturing device 120 and a processing circuit 130. The storage device 150 is used to store the tooth position recognition module 151 and the dental plaque recognition module 152. The image capturing device 120 is used to obtain white light images and ultraviolet light images corresponding to the teeth. The processing circuit 130 is coupled to the storage device 150 and the image capturing device 120. In this embodiment, the processing circuit 130 inputs the white light image to the tooth position recognition module 151, and the tooth position recognition module 151 outputs tooth images defined with multiple tooth regions corresponding to multiple teeth positions of the tooth. In addition, the processing circuit 130 inputs the ultraviolet light image to the dental plaque recognition module 152, and the dental plaque recognition module outputs the plaque distribution image according to the ultraviolet light image. In this way, the processing circuit 130 can compare the plaque distribution image and the tooth image with the tooth position recognition module to analyze the respective plaque distribution in the multiple tooth regions, and create a tooth corresponding to the tooth. Plaque value table. Therefore, the image analysis system of the present invention can automatically determine the tooth number and the distribution of dental plaque, effectively reducing the time required for the inspection of dental plaque.

In this embodiment, the storage device 150 may include, for example, random-access memory (Random-Access Memory; RAM), read-only memory (Read-Only Memory; ROM), optical disc (Optical disc), and magnetic disk (Magnetic disk). ), Hard drive, Solid-state drive, Flash drive, Security digital (SD) card, Memory stick, compact flash ( Compact flash; CF) card or any type of storage device, but the present invention is not limited to this. In this embodiment, the image capturing device 120 may include, for example, a visible light camera, an infrared (Infrared light) camera, a thermal imaging camera, and a color image depth camera, but the invention is not limited to this. In this embodiment, the processing circuit 130 may include, for example, a central processing unit (CPU), a microprocessor (Microprocessor Control Unit, MCU), or a field programmable gate array (Field Programmable Gate Array, FPGA). The invention is not limited to this.

FIG. 2 is a flowchart of an image analysis method according to an embodiment of the invention. 1 and 2, the image analysis system 100 may perform the following steps S210 to S240 to create a table of plaque values corresponding to teeth. In step S210, the image analysis system 100 can illuminate the teeth through the white light source 101. Moreover, when the white light source 101 illuminates the teeth, the image analysis system 100 can capture the teeth through the image capturing device 120 to obtain white light images. In step S220, the image analysis system 100 can illuminate the teeth via the ultraviolet light source 102. In addition, when the ultraviolet light source 102 illuminates the teeth, the image analysis system 100 can photograph the teeth through the image capturing device 120 to obtain an ultraviolet image.

It should be noted that, referring to FIGS. 3A to 3E, in step S210 and step S220, the imaging device 120 may correspond to different sampling positions as shown in FIGS. 3A to 3E, and shoot multiple sets of white light images and ultraviolet light images, but The present invention is not limited to this. 3A is a schematic diagram of a frontal image 310 of the buccal side of a tooth according to an embodiment of the present invention. FIG. 3B is a schematic diagram of a silhouette image 320 of the buccal side of a tooth according to an embodiment of the present invention. FIG. 3C is a schematic diagram of another silhouette image 330 of the buccal side of a tooth according to an embodiment of the present invention. 3D is a schematic diagram of an upper jaw image 340 on the occlusal side of a tooth according to an embodiment of the present invention. 3E is a schematic diagram of a lower jaw image 350 on the occlusal side of a tooth according to an embodiment of the present invention. In other words, the imaging device 120 can obtain white light images and ultraviolet light images from the same sampling position corresponding to the tooth at different times. In addition, the image capturing device 120 can obtain multiple sets of white light images and ultraviolet light images as shown in FIGS. 3A to 3E from multiple different sampling positions corresponding to the teeth.

Incidentally, in this embodiment, the image analysis system 100 may include a light source driving circuit 110 for driving the white light source 101 and the ultraviolet light source 102. For example, in step S210, the light source driving circuit 110 can drive the white light source 101 so that the white light source 101 illuminates the teeth. In addition, in step S220, the light source driving circuit 110 can drive the ultraviolet light source 102 so that the ultraviolet light source 102 illuminates the teeth. By driving different light sources through the light source driving circuit 110, the image capturing device 120 can obtain the white light image and the ultraviolet light image respectively. It is worth noting that in this embodiment, the brightness value of the ultraviolet light source 102 may be greater than the brightness value of the white light source 101, so that the subsequent image processing has a better effect, but the present invention is not limited to this. In other embodiments, the brightness value of the white light source 101 may be greater than the brightness value of the ultraviolet light source 102. Alternatively, the length of the exposure time for the image capturing device 120 to obtain the ultraviolet image may be greater than the length of the exposure time for the white light image to obtain an effect similar to that the brightness value of the ultraviolet light source 102 is greater than the brightness value of the white light source 101. Alternatively, the image capturing device 120 may set the white balance setting for obtaining the ultraviolet light image different from the white balance setting for obtaining the white light image, so as to obtain an effect similar to that the brightness value of the ultraviolet light source 102 is greater than the brightness value of the white light source 101.

Returning to FIG. 2, in step S211, referring to FIGS. 3A to 4, the processing circuit 130 can input the white light image as shown in FIGS. 3A to 3E to the tooth position recognition module 151, so that the tooth position recognition module 151 can output A tooth image 400 of a plurality of tooth regions 401-1 to 401-N (N is a positive integer greater than 1) corresponding to a plurality of tooth positions of the tooth is defined. Specifically, the processing circuit 130 can input the white light images shown in FIGS. 3A to 3E to the tooth position recognition module 151. In addition, the processing circuit 130 may mark the tooth position of the tooth image 400 according to the tooth database, so as to identify a plurality of tooth features in the tooth image 400 to define the plurality of tooth positions in the tooth image 400 Multiple tooth areas 401-1~401-N. For example, the processing circuit 130 can use models such as faster-RCNN and YoLo to establish a deep learning object detection algorithm, and use the feature extraction method VGG16 or Resnet101 as a backbone network to complete the tooth inference model. Then, the processing circuit 130 can select the position of each tooth in the white light image according to the tooth inference model frame to output a tooth image 400 defined with a plurality of tooth regions 401-1 to 401-N corresponding to a plurality of tooth positions of the tooth .

In step S221, referring to FIGS. 5A to 5C, the processing circuit 130 may input the ultraviolet light image 510 to the dental plaque recognition module 152, so that the dental plaque recognition module 152 outputs the plaque distribution according to the ultraviolet light image 510 Image 520. Specifically, the dental plaque recognition module 152 can convert the ultraviolet light image 510 into the HSV format, and use the fluorescence range threshold (or preset range) to find the position of the dental plaque in the ultraviolet light image 510 . For example, the original ultraviolet image 510 may be in the format of the RGB color space. In addition, the plaque recognition module 152 can convert the ultraviolet image 510 from the RGB color space to the HSV color space. Furthermore, the plaque recognition module 152 can determine a plurality of plaque pixels 521 in the ultraviolet light image 510 after the color space conversion according to a preset range, so as to generate a plaque distribution image 520 as shown in FIG. 5B. In this embodiment, the plaque distribution image 520 in FIG. 5B may be a binary image. Then, the plaque recognition module 152 can stack the plaque distribution image 520 of FIG. 5B and the ultraviolet light image 510 of FIG. 5A to generate a stacked image 530 including a plurality of plaque pixels 531 as shown in FIG. 5C , But the present invention is not limited to this. In one embodiment, the plaque recognition module 152 can stack the plaque distribution image 520 of FIG. 5B and the tooth image 400 (white light image) similar to that of FIG. 4 to generate a white light image of the tooth image. Stacked images of the background, so that it can also be used to show the distribution of dental plaque information. The image analysis system 100 can also display the stacked images 530 through the display device 140 to provide information on the distribution of dental plaque for the medical staff to make judgments. In addition, in other embodiments, the plaque recognition module 152 can convert the ultraviolet light image 510 from the format of the first color space to the format of the second color space, wherein the first color space and the second color space The space can be selected from various color spaces such as RGB color space, HSV color space or HSL color space, and is not limited to the above-mentioned embodiments.

It should be noted that, referring to FIGS. 3A to 4, in step S211, in addition, in step S211, the tooth position recognition module 151 can selectively perform image pre-processing on the white light image to generate the tooth image 400. For example, the tooth position recognition module 151 can crop the white light image to remove the part included in the specific area of the gum in the white light image. In addition, the tooth position recognition module 151 can perform image contrast adjustment, brightness adjustment, and image normalization processing on the white light image. Similarly, in step S221, the plaque recognition module 152 can selectively perform image pre-processing on the ultraviolet image 510 to generate a plaque distribution image 520.

Returning to FIG. 2, in step S230, the processing circuit 130 compares the plaque distribution image 520 and the tooth image 400 through the tooth position identification module 151 to analyze individual teeth in the plurality of tooth regions 401-1 to 401-N. Plaque distribution. For example, the tooth position recognition module 151 can compare the plaque distribution image 520 of FIG. 5B and the tooth image 400 of FIG. The number of plaque pixels 521. Next, in step S240, the processing circuit 130 creates a table of plaque values corresponding to the teeth based on the above information. For example, the tooth position recognition module 151 can create a plaque value table according to the respective plaque values of the multiple tooth positions. That is, the tooth position identification module 151 can calculate the plaque value according to the number of the respective plaque pixels 521 of the plurality of tooth positions, and establish the value as shown in Table 1 below according to the plaque value Table of dental plaque values. Tooth number 11 12 13 14 15 16 17 18 Plaque value Tooth number twenty one twenty two twenty two twenty four 25 26 27 28 Plaque value Tooth number 31 32 33 34 35 36 37 38 Plaque value Tooth number 41 42 43 44 45 46 47 48 Plaque value Table 1

Table 1 is a numerical table of dental plaque according to an embodiment of the present invention. The tooth numbers in Table 1 are based on the tooth position notation proposed by the Fédération Dentaire Internationale (FDI). Specifically, each tooth is represented by two digits. The first digit can be used to indicate the quadrant in which the tooth is located, where the quadrant can, for example, include positions corresponding to the upper right, upper left, lower left, and lower right of the tooth. And, the second digit can be used to indicate the position of the teeth, for example, from the middle incisor to the third molar, 1-8 respectively. In addition, the plaque values in Table 1 are obtained by the tooth position recognition module 151, which respectively calculates the number of plaque pixels 521 in the tooth region 401-1~401-N corresponding to each tooth number. The value of.

In this embodiment, the processing circuit 130 can display at least one of the plaque value table and the stacked image 530 as shown in Table 1 through the display device, so as to effectively provide plaque distribution information for medical personnel to make judgments. By the way, in one embodiment, the image analysis system 100 can also be integrated into a mobile electronic device such as a mobile phone, so that the user can check the plaque distribution of the teeth by themselves.

In summary, the image analysis system and image analysis method of the present invention can automatically determine the number of teeth and the distribution of dental plaque, effectively reducing the time required for checking dental plaque. In this way, not only can the efficiency of medical staff be increased, but also the waiting time allowed by the testees can be reduced.

100: image analysis system 101: White light source 102: UV light source 110: Light source drive circuit 120: Capture device 130: processing circuit 140: display device 150: storage device 151: Tooth recognition module 152: Dental plaque recognition module 310: Front image 320, 330: Silhouette 340: Upper jaw image 350: jaw image 400: tooth image 401-1~401-N: tooth area 510: Ultraviolet image 520: Image of plaque distribution 521, 531: plaque pixels 530: Stacked images S210, S211, S220, S221, S230, S240: steps

FIG. 1 is a schematic diagram of an image analysis system according to an embodiment of the invention. FIG. 2 is a flowchart of an image analysis method according to an embodiment of the invention. 3A is a schematic diagram of a frontal image of the buccal side of a tooth according to an embodiment of the present invention. 3B is a schematic diagram of a profile image of the buccal side of a tooth according to an embodiment of the present invention. FIG. 3C is a schematic diagram of another silhouette image of the buccal side of a tooth according to an embodiment of the present invention. 3D is a schematic diagram of an image of the upper jaw on the occlusal side of a tooth according to an embodiment of the present invention. 3E is a schematic diagram of a lower jaw image of the occlusal side of a tooth according to an embodiment of the present invention. Fig. 4 is a schematic diagram of tooth position identification according to an embodiment of the present invention. FIG. 5A is a schematic diagram of an ultraviolet light image according to an embodiment of the invention. 5B is a schematic diagram of an image of dental plaque distribution according to an embodiment of the present invention. FIG. 5C is a schematic diagram of stacked images according to an embodiment of the invention.

100: image analysis system

101: White light source

102: UV light source

110: Light source drive circuit

120: Capture device

130: processing circuit

140: display device

150: storage device

151: Tooth recognition module

152: Dental plaque recognition module

Claims (18)

An image analysis system includes: a storage device for storing a tooth position recognition module and a dental plaque recognition module; an image capturing device for obtaining a white light image and an ultraviolet light image corresponding to a tooth And a processing circuit coupled to the storage device and the image capturing device, wherein the processing circuit inputs the white light image to the tooth position identification module, and the tooth position identification module output is defined to correspond to the tooth A tooth image of a plurality of tooth regions of a plurality of tooth positions, wherein the tooth position recognition module performs an image pre-processing on the white light image to generate the tooth image, and the tooth position recognition module compares the teeth according to a tooth database The tooth image is marked with a tooth position to define the tooth regions of the tooth positions in the tooth image by identifying a plurality of tooth features in the tooth image by the image, wherein the processing circuit inputs the ultraviolet light image To the dental plaque recognition module, and the dental plaque recognition module outputs a dental plaque distribution image according to the ultraviolet light image, wherein the processing circuit compares the dental plaque distribution with the dental position recognition module The image and the tooth image are used to analyze the distribution of a plaque in each of the tooth regions, and establish a plaque value table corresponding to the tooth. The image analysis system according to claim 1, further comprising: a white light source coupled to the processing circuit; and An ultraviolet light source is coupled to the processing circuit, wherein when the processing circuit operates the white light source to illuminate the tooth, the processing circuit captures the tooth through the imaging device to obtain the white light image, wherein when the processing circuit operates the tooth When the ultraviolet light source illuminates the tooth, the processing circuit captures the tooth through the imaging device to obtain the ultraviolet light image, wherein the imaging device obtains the white light image from the same sampling position corresponding to the tooth at different times, and The UV image. The image analysis system according to claim 2, wherein the brightness value of the ultraviolet light source is greater than the brightness value of the white light source. The image analysis system according to claim 1, wherein the image capturing device is used to obtain multiple sets of white light images and ultraviolet light images from a plurality of different sampling positions corresponding to the tooth. The image analysis system according to claim 1, wherein the plaque recognition module converts the ultraviolet light image from an RGB color space to an HSV color space, and determines the converted color space according to a preset range A plurality of plaque pixels in the ultraviolet light image are used to generate the plaque distribution image. The image analysis system according to claim 5, wherein the plaque distribution image is a binary image, and the tooth position recognition module compares the plaque distribution image with the tooth image to calculate the The number of plaque pixels in each of the tooth regions, wherein the tooth position identification module establishes the plaque value table according to the respective plaque values of the tooth positions. The image analysis system according to claim 1, wherein the image analysis system displays the dental plaque value table through a display device. The image analysis system according to claim 1, wherein the plaque recognition module performs image stacking of the plaque distribution image and the ultraviolet image to generate a stacked image, and the image analysis system transmits through a display device Display the stacked image. The image analysis system according to claim 1, wherein the exposure time length for the ultraviolet light image obtained by the image capturing device is longer than the exposure time length for the white light image obtained by the image capturing device. An image analysis method includes: obtaining a white light image and an ultraviolet light image corresponding to a tooth through an image capturing device; Defining a tooth image of a plurality of tooth regions corresponding to a plurality of tooth positions of the tooth includes: performing an image pre-processing on the white light image to generate the tooth image; and performing a tooth image on the tooth image according to a tooth database Position mark, and the image recognizes multiple tooth features in the tooth image to define the tooth regions of the teeth in the tooth image; input the ultraviolet light image to a plaque recognition module to make The dental plaque recognition module outputs a dental plaque distribution image according to the ultraviolet light image; and The plaque distribution image and the tooth image are compared by the tooth position recognition module to analyze the plaque distribution in each of the tooth regions, and create a plaque value table corresponding to the tooth . The image analysis method according to claim 10, wherein the step of obtaining the white light image and the ultraviolet light image corresponding to the tooth comprises: when a white light source illuminates the tooth, the tooth is photographed through the image capturing device to obtain The white light image; and when an ultraviolet light source illuminates the tooth, the tooth is photographed through the imaging device to obtain the ultraviolet image, wherein the imaging device obtains the ultraviolet image from the same sampling position corresponding to the tooth at different times The white light image and the ultraviolet light image. The image analysis method according to claim 11, wherein the brightness value of the ultraviolet light source is greater than the brightness value of the white light source. The image analysis method according to claim 10, wherein the image capturing device is used to obtain multiple sets of white light images and ultraviolet light images from a plurality of different sampling positions corresponding to the tooth. The image analysis method according to claim 10, wherein the ultraviolet light image is input to the dental plaque recognition module, so that the dental plaque recognition module outputs the distribution image of the dental plaque according to the ultraviolet light image The steps include: converting the ultraviolet light image from an RGB color space to an HSV color space; and judging the ultraviolet light image after the color space conversion according to a preset range A plurality of plaque pixels of the plaque to produce an image of the plaque distribution. The image analysis method according to claim 14, wherein the plaque distribution image is a binary image, and the step of establishing the plaque value table corresponding to the tooth includes: comparing the plaque distribution image And the tooth image is used to separately calculate the number of plaque pixels in the tooth regions; and the plaque value table is established according to the respective plaque values of the tooth positions. The image analysis method according to claim 10, further comprising: displaying the dental plaque value table through a display device. The image analysis method according to claim 10, further comprising: image stacking the plaque distribution image and the ultraviolet light image by the plaque recognition module to generate a stacked image; and through a display device Display the stacked image. The image analysis method according to claim 10, wherein the exposure time length for the ultraviolet light image obtained by the image capturing device is longer than the exposure time length for the white light image obtained by the image capturing device.

Images