JP7067743B2 - Oral Cancer Diagnostic System and Oral Cancer Diagnostic Program - Google Patents

Description

The present invention relates to an oral cancer diagnostic system and an oral cancer diagnostic program.

Various diseases have been diagnosed based on the images in the oral cavity. The diagnosis of the disease is performed, for example, by visual inspection and image analysis of the image by a doctor. Intraoral photographs are mainly used for diagnosing diseases. An intraoral photograph consists of a plurality of images showing a predetermined area in the oral cavity. As an example of the intraoral photograph, a predetermined intraoral standard photograph may be used. Each image constituting the intraoral photograph is generated as a so-called panoramic image in which a plurality of images representing a portion of the oral cavity are combined based on a superposed region. Further, there is known a technique for classifying an image of a tooth surface for caries detection based on an intraoral image (see, for example, Patent Document 1).

Japanese Patent Publication No. 2014-509908

Diagnosis of oral cancer, which is one of the diseases in the oral cavity, is made based on the color and shape of the target portion of the diagnosis. In order to make a highly accurate diagnosis, it is necessary that the target site is correctly represented in the intraoral photograph. However, saliva may cause reflexes in images of the oral cavity taken normally. In such an image, the part where the reflection occurs is represented by a color or the like different from the actual state. Misdiagnosis can occur when oral cancer is diagnosed based on images that include areas of reflexes.

Therefore, the present invention has been made in view of the above problems, and an object thereof is to reduce the possibility of misdiagnosis in the diagnosis of oral cancer based on an intraoral photograph.

In order to solve the above problems, the oral cancer diagnostic system according to one embodiment of the present invention is a disease related to oral cancer in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity. An oral cancer diagnosis system for diagnosing oral cancer, which includes an acquisition unit that acquires a plurality of partial images of a part of the oral cavity of a user, and a removal or reflection portion of a reflection portion that is a region where reflection occurs in the partial image. A removal unit that acquires a partial image that does not include a reflective portion by extracting a partial image that does not exist, a generation unit that synthesizes a plurality of partial images including a partial image acquired by the removal unit to generate an intraoral photograph, and an oral cavity. Based on the characteristics of the image of the predetermined determination target area shown in the internal photograph, the determination unit for determining the presence / absence and state of the oral cancer-related disease and the presence / absence and state of the oral cancer-related disease determined by the determination unit. It includes an output unit that outputs determination information.

Further, the oral cancer diagnosis program according to one embodiment of the present invention uses a computer to diagnose a disease related to oral cancer in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity. It is an oral cancer diagnosis program for functioning as an oral cancer diagnosis system, and is an acquisition function for acquiring a plurality of partial images of a part of the oral cavity of a user by a computer, and a region where reflection occurs in the partial image. By removing the reflective part or extracting the partial image that does not include the reflective part, the removal function that acquires the partial image that does not include the reflective part and the multiple partial images that include the partial image acquired by the removal function are combined into the oral cavity. Judgment was made by a generation function for generating a photograph, a determination function for determining the presence or absence and a state of a disease related to oral cancer based on the characteristics of an image of a predetermined determination target area shown in an intraoral photograph, and a determination function. It realizes an output function that outputs determination information regarding the presence / absence and condition of a disease related to oral cancer.

According to the above embodiment, an intraoral photograph is generated based on a partial image that does not include a reflective portion. By diagnosing oral cancer based on the intraoral photograph generated in this way, the possibility of misdiagnosis can be reduced.

In the diagnosis of oral cancer based on intraoral photographs, it is possible to reduce the possibility of misdiagnosis.

It is a figure which shows the apparatus configuration of the oral cancer diagnostic system which concerns on this embodiment. It is a block diagram which shows the functional structure of the oral cancer diagnostic apparatus of this embodiment. It is a hard block diagram of an oral cancer diagnostic apparatus. It is a figure which shows the functional configuration of a user's terminal. It is a figure which shows the functional structure of the terminal of a doctor. It is a figure which shows typically each part in the oral cavity which is the object of diagnosis. It is a figure which shows the example of the panoramic image which was combined based on a partial image and a plurality of partial images. It is a figure which shows the example of the extraction processing of the partial image which does not include the reflection part by the removal part. It is a figure which shows the example of the plurality of partial images including the reflection part, and the part image which removed the reflection part schematically. It is a flowchart which shows the processing content of the removal of the reflection part by the removal part. It is a figure which shows the example of the intraoral photograph generated by the generation part. It is a figure which shows the intraoral photograph which the tooth region was removed. It is a figure which shows the flowchart of the determination process of the presence / absence and the state of oral cancer by the determination unit. It is a figure which shows the example of the information of the result of the determination of the progression level of oral cancer, and is the figure which shows typically the intraoral photograph which marked the possibility of oral cancer. It is a flowchart which shows the example of the processing content of the oral cancer diagnosis method of this embodiment. It is a flowchart which shows the example of the processing content in a user's terminal. It is a flowchart which shows the example of the processing content in the terminal of a doctor. It is a figure which shows the structure of the oral cancer diagnosis program.

An embodiment of the oral cancer diagnostic system according to the present invention will be described with reference to the drawings. If possible, the same parts will be designated by the same reference numerals, and duplicate description will be omitted.

FIG. 1 is a diagram showing an apparatus configuration of an oral cancer diagnostic system according to the present embodiment. The oral cancer diagnosis system 1 is a system for diagnosing a disease related to oral cancer in the user's oral cavity based on an intraoral photograph including a plurality of images showing a predetermined area in the oral cavity. The oral cancer diagnostic system 1 shown in FIG. 1 includes an oral cancer diagnostic device 10, a user's terminal 20, and a doctor's terminal 30, and the oral cancer diagnostic device 10 and the terminals 20, 30 can communicate with each other via the network N. be.

The oral cancer diagnostic apparatus 10 is composed of, for example, a computer such as a server, but the apparatus constituting the oral cancer diagnostic apparatus 10 is not limited. The terminals 20 and 30 are composed of, for example, a stationary or portable personal computer, a high-performance mobile phone (smartphone), etc., but the devices constituting the terminals 20 and 30 are not limited, and for example, the mobile phone and mobile information. It may be a mobile terminal such as a terminal (PDA). In FIG. 1, one terminal 20 and one terminal 30 are shown as an example, but the number thereof is not limited to this.

FIG. 2 is a diagram showing a functional configuration of the oral cancer diagnostic apparatus 10 according to the present embodiment. The oral cancer diagnostic apparatus 10 of the present embodiment is an apparatus for diagnosing a disease related to oral cancer in the user's oral cavity based on an intraoral photograph. The intraoral photograph is a photograph of the oral cavity obtained for diagnosing a disease in the oral cavity. An intraoral photograph comprises, for example, a plurality of images showing a predetermined area in the oral cavity.

As shown in FIG. 2, the oral cancer diagnostic apparatus 10 functionally includes an acquisition unit 11, a pretreatment unit 12, a removal unit 13, a correction unit 14, a generation unit 15, a determination unit 16, and an output unit 17. In the present embodiment, the oral cancer diagnostic device 10 is composed of one device as shown in FIG. 2, but each functional unit included in the oral cancer diagnostic device 10 is dispersed in a plurality of devices. May be done. Each functional part of the oral cancer diagnostic apparatus 10 will be described in detail later.

The block diagram shown in FIG. 2 shows a block of functional units. These functional blocks (components) are realized by any combination of hardware and / or software. Further, the means for realizing each functional block is not particularly limited. That is, each functional block may be realized by one physically and / or logically coupled device, or directly and / or indirectly by two or more physically and / or logically separated devices. (For example, wired and / or wireless) may be connected and realized by these plurality of devices.

Functions include judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, solution, selection, selection, establishment, comparison, assumption, expectation, and assumption. Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc., but limited to these I can't. For example, a functional block (component) that makes transmission function is called a transmitting unit or a transmitter. In each case, as described above, the realization method is not particularly limited.

For example, the oral cancer diagnostic apparatus 10 according to the embodiment of the present invention may function as a computer. FIG. 3 is a diagram showing an example of the hardware configuration of the oral cancer diagnostic apparatus 10 according to the present embodiment. The oral cancer diagnostic device 10 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

In the following description, the word "device" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the oral cancer diagnostic device 10 may be configured to include one or more of the devices shown in FIG. 2, or may be configured not to include some of the devices.

For each function of the oral cancer diagnostic apparatus 10, the processor 1001 performs calculations by loading predetermined software (program) on hardware such as the processor 1001 and the memory 1002, and communication by the communication device 1004, the memory 1002, and the memory 1002. It is realized by controlling the reading and / or writing of data in the storage 1003.

Processor 1001 operates, for example, an operating system to control the entire computer. The processor 1001 may be composed of a central processing unit (CPU) including an interface with a peripheral device, a control device, an arithmetic unit, a register, and the like. For example, each of the functional units 11 to 17 shown in FIG. 1 may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), a software module and data from the storage 1003 and / or the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above-described embodiment is used. For example, each functional unit 11 to 17 of the oral cancer diagnostic apparatus 10 may be realized by a control program stored in the memory 1002 and operated by the processor 1001. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be mounted on one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, and the like that can be executed to implement the shelving allocation information generation method according to the embodiment of the present invention.

The storage 1003 is a computer-readable recording medium, and is, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, an optical magnetic disk (for example, a compact disk, a digital versatile disk, a Blu-ray). It may consist of at least one (registered trademark) disk), smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server or other suitable medium containing memory 1002 and / or storage 1003.

The communication device 1004 is hardware (transmission / reception device) for communicating between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be composed of a single bus or may be composed of different buses between the devices.

Further, the oral cancer diagnostic apparatus 10 includes hardware such as a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). It may be configured to include, and a part or all of each functional block may be realized by the hardware. For example, the processor 1001 may be implemented on at least one of these hardware.

FIG. 4 is a diagram showing a functional configuration of the user terminal 20 of the present embodiment. As shown in FIG. 4, the terminal 20 includes an image acquisition unit 21, an image transmission unit 22, a determination result receiving unit 23, and a determination result presenting unit 24. Further, the terminal 20 includes an image sensor 25 and a display 29.

The image sensor 25 is a sensor that optically acquires an image to be imaged. Further, the terminal 20 may include a depth sensor 26. The depth sensor 26 is a sensor that acquires a depth indicating a distance to an object by infrared rays.

Further, the terminal 20 may include an acceleration sensor 27 and an angular velocity sensor 28. The acceleration sensor 27 is a sensor that detects the acceleration generated in the terminal 20. The angular velocity sensor 28 is a sensor that detects the angular velocity generated in the terminal 20.

The image acquisition unit 21 acquires a partial image captured by the image sensor 25. The partial image is an image obtained by capturing a portion of the oral cavity. The image acquisition unit 21 acquires a plurality of partial images.

The image acquisition unit 21 can acquire the depth indicating the distance to the image pickup target detected by the depth sensor 26 at the time of acquiring the partial image. As a result, the image acquisition unit 21 can acquire a partial image in which the depth is associated with each region consisting of one or more pixels.

Further, the terminal 20 can detect the acceleration at the time of acquiring the partial image by the acceleration sensor 27 and send the detected value in association with the partial image. Further, the terminal 20 can detect the angular velocity at the time of acquiring the partial image by the angular velocity sensor 28 and send the detected value in association with the partial image.

Each of the plurality of partial images may be acquired as a still image, or may be extracted from each frame acquired as a moving image. As a result, the image acquisition unit 21 can acquire a plurality of partial images of the same portion in the oral cavity. Further, the image acquisition unit 21 can acquire a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions. In the present embodiment, the user can take an image of his / her oral cavity by using the terminal 20.

The image transmission unit 22 transmits a plurality of partial images acquired by the image acquisition unit 21 to the oral cancer diagnostic apparatus 10. The image transmission unit 22 can transmit the detection value detected by the sensor at the time of acquiring each partial image in association with the partial image. The detected value can include at least one of the acceleration detected by the acceleration sensor 27 and the angular velocity detected by the angular velocity sensor 28.

The determination result receiving unit 23 receives the determination information from the oral cancer diagnostic apparatus 10. The determination information is information indicating the result of determination by the determination unit 16 of the oral cancer diagnostic apparatus regarding the presence / absence and state of a disease related to oral cancer. The determination of the disease related to oral cancer in the oral cavity is carried out by the determination unit 16 of the oral cancer diagnostic apparatus 10 based on the plurality of partial images transmitted by the image transmission unit 22. The determination result includes, for example, an intraoral photograph in which the location of the disease is marked, a comment on the discovered disease, and the like. Further, the determination result receiving unit 23 may receive the determination information transmitted from the oral cancer diagnostic apparatus 10 to the terminal 30 via the terminal 30.

Further, the determination result receiving unit 23 receives the diagnostic information from the terminal 30. The diagnostic information is information showing the result of diagnosis of a disease related to oral cancer by a doctor, and is input to the terminal 30 by a doctor who referred to the determination information transmitted from the oral cancer diagnostic device 10 to the terminal 30 and the intraoral photograph. Information. In addition, the diagnostic information may include consultation necessity information regarding the necessity of consultation of the user. Further, the determination result receiving unit 23 may receive the diagnostic information transmitted from the terminal 30 to the oral cancer diagnostic device 10 via the oral cancer diagnostic device 10.

The determination result presenting unit 24 displays the diagnostic information received by the determination result receiving unit 23 on the display 29. Further, the determination result presentation unit 24 may display the determination information on the display 29.

FIG. 5 is a diagram showing a functional configuration of the doctor's terminal 30 of the present embodiment. As shown in FIG. 5, the terminal 30 includes a receiving unit 31, a presenting unit 32, a receiving unit 33, and a transmitting unit 34. Further, the terminal 30 includes a display 35.

The receiving unit 31 receives the determination information indicating the result of the determination regarding the disease related to oral cancer from the oral cancer diagnostic apparatus 10. The determination information can include, for example, an intraoral photograph in which the location of the disease is marked, a comment regarding the discovered disease, and the like as the result of the determination.

The presenting unit 32 causes the display 35 to display the determination information received by the receiving unit 31. Thereby, the determination information can be presented to the doctor.

The reception unit 33 receives the diagnostic information input by the doctor to the terminal 30. That is, since the doctor can refer to the determination information and the intraoral photograph presented by the presentation unit 32, the doctor can input the result of the diagnosis based on his / her own knowledge and experience to the terminal 30 based on these references. The reception unit 33 receives the diagnosis result input in this way as diagnostic information. In addition, the doctor can examine the necessity of the user's consultation based on the judgment information and the reference of the intraoral photograph and the like. Then, the doctor can input the diagnosis information including the reception necessity information regarding the necessity of the consultation for encouraging the user to receive the consultation.

The transmission unit 34 transmits the diagnostic information to the user's terminal 20. Thereby, the diagnostic information can be presented on the user's terminal 20. Further, the transmission unit 34 may transmit the diagnostic information to the terminal 20 via the oral cancer diagnostic apparatus 10. In this case, the output unit 17 of the oral cancer diagnostic apparatus 10 transfers the diagnostic information from the transmission unit 34 to the terminal 20, and causes the terminal 20 to output the diagnostic information.

Each functional part of the oral cancer diagnostic apparatus 10 will be described with reference to FIG. 2 again. The acquisition unit 11 acquires a plurality of partial images of the portion in the oral cavity of the user. Specifically, the acquisition unit 11 acquires a partial image captured by the image sensor 25 of the terminal 20.

FIG. 6 is a diagram schematically showing each part in the oral cavity to be diagnosed in the oral cancer diagnosis system 1 of the present embodiment. In the present embodiment, the parts to be diagnosed are the part shown in the so-called intraoral standard photograph and the tongue d1, buccal mucosa d2, hard palate d3, posterior acetabular triangle d4 and gingiva d5 shown in FIG. The buccal mucosa d2 is a tissue covering the inside of the cheek. The hard palate d3 is the ceiling portion of the oral cavity. The posterior acetabular triangle d4 is the posterior region of the wisdom tooth. The acquisition unit 11 acquires a partial image showing these parts to be diagnosed.

FIG. 7 is a diagram showing an example of a panoramic image synthesized based on a partial image and a plurality of partial images. As shown in FIG. 7, the acquisition unit 11 acquires, for example, a partial image A01 on the front right side of the dentition, a partial image A02 in the center of the front surface, and a partial image A03 on the left side of the front surface. Each of these partial images may be acquired as a still image, or may be extracted from each frame acquired as a moving image.

As will be described later, in the present embodiment, the panoramic image B is generated by synthesizing a plurality of partial images A01 to A03 based on the superimposed region in which the same portion in the oral cavity is represented in each of the partial images A01 to A03. Will be done. The panoramic image B shown in FIG. 7 constitutes a frontal image in an intraoral photograph. Various well-known techniques can be applied to the composition of panoramic images based on the superimposed region of partial images.

The preprocessing unit 12 excludes the partial image corresponding to the predetermined condition from the partial image for forming the intraoral photograph. Specifically, when the acquisition unit 11 acquires a partial image to which the detection value of at least one of the angular velocity sensor 28 and the acceleration sensor 27 at the time of imaging is associated, the preprocessing unit 12 has a predetermined value or more. The partial image associated with the detection value of is not used for the generation of the intraoral photograph.

If a large angular velocity or acceleration occurs when the partial image is acquired, it is highly possible that the partial image is blurred. The preprocessing unit 12 excludes the partial image associated with the detection value of the predetermined value or more from the partial image for forming the intraoral photograph, so that the partial image in which the blur occurs is generated as the intraoral photograph. It can be prevented from being used.

The removal unit 13 acquires a partial image that does not include the reflection portion by removing the reflection portion that is a region where reflection occurs in the partial image or extracting the partial image that does not include the reflection portion. Images taken in the oral cavity may cause reflexes due to saliva present in the oral cavity. In order to make a highly accurate diagnosis of oral cancer, it is necessary to correctly show the condition of the oral cavity in the intraoral photograph, but the color etc. of the image of the part where the reflection occurred is different from the actual condition. Is represented. In the present embodiment, high-precision diagnosis is realized by acquiring a partial image that does not include the reflection portion.

Specifically, the removal unit 13 can extract a partial image that does not include the reflection portion from the partial image acquired by the acquisition unit 11. FIG. 8 is a diagram showing an example of extraction processing of a partial image not including a reflection portion by the removal unit 13. As shown in FIG. 8, the set as0 of the partial images is a plurality of partial images acquired by the acquisition unit 11, and includes the partial images a1, a2, a3, and so on.

For example, the removing unit 13 uses a well-known image analysis technique to extract a partial image having a unique image feature of the reflective portion (for example, having a brightness of a predetermined value or more). Specifically, first, the removal unit 13 performs grayscale conversion on the partial images a1, a2, a3 .... Subsequently, the removing unit 13 acquires the luminance value for each pixel of each of the partial images a1, a2, a3 .... Then, the removing unit 13 extracts a partial image having pixels having a luminance value equal to or higher than a predetermined threshold value as a partial image including the reflection portion.

In the example shown in FIG. 8, the removing unit 13 extracts the partial image a2 including the portion rf0 having a luminance value equal to or higher than a predetermined threshold value as the partial image including the reflection portion among the partial images included in the set as0. Then, the removing unit 13 extracts a set as1 of the partial images including the partial images a1, a3 ... Other than the partial image a2 as a partial image for use in generating an intraoral photograph. In addition, the removing unit 13 can extract a partial image that does not include the reflective portion by various other well-known techniques.

Further, the removing unit 13 can acquire a partial image that does not include the reflecting portion by removing the reflecting portion that is a region where the reflection occurs in the partial image. More specifically, the removing unit 13 converts a plurality of partial images so that the projection direction of the portion represented in the partial image is aligned with a predetermined direction, and the pixels representing the same portion in the oral cavity are associated with each other. A partial image in which the reflection portion is removed is acquired by performing the alignment so as to be performed and adding and averaging a plurality of partial images.

The process of removing the reflective portion by the removing unit 13 will be described in detail with reference to FIGS. 9 and 10. FIG. 9 is a diagram schematically showing an example of a plurality of partial images including a reflective portion and a partial image in which the reflective portion is removed. FIG. 10 is a flowchart showing the processing contents of the removal of the reflection portion by the removal unit 13.

In step S21, the removing unit 13 acquires one partial image. In the example shown in FIG. 9, the removing unit 13 acquires the partial image A11. The partial image A11 includes a reflection portion rf1.

Subsequently, in step S22, the removing unit 13 converts a plurality of partial images so that the projection direction of the portion represented by the partial image is aligned with a predetermined direction. Specifically, the removing unit 13 performs homography conversion on the partial image so as to meet the standard of each image constituting the intraoral photograph, for example. Homography conversion is a well-known image conversion technique, in which a two-dimensional image represented by projecting a three-dimensional object onto one plane is projected onto another plane using a projective transformation. Is a technology to generate. The removal unit 13 performs homography conversion on the partial image A11 so as to meet the standard of the front image in the intraoral photograph, and obtains the converted partial image A21.

In step S23, the removal unit 13 determines whether or not the processing of all the partial images has been completed. If it is determined that the processing of all the partial images is completed, the processing proceeds to step S24. On the other hand, if it is not determined that the processing of all the partial images is completed, the processing proceeds to step S21.

In the example described with reference to FIG. 9, since the processing of the partial images A12 and A13 is not completed, the processing of steps S31 and S32 for these partial images A12 and A13 is repeated. That is, the removal unit 13 performs homography transformation on the partial image A12 including the reflection portion rf2 to obtain the partial image A22. Further, the removal unit 13 performs homography transformation on the partial image A13 including the reflection portion rf3 to obtain the partial image A23.

In step S24, the removing unit 13 aligns the pixels representing the same portion in the oral cavity so as to be associated with each other, and adds and averages a plurality of partial images. In the example described with reference to FIG. 9, the removing unit 13 aligns the converted partial images A21 to A23 so that pixels representing the same portion in the oral cavity are associated with each other, and adds and averages them. Synthesize. The additive average composition is a well-known technique in the field of image processing.

Each of the partial images A11 to A13 is an image obtained by capturing the same portion in the oral cavity from different directions. Therefore, in the partial image A22 and the partial image A23, no reflection occurs at the position of the reflection portion rf1 in the partial image A21. Further, in the partial image A21 and the partial image A23, no reflection occurs at the position of the reflection portion rf2 in the partial image A22. Further, in the partial image A21 and the partial image A22, no reflection occurs at the position of the reflection portion rf3 in the partial image A23.

Even if reflection occurs in a certain part of one partial image, the reflection is removed by adding and averaging a plurality of other partial images in which the reflection does not occur in the part and the one partial image. You can get an image. In the example described with reference to FIG. 9, the removal unit 13 can acquire the partial image C1 from which the reflection portion has been removed by aligning the partial images A21 to A23 and performing additive averaging synthesis.

With reference to FIG. 2 again, the correction unit 14 corrects the brightness of each region based on the depth associated with each region of the partial image. In the imaging of the oral cavity, the deeper the depth, the smaller the amount of light, so the deeper the depth, the smaller the luminance value tends to be. The correction unit 14 corrects the luminance value in order to correct this difference in the amount of light.

Specifically, the correction unit 14 acquires the depth of each region, and based on the correspondence between the preset depth and the luminance correction value, the luminance correction value associated with the acquired depth is used as the luminance of the region. By adding to the value, the brightness of each area is corrected. An example of this correction will be described below.

For example, the depth of a certain area is D, the RGB value of the pixel before correction is (r ₀ , g ₀ , b ₀ ), and the pixel correction value is (r ₁ , g ₁ , b ₁ ), (r ₂ , g ₂ , As b ₂ ), it is assumed that the corresponding relationship between the depth D and the pixel correction value is set in advance as follows. In this example, the correction for increasing the luminance value is realized by reducing the value of each color of the RGB values of the pixels constituting each region. That is, the pixel correction value corresponds to the luminance correction value.

Correspondence between depth D and pixel correction value:
When the depth D is less than n, the RGB value of the pixel is not corrected.
When the depth D is n or more and less than m, the RGB value of the pixel is corrected (r ₀ -r ₁ , g ₀ -g ₁ , b ₀ -b ₁ ).
When the depth D is m or more, the RGB value of the pixel is corrected (r ₀ -r ₂ , g ₀ -g ₂ , b ₀ -b ₂ ).

The correction unit 14 corrects the RGB values of the pixels in each region according to the depth of each region with reference to the above correspondence. In the above correspondence, each value of n, m and a pixel correction value may be set by design. In this way, the brightness of the pixels in each region is appropriately corrected according to the depth. This makes it possible to obtain an intraoral photograph in which the state of the oral cavity is uniformly and easily recognized.

Further, the correction unit 14 determines the brightness of the first region when the first region of each region of the plurality of partial images has a depth deeper than the depth of the second region different from the first region. May be corrected to a value larger than that before the correction.

As described above, the amount of light in the deep region of the partial image representing the portion in the oral cavity tends to be small, and the image in that region tends to be dark. Therefore, in the oral cancer diagnostic apparatus 10 of the present embodiment, the correction unit 14 corrects the luminance value in the region deeper than the other regions to a larger value.

The generation unit 15 generates an intraoral photograph by synthesizing a plurality of partial images including the partial image acquired by the removal unit 13. The partial image used for generating the intraoral photograph may have undergone pretreatment and correction processing by the pretreatment unit 12 and the correction unit 14.

Specifically, as shown in FIG. 7, the generation unit 15 generates a panoramic image by synthesizing a partial image based on a superposed region in which the same part in the oral cavity is represented, thereby producing a panoramic image of the oral cavity photograph. Generate multiple images according to the standard. A well-known method for generating a so-called panoramic image can be applied to the generation of an intraoral photograph by synthesizing a partial image.

Specifically, the generation unit 15 adjusts the orientation and arrangement of the acquired plurality of partial images. Subsequently, the generation unit 15 generates one panoramic image by superimposing and synthesizing partial images adjacent to each other based on the superposed region in which the same portion in the oral cavity is represented. One panoramic image constitutes one of a plurality of images constituting an intraoral photograph.

When generating an intraoral photograph based on a partial image in which teeth are represented, the generation unit 15 superimposes each partial image based on a superposed region in which teeth are represented to generate an intraoral photograph. In addition, when generating an intraoral photograph showing the tongue, buccal mucosa, hard palate, and posterior uvula, the generation unit 15 uses each partial image as a superposed region with regions showing teeth, lips, uvula, and the like. Can be superimposed to generate an intraoral photograph.

FIG. 11 is a diagram showing an example of an intraoral photograph generated by the generation unit 15. The intraoral photograph MP consists of a plurality of images showing a predetermined area in the oral cavity. The example shown in FIG. 11 is an intraoral standard photograph by a so-called five-photographing method. The method of taking an intraoral standard photograph includes a 5-image photography method, a 9-image photography method, a 12-image photography method, a 14-image photography method, and the like, depending on the number of constituent images and the imaged portion of each image. As shown in FIG. 11, the generation unit 15 can generate, for example, an intraoral photograph MP taken by a five-photographing method. The intraoral photograph MP includes a palatal side image mp1, a right side image mp2, a frontal image mp3, a left side image mp4, and a lingual side image mp5.

In addition, the generation unit 15 removes the tooth region, which is the region where the teeth are represented, from the intraoral photograph by a predetermined image processing technique. The oral cancer diagnostic system of the present embodiment is a system for diagnosing oral cancer. Therefore, the tooth region is not a diagnostic target region. By removing the tooth region, the generation unit 15 can generate an intraoral photograph showing the region to be diagnosed.

Specifically, the generation unit 15 acquires an intraoral photograph generated by superimposing a partial image. Subsequently, the generation unit 15 performs a binarization process on the intraoral photograph to obtain a binarized image. In the binarized image, the tooth region is represented by, for example, white, and the region other than the tooth region is represented by black. Then, the generation unit 15 performs blob analysis on the binarized image, recognizes the tooth region and the region other than the tooth region, and removes the tooth region. The method for recognizing and removing the tooth region from the intraoral photograph is not limited to these methods, and various well-known image processing techniques can be applied.

FIG. 12 is a diagram showing an intraoral photograph MP1 from which a tooth region has been removed. The intraoral photograph MP1 is the one in which the tooth region is removed from the intraoral photograph MP shown in FIG. As shown in FIG. 12, the intraoral photograph MP1 includes a palatal side image mp11, a right side image mp12, a frontal image mp13, a left side image mp14 and a lingual side image mp15 from which the tooth region has been removed.

The palate-side image mp11 is an image in which the tooth region t11 is removed from the palate-side image mp1 and includes a determination target region r11. The determination target region r11 includes, for example, a region in which the gingiva is represented. The right image mp12, the front image mp13, the left image mp14, and the lingual image mp15 have tooth regions t12, t13, t14, and t15 from the right image mp2, the front image mp3, the left image mp4, and the lingual image mp5, respectively. The removed image is composed of each of the determination target areas r12, r13, r14, and r15.

The determination unit 16 determines the presence / absence and state of a disease related to oral cancer based on the characteristics of the image of the predetermined determination target region shown in the intraoral photograph. Specifically, the determination unit 16 makes a determination based on an intraoral photograph in which the tooth region is removed. That is, the determination unit 16 determines the presence / absence and state of oral cancer by using a region other than the tooth region among the regions shown in the intraoral photograph as a determination target region. In the place where oral cancer has occurred, a change in color such as vitiligo and a change in shape such as convexity may occur. Therefore, the determination unit 16 describes the characteristics of the image showing these changes in the oral cavity. Detect from the inside photo.

The determination of a disease related to oral cancer will be described with reference to FIGS. 13 and 14. FIG. 13 is a diagram showing a flowchart of the process of determining the presence / absence and state of oral cancer by the determination unit 16.

In step S31, the determination unit 16 acquires an intraoral photograph generated by the generation unit 15. As described above, the determination unit 16 can acquire an intraoral photograph from which the tooth region has been removed.

In step S32, the determination unit 16 inputs the intraoral photograph acquired in step S31 as input data to the oral cancer determination model. The oral cancer determination model is a model for determining the presence / absence and progression level of oral cancer by using image data of an intraoral photograph as an input value, and is constructed in advance by machine learning.

The oral cancer determination model is constructed as a classifier such as a so-called support vector machine (SVM), for example, using an intraoral photograph masking a cancerous part as learning data. The judgment results output from the oral cancer judgment model include, for example, information on classification of the presence or absence of oral cancer and the degree of progression such as soundness, progression level T1, T2, T3, T4 (low: T1 → high: T4) and classification. It is output as information indicating the location of oral cancer.

Further, the oral cancer determination model may be constructed by a neural network. An oral cancer judgment model including a neural network is generated by performing machine learning including error calculation (error back propagation) based on the judgment value of the progression level of oral cancer obtained by inputting an intraoral photograph for learning. To. Various loss functions can be applied to the error calculation.

Further, the determination unit 16 may determine the region where oral cancer is occurring and the progression level based on the threshold value set for each site with respect to the pixel value of the intraoral photograph regardless of the model.

In step S33, the determination unit 16 outputs the classification result by the model, that is, the information indicating the presence / absence of oral cancer and the progress level, which was output in step S32. FIG. 14 is a diagram showing an example of information on the result of determination of the progression level of oral cancer, and is a diagram schematically showing an intraoral photograph in which a portion having a possibility of oral cancer is marked. As shown in FIG. 14, the intraoral photograph mp20 output by the determination unit 16 includes the mark D at the site of oral cancer. Even if the information on the progression level of the site is associated with the mark D of the oral cancer site and is output by a text such as "The marked site is suspected to have the progression level T0 of oral cancer". good.

With reference to FIG. 2 again, the output unit 17 outputs the determination information regarding the presence / absence and the state of the disease related to oral cancer determined by the determination unit 16. Specifically, the output unit 17 outputs an intraoral photograph and a text in which the portion of oral cancer is marked, which is output as determination information, on the terminal 30 of the doctor. More specifically, the output unit 17 causes the display 35 of the doctor's terminal 30 to display the determination information. Further, the output unit 17 may output the determination information to the user's terminal 20. Further, the output unit 17 may store the determination information in a predetermined storage means.

Next, with reference to FIG. 15, the oral cancer diagnosis method in the oral cancer diagnosis system 1 and the oral cancer diagnosis device 10 will be described. FIG. 15 is a flowchart showing an example of the processing content of the oral cancer diagnosis method of the present embodiment.

In step S1, the acquisition unit 11 acquires a plurality of partial images of the portion in the oral cavity of the user. Next, in step S2, the preprocessing unit 12 excludes the partial image having a possibility of blurring from the partial image for generating the intraoral photograph based on the predetermined detection value in the partial image. The detected values used for the determination here are, for example, the acceleration and the angular velocity generated in the image pickup device acquired at the time of capturing the partial image.

In step S3, the removal unit 13 acquires a partial image that does not include the reflection portion by removing the reflection portion that is a region where reflection occurs in the partial image or extracting the partial image that does not include the reflection portion. The processing for removing the reflection portion is as described, for example, with reference to FIG.

In step S4, the correction unit 14 corrects the brightness of each region based on the depth associated with each region of the partial image.

In step S5, the generation unit 15 synthesizes a plurality of partial images to generate an intraoral photograph. Subsequently, in step S6, the generation unit 15 removes the tooth region from the intraoral photograph generated in step S5 by a predetermined image processing technique.

In step S7, the determination unit 16 determines the presence / absence and state of a disease related to oral cancer based on the characteristics of the image of the predetermined determination target region shown in the intraoral photograph. Specifically, the determination unit 16 makes a determination based on the intraoral photograph from which the tooth region is removed in step S6. The determination process is, for example, as described with reference to FIG.

In step S8, the determination unit 16 determines the presence or absence of a disease related to oral cancer in the determination result of the determination process. If it is determined that there is a disease related to oral cancer, the process proceeds to step S9. On the other hand, if it is not determined that there is a disease related to oral cancer, the process proceeds to step S10.

In step S9, the determination unit 16 generates an intraoral photograph marked at a location of a disease related to oral cancer as determination information. The determination unit 16 may also generate and output text information indicating the progress level of the relevant portion.

In step S10, the output unit 17 outputs the determination information regarding the presence / absence and the state of the disease related to oral cancer determined by the determination unit 16. The output unit 17 may output the determination information on the doctor's terminal 30 or the user's terminal 20.

Next, with reference to FIG. 16, the processing in the terminal 30 of the doctor of the oral cancer diagnostic system 1 will be described. FIG. 16 is a flowchart showing an example of processing contents in the terminal 30 of a doctor.

In step S41, the receiving unit 31 receives the determination information indicating the result of the determination regarding the disease related to oral cancer in the oral cavity from the oral cancer diagnostic apparatus 10. The determination information may include an intraoral photograph in which the location of oral cancer is marked, a comment on the discovered disease, and the like.

In step S42, the presenting unit 32 presents the determination information received by the receiving unit 31 in step S41 to the doctor. The presentation unit 32 displays, for example, the determination information on the display 29.

In step S43, the reception unit 33 receives the diagnostic information input by the doctor in response to the presentation of the determination information. The diagnostic information may include consultation necessity information regarding the necessity of consultation of the user.

In step S44, the transmission unit 34 transmits the diagnostic information received in step S43 to the user's terminal 20. Further, the transmission unit 34 may transmit the diagnostic information to the terminal 20 via the oral cancer diagnostic apparatus 10.

Next, with reference to FIG. 17, the processing in the terminal 20 of the user of the oral cancer diagnostic system 1 will be described. FIG. 17 is a flowchart showing an example of processing contents in the user's terminal 20.

In step S51, the image sensor 25 captures a plurality of partial images. Then, the image acquisition unit 21 acquires a plurality of partial images captured by the image sensor 25.

In step S52, the image transmission unit 22 transmits a plurality of partial images acquired by the image acquisition unit 21 in step S51 to the oral cancer diagnostic apparatus 10.

In step S53, the determination result receiving unit 23 receives the diagnostic information that is the result of the determination of the disease related to oral cancer from the terminal 30 of the doctor. The determination result receiving unit 23 may receive the diagnostic information transmitted via the oral cancer diagnostic apparatus 10.

In step S54, the determination result presenting unit 24 presents the diagnostic information received by the determination result receiving unit 23 in step S53 to the user. The determination result presentation unit 24 displays, for example, diagnostic information on the display 29.

Next, an oral cancer diagnostic program for causing the computer to function as the oral cancer diagnostic apparatus 10 of the present embodiment will be described. FIG. 18 is a diagram showing the configuration of the oral cancer diagnostic program P1.

The oral cancer diagnosis program P1 includes a main module m10, an acquisition module m11, a pretreatment module m12, a removal module m13, a correction module m14, a generation module m15, and a determination module that comprehensively control the oral disease diagnosis process in the oral cancer diagnosis device 10. It is configured to include m16 and an output module m17. Then, each module m11 to m17 realizes each function for the acquisition unit 11, the pretreatment unit 12, the removal unit 13, the correction unit 14, the generation unit 15, the determination unit 16, and the output unit 17 in the oral cancer diagnostic apparatus 10. Will be done. The oral cancer diagnosis program P1 may be transmitted via a transmission medium such as a communication line, or may be stored in the recording medium M1 as shown in FIG. ..

In the oral cancer diagnosis system 1, the oral cancer diagnosis device 10, the oral cancer diagnosis method, and the oral cancer diagnosis program P1 of the present embodiment described above, an intraoral photograph is generated based on a partial image that does not include a reflex portion. By diagnosing oral cancer based on the intraoral photograph generated in this way, the possibility of misdiagnosis can be reduced.

Further, in the oral cancer diagnosis system according to another embodiment, the removal unit determines whether or not the partial image includes the reflection portion based on the pixel value of each pixel included in the partial image, and is acquired by the acquisition unit. A partial image that does not include a reflective portion may be extracted from the plurality of partial images.

According to the above embodiment, only a partial image that does not include a reflective portion can be used for generating an intraoral photograph.

Further, in the oral cancer diagnosis system according to another embodiment, the removing portion may remove the reflected portion in the partial image by a predetermined image processing technique.

According to the above embodiment, it is possible to obtain a partial image that does not include the reflection portion based on the acquired partial image.

Further, in the oral cancer diagnosis system according to another form, the acquisition unit acquires a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions, and the removal unit is the portion represented by the partial image. By converting multiple partial images so that the projection direction is aligned with a predetermined direction, aligning the pixels representing the same part in the oral cavity so that they are associated with each other, and averaging the multiple partial images. It may be possible to acquire a partial image from which the reflective portion has been removed.

According to the above embodiment, a plurality of partial images obtained by capturing the same portion from different directions are acquired. In the plurality of partial images acquired in this way, even if reflection occurs at a specific portion in one partial image, there is a high possibility that the reflection portion and the reflection state are different in the other partial images. Since a plurality of partial images converted so that the projection direction of a certain part is aligned in a predetermined direction are aligned and then added and averaged, it is possible to obtain a partial image in which the reflection part is removed. Become.

Further, in the oral cancer diagnosis system according to another embodiment, the generation unit removes the tooth region, which is the region where the tooth is represented, from the intraoral photograph by a predetermined image processing technique, and the determination unit uses the generation unit to remove the tooth. The presence or absence and state of a disease related to oral cancer may be determined based on the characteristics of the image of the predetermined determination target region shown in the intraoral photograph in which the region is removed.

According to the above embodiment, since the determination regarding oral cancer is made based on the intraoral photograph in which the region of the tooth that is not the target of diagnosis of oral cancer is removed, the diagnosis regarding oral cancer is appropriately performed.

Further, in the oral cancer diagnosis system according to another embodiment, the output unit outputs the determination information to the first terminal of a doctor different from the user, and inputs the determination information to the first terminal according to the output of the determination information. The diagnostic information regarding the diagnosis of the disease may be output to the user's second terminal.

According to the above embodiment, since the result of the determination regarding oral cancer is output on the terminal of the doctor, the doctor can be easily notified of the determination result regarding the oral cancer of the user. Further, since the diagnostic information input by the doctor according to the notified determination result can be notified to the user, it is possible to make the user appropriately recognize the diagnostic result.

Further, in the oral cancer diagnosis system according to another embodiment, the output unit may output the determination information at the second terminal.

According to the above embodiment, since the result of the determination regarding oral cancer is output on the terminal of the user, the user can be easily notified of the result of the determination regarding oral cancer of the user himself / herself.

Further, in the oral cancer diagnostic system according to another embodiment, the acquisition unit is associated with the detection value of at least one of the angular velocity and the acceleration detected by the device that captured the partial image at the time of capturing the partial image. The oral cancer diagnostic system further includes a preprocessing unit, which excludes a partial image associated with a detection value equal to or higher than a predetermined value from the partial image used for generating an intraoral photograph by the generation unit. It may be that.

According to the above embodiment, the partial image associated with the detection value equal to or higher than the predetermined value of the angular velocity sensor or the acceleration sensor is excluded from the image used for generating the intraoral photograph. As a result, the partially image with blurring is not used for compositing the intraoral photograph. Therefore, it is possible to obtain an intraoral photograph that makes it easy to recognize the state of the oral cavity.

Further, in the oral cancer diagnosis system according to another form, the acquisition unit acquires a partial image in which the depth indicating the distance to the imaging target is associated with each region consisting of one or more pixels, and the oral cancer diagnosis system , A correction unit that corrects the brightness of each region of the partial image based on the depth, may be further included.

According to the above embodiment, based on the finding that the deeper the depth of the image taken in the oral cavity, the more difficult it is for the light from the light source to hit, the oral cavity is synthesized by synthesizing a partial image in which the brightness of each region is corrected based on the depth. Since the internal photograph is generated, it is possible to make a diagnosis based on the intraoral photograph that can uniformly grasp the state of the oral cavity.

Although the present embodiment has been described in detail above, it is clear to those skilled in the art that the present embodiment is not limited to the embodiments described in the present specification. This embodiment can be implemented as an amendment or modification without departing from the spirit and scope of the present invention as determined by the description of the scope of claims. Therefore, the description of the present specification is for the purpose of illustration and does not have any limiting meaning to the present embodiment.

Each aspect / embodiment described in the present disclosure includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA ( Registered Trademarks), GSM (Registered Trademarks), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth It may be applied to (Registered Trademarks), other systems that utilize appropriate systems and / or next-generation systems that are extended based on them.

The order of the processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in the present disclosure may be changed as long as there is no contradiction. For example, the methods described herein present elements of various steps in an exemplary order and are not limited to the particular order presented.

Information and the like can be output from the upper layer (or lower layer) to the lower layer (or upper layer). Input / output may be performed via a plurality of network nodes.

The input / output information and the like may be stored in a specific place (for example, a memory) or may be managed by a management table. Information to be input / output may be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (Boolean: true or false), or by comparing numerical values (for example, a predetermined value). It may be done by comparison with the value).

Each aspect / embodiment described in the present disclosure may be used alone, in combination, or may be switched and used according to the execution. Further, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit one, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.

Software, whether called software, firmware, middleware, microcode, hardware description language, or other names, instructions, instruction sets, codes, code segments, program codes, programs, subprograms, software modules. , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, features, etc. should be broadly interpreted.

Further, software, instructions, and the like may be transmitted and received via a transmission medium. For example, the software may use wired technology such as coaxial cable, fiber optic cable, twist pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave to website, server, or other. When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

The information, signals, etc. described in the present disclosure may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.

The terms described in the present disclosure and / or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

The terms "system" and "network" used in this disclosure are used interchangeably.

Further, the information, parameters, and the like described in the present disclosure may be represented by absolute values, relative values from predetermined values, or other corresponding information.

The terms "determining" and "determining" used in this disclosure may include a wide variety of actions. "Judgment" and "decision" are, for example, judgment, calculation, computing, processing, deriving, investigating, looking up, search, inquiry. It may include (eg, searching in a table, database or another data structure), ascertaining as "judgment" or "decision". Also, "judgment" and "decision" are receiving (for example, receiving information), transmitting (for example, transmitting information), input (input), output (output), and access. It may include (for example, accessing data in memory) to be regarded as "judgment" or "decision". In addition, "judgment" and "decision" are considered to be "judgment" and "decision" when the things such as solving, selecting, choosing, establishing, and comparing are regarded as "judgment" and "decision". Can include. That is, "judgment" and "decision" may include considering some action as "judgment" and "decision". Further, "judgment (decision)" may be read as "assuming", "expecting", "considering" and the like.

The terms "connected", "coupled", or any variation thereof, mean any direct or indirect connection or connection between two or more elements and each other. It can include the presence of one or more intermediate elements between two "connected" or "combined" elements. The connection or connection between the elements may be physical, logical, or a combination thereof. For example, "connection" may be read as "access". As used in the present disclosure, the two elements use at least one of one or more wires, cables and printed electrical connections, and as some non-limiting and non-comprehensive examples, the radio frequency region. Can be considered to be "connected" or "coupled" to each other using electromagnetic energy having wavelengths in the microwave and light (both visible and invisible) regions.

The phrase "based on" as used in this disclosure does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

When the terms "first", "second", etc. are used in the present disclosure, any reference to the elements does not generally limit the quantity or order of those elements. These designations can be used herein as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements do not mean that only two elements can be adopted there, or that the first element must somehow precede the second element.

As long as "include", "including", and variations thereof are used herein or within the scope of the claims, these terms are similar to the term "comprising". In addition, it is intended to be inclusive. Moreover, the term "or" as used herein or in the claims is intended to be non-exclusive.

In the present disclosure, if articles are added by translation, for example a, an and the in English, the disclosure may include the plural nouns following these articles.

In the present disclosure, a plurality of devices are also included unless the device has only one apparently in context or technically.

The entire disclosure is intended to include the plural, unless the context clearly indicates the singular.

1 ... Oral cancer diagnostic system, 10 ... Oral cancer diagnostic device, 11 ... Acquisition unit, 12 ... Pretreatment unit, 13 ... Removal unit, 14 ... Correction unit, 15 ... Generation unit, 16 ... Judgment unit, 17 ... Output unit, 20 ... Terminal, 21 ... Image acquisition unit, 22 ... Image transmission unit, 23 ... Judgment result reception unit, 24 ... Judgment result presentation unit, 25 ... Image sensor, 26 ... Depth sensor, 27 ... Acceleration sensor, 28 ... Angle speed sensor, 29 ... Display, 30 ... Terminal, 31 ... Receiving unit, 32 ... Presenting unit, 33 ... Reception unit, 34 ... Transmitting unit, 35 ... Display, M1 ... Recording medium, m10 ... Main module, m11 ... Acquisition module, m12 ... Previous Processing module, m13 ... removal module, m14 ... correction module, m15 ... generation module, m16 ... judgment module, m17 ... output module, N ... network, P1 ... oral cancer diagnosis program.

Claims (8)

An oral cancer diagnostic system for diagnosing diseases related to oral cancer in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity.
An acquisition unit that acquires a plurality of partial images of the portion in the oral cavity of the user, and
A removal unit that acquires a partial image that does not include the reflection portion by removing a reflection portion that is a region where reflection occurs in the partial image or extracting a partial image that does not include the reflection portion.
A generation unit that generates the intraoral photograph by synthesizing a plurality of partial images including the partial image acquired by the removal unit.
A determination unit for determining the presence or absence and a state of a disease related to oral cancer based on the characteristics of an image of a predetermined determination target area shown in the intraoral photograph.
It is provided with an output unit that outputs determination information regarding the presence / absence and state of a disease related to oral cancer determined by the determination unit .
The acquisition unit acquires a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions.
The removal part
The reflection portion in the partial image is removed by a predetermined image processing technique, and the reflection portion is removed.
A plurality of the partial images are converted so that the projection direction of the portion represented in the partial image is aligned with a predetermined direction, and the pixels representing the same portion in the oral cavity are aligned so as to be associated with each other. By adding and averaging the plurality of partial images, a partial image from which the reflected portion is removed is acquired.
Oral cancer diagnostic system. The removing unit determines whether or not the partial image includes the reflection portion based on the pixel value of each pixel included in the partial image, and the reflection from the plurality of partial images acquired by the acquisition unit. Extract a partial image that does not contain a part,
The oral cancer diagnostic system according to claim 1. The generation unit removes the tooth region, which is the region where the teeth are represented, from the intraoral photograph by a predetermined image processing technique.
The determination unit determines the presence or absence and a state of a disease related to oral cancer based on the characteristics of the image of the predetermined determination target area shown in the intraoral photograph in which the tooth region is removed by the generation unit.
The oral cancer diagnostic system according to claim 1 or 2 . The output unit outputs the determination information on a first terminal of a doctor different from the user.
The diagnostic information regarding the diagnosis of the disease, which is input to the first terminal in response to the output of the determination information, is output to the second terminal of the user.
The oral cancer diagnostic system according to any one of claims 1 to 3 . The output unit outputs the determination information at the second terminal.
The oral cancer diagnostic system according to claim 4 . The acquisition unit acquires the partial image to which the detection value of at least one of the angular velocity and the acceleration detected in the apparatus that captured the partial image at the time of capturing the partial image is associated.
The oral cancer diagnostic system is
A pretreatment unit for excluding the partial image associated with the detection value of a predetermined value or more from the partial image used for generating the intraoral photograph by the generation unit is further included.
The oral cancer diagnostic system according to any one of claims 1 to 5 . The acquisition unit acquires the partial image in which the depth indicating the distance to the image pickup target is associated with each region consisting of one or more pixels.
The oral cancer diagnostic system is
Further includes a correction unit, which corrects the brightness of each region of the partial image based on the depth.
The oral cancer diagnostic system according to any one of claims 1 to 6 . It is an oral cancer diagnosis program for making a computer function as an oral cancer diagnostic system for diagnosing diseases related to oral cancer in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity. hand,
To the computer
The acquisition function to acquire a plurality of partial images of the part in the oral cavity of the user, and
A removal function for acquiring a partial image that does not include the reflective portion by removing a reflective portion that is a region where reflection occurs in the partial image or extracting a partial image that does not include the reflective portion.
A generation function for generating the intraoral photograph by synthesizing a plurality of partial images including the partial image acquired by the removal function, and
A determination function for determining the presence or absence and a state of a disease related to oral cancer based on the characteristics of an image of a predetermined determination target area shown in the intraoral photograph.
The output function that outputs the judgment information regarding the presence / absence and the state of the disease related to oral cancer, which is judged by the judgment function, is realized .
The acquisition function acquires a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions.
The removal function
The reflection portion in the partial image is removed by a predetermined image processing technique, and the reflection portion is removed.
A plurality of the partial images are converted so that the projection direction of the portion represented in the partial image is aligned with a predetermined direction, and the pixels representing the same portion in the oral cavity are aligned so as to be associated with each other. By adding and averaging the plurality of partial images, a partial image from which the reflected portion is removed is acquired.
Oral cancer diagnostic program.

Images