JP2020054444A - Oral cavity cancer diagnostic system and oral cavity cancer diagnostic program - Google Patents

Abstract

To reduce possibility of wrong diagnosis in a diagnosis of oral cavity cancer based on an oral cavity photograph.SOLUTION: An oral cavity cancer diagnostic device 10 is a device for diagnosing a disease on an oral cavity cancer in an oral cavity of a user on the basis of an oral cavity photograph consisting of a plurality of images indicating a predetermined region in the oral cavity, and includes: an acquisition unit 11 for acquiring a plurality of partial images that image a part in the oral cavity of a user; a removal unit 13 for acquiring a partial image that does not include reflection parts by removal of reflection parts in the partial image or extraction of the partial image that does not include the reflection parts; a creation unit 15 for compositing the plurality of partial images including the partial images acquired by the removal unit 13 and creating an oral cavity photograph; a determination unit 16 for determining presence/absence and status of a disease on an oral cavity cancer on the basis of feature of images of a predetermined determination object region represented in the oral cavity photograph; and an output unit 17 for outputting determination information on presence/absence and status of the disease on the oral cavity cancer.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an oral cancer diagnostic system and an oral cancer diagnostic program.

  Various diseases have been diagnosed based on images in the oral cavity. The diagnosis of the disease is performed, for example, by visual observation of the image by a doctor and image analysis. Intraoral photography is mainly used for diagnosis of the disease. The intraoral photograph is composed of a plurality of images representing a predetermined region in the oral cavity. As an example of the intraoral photograph, a predetermined intraoral standard photograph may be used. Each image constituting the intra-oral photograph is generated as a so-called panoramic image in which a plurality of images representing portions in the oral cavity are synthesized based on a superimposed region. Further, there is known a technique of classifying an image of a tooth surface for caries detection based on an intraoral image (for example, see Patent Document 1).

JP, 2014-509908, A

  Diagnosis of oral cancer, which is one of the diseases in the oral cavity, is performed based on the color and shape of the diagnosis target portion. In order to perform highly accurate diagnosis, it is necessary that the target site is correctly represented in the intraoral photograph. However, in a normally taken image of the oral cavity, reflection may occur due to saliva. In such an image, the location where the reflection has occurred is represented in a different color or the like from the actual state. If a diagnosis of oral cancer is made based on an image including a portion where reflection occurs, an erroneous diagnosis may occur.

  Therefore, the present invention has been made in view of the above problems, and has as its object to reduce the possibility of erroneous diagnosis in the diagnosis of oral cancer based on intraoral photographs.

  In order to solve the above problems, an oral cancer diagnostic system according to an aspect of the present invention provides a disease related to oral cancer in an oral cavity of a user based on an intraoral photograph including a plurality of images representing a predetermined region in the oral cavity. An oral cancer diagnosis system for diagnosing a part, comprising: an acquisition unit that acquires a plurality of partial images obtained by imaging a part in the oral cavity of a user; and a removal or reflection part of a reflection part that is an area where reflection occurs in the partial image A removing unit that obtains a partial image that does not include a reflection portion by extracting a partial image that does not include a generating unit that generates an intraoral photograph by combining a plurality of partial images that include the partial image obtained by the removing unit; A determining unit that determines the presence and state of a disease related to oral cancer based on the characteristics of the image of the predetermined determination target region represented in the inner photograph, and the presence or absence of a disease related to oral cancer determined by the determining unit And an output unit for outputting the determination information about the fine state.

  In addition, an oral cancer diagnosis program according to an embodiment of the present invention diagnoses a disease related to oral cancer in a user's oral cavity based on an intraoral photograph including a plurality of images representing a predetermined region in the oral cavity. An oral cancer diagnostic program for causing the computer to function as an oral cancer diagnostic system, the computer having an acquisition function of acquiring a plurality of partial images of a portion of the user's oral cavity, and an area where reflection occurs in the partial images. By removing the reflection part or extracting the partial image that does not include the reflection part, the removal function that obtains the partial image that does not include the reflection part, and a plurality of partial images including the partial image obtained by the removal function are combined into the oral cavity The presence / absence and state of a disease related to oral cancer are determined based on the generation function of generating a photograph and the characteristics of the image of the predetermined determination target region represented in the intraoral photograph. A determination function of, determined by the determination function, and an output function of outputting the determination information about the presence and status of diseases related oral cancer, to realize.

  According to the above aspect, the intraoral photograph is generated based on the partial image that does not include the reflection part. By diagnosing oral cancer based on the intraoral photograph thus generated, the possibility of erroneous diagnosis can be reduced.

  In the diagnosis of oral cancer based on intraoral photographs, the possibility of erroneous diagnosis can be reduced.

It is a figure showing the device composition of the oral cancer diagnostic system concerning this embodiment. It is a block diagram showing functional composition of an oral cancer diagnostic device of this embodiment. It is a hardware block diagram of an oral cancer diagnostic device. It is a figure showing the functional composition of a user's terminal. It is a figure showing the functional composition of a doctor's terminal. It is a figure which shows each part in the oral cavity used for a diagnosis typically. FIG. 4 is a diagram illustrating an example of a panoramic image synthesized based on a partial image and a plurality of partial images. FIG. 9 is a diagram illustrating an example of a process of extracting a partial image that does not include a reflection portion by a removing unit. FIG. 4 is a diagram schematically illustrating an example of a plurality of partial images including a reflective portion and a partial image from which the reflective portion has been removed. 9 is a flowchart illustrating processing of removing a reflection portion by a removing unit. It is a figure showing an example of an intraoral photograph generated by a generating part. It is a figure which shows the intraoral photograph from which the tooth area was removed. It is a figure which shows the flowchart of the determination processing of the presence or absence and state of an oral cancer by a determination part. It is a figure which shows the example of the information of the result of the determination of the progress level of oral cancer, and is a figure which shows typically the intraoral photograph in which the location with the possibility of oral cancer was marked. It is a flow chart which shows an example of processing contents of an oral cancer diagnostic method of this embodiment. 6 is a flowchart illustrating an example of processing contents in a user terminal. It is a flowchart which shows the example of the processing content in a doctor's terminal. It is a figure showing composition of an oral cancer diagnostic program.

  An embodiment of an oral cancer diagnosis system according to the present invention will be described with reference to the drawings. Note that, where possible, the same portions are denoted by the same reference numerals, and redundant description will be omitted.

  FIG. 1 is a diagram showing a device configuration of the oral cancer diagnosis system according to the present embodiment. The oral cancer diagnosis system 1 is a system that diagnoses a disease related to oral cancer in an oral cavity of a user based on an intraoral photograph including a plurality of images representing a predetermined region in the oral cavity. The oral cancer diagnostic system 1 shown in FIG. 1 includes an oral cancer diagnostic device 10, a user terminal 20, and a doctor terminal 30. The oral cancer diagnostic device 10 and the terminals 20, 30 can communicate with each other via a network N. is there.

  The oral cancer diagnostic device 10 is configured by, for example, a computer such as a server, but the device configuring the oral cancer diagnostic device 10 is not limited. The terminals 20 and 30 are configured by, for example, a stationary or portable personal computer, a high-performance mobile phone (smartphone), and the like. However, the devices that configure the terminals 20 and 30 are not limited. A portable terminal such as a terminal (PDA) may be used. Although FIG. 1 shows one terminal 20 and one terminal 30 as an example, the number is not limited to this.

  FIG. 2 is a diagram showing a functional configuration of the oral cancer diagnostic device 10 according to the present embodiment. The oral cancer diagnostic device 10 of the present embodiment is a device that diagnoses a disease related to oral cancer in the oral cavity of a user based on an intraoral photograph. The intraoral photograph is a photograph of the inside of the oral cavity acquired for diagnosing a disease in the oral cavity. The intraoral photograph is composed of, for example, a plurality of images representing a predetermined region in the oral cavity.

  As shown in FIG. 2, the oral cancer diagnostic device 10 functionally includes an acquisition unit 11, a preprocessing unit 12, a removal unit 13, a correction unit 14, a generation unit 15, a determination unit 16, and an output unit 17. In the present embodiment, the oral cancer diagnostic device 10 is configured by one device as shown in FIG. 2, but each functional unit included in the oral cancer diagnostic device 10 is distributed to a plurality of devices. May be done. Each functional unit of the oral cancer diagnostic device 10 will be described later in detail.

  Note that the block diagram shown in FIG. 2 shows blocks in functional units. These functional blocks (components) are realized by an arbitrary combination of hardware and / or software. The means for implementing each functional block is not particularly limited. That is, each functional block may be realized by one device physically and / or logically coupled, or two or more devices physically and / or logically separated from each other directly and / or indirectly. (For example, wired and / or wireless) and may be realized by these multiple devices.

  Functions include judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, deemed, Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, but not limited to these I can't. For example, a functional block (configuration unit) that causes transmission to function is called a transmitting unit (transmitting unit) or a transmitter (transmitter). In any case, as described above, the realization method is not particularly limited.

  For example, the oral cancer diagnosis device 10 according to one embodiment of the present invention may function as a computer. FIG. 3 is a diagram illustrating an example of a hardware configuration of the oral cancer diagnostic device 10 according to the present embodiment. The oral cancer diagnosis device 10 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

  In the following description, the term “apparatus” can be read as a circuit, a device, a unit, or the like. The hardware configuration of the oral cancer diagnostic device 10 may be configured to include one or more devices illustrated in FIG. 2, or may be configured without including some devices.

  The functions of the oral cancer diagnostic apparatus 10 are performed by reading predetermined software (program) on hardware such as the processor 1001 and the memory 1002 so that the processor 1001 performs an arithmetic operation. This is realized by controlling reading and / or writing of data in the storage 1003.

  The processor 1001 controls the entire computer by operating an operating system, for example. The processor 1001 may be configured by a central processing unit (CPU) including an interface with a peripheral device, a control device, an arithmetic device, a register, and the like. For example, each of the functional units 11 to 17 illustrated in FIG. 1 may be realized by the processor 1001.

  In addition, the processor 1001 reads a program (program code), a software module, and data from the storage 1003 and / or the communication device 1004 to the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operation described in the above embodiment is used. For example, each of the functional units 11 to 17 of the oral cancer diagnostic device 10 may be realized by a control program stored in the memory 1002 and operated by the processor 1001. Although it has been described that the various processes described above are executed by one processor 1001, the processes may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be implemented with one or more chips. Note that the program may be transmitted from a network via a telecommunication line.

  The memory 1002 is a computer-readable recording medium, and includes, for example, at least one of a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be called a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, and the like that can be executed to execute the shelf allocation information generation method according to one embodiment of the present invention.

  The storage 1003 is a computer-readable recording medium such as an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, and a magneto-optical disk (eg, a compact disk, a digital versatile disk, a Blu-ray). (Registered trademark) disk), a smart card, a flash memory (for example, a card, a stick, a key drive), a floppy (registered trademark) disk, and a magnetic strip. The storage 1003 may be called an auxiliary storage device. The storage medium described above may be, for example, a database including the memory 1002 and / or the storage 1003, a server, or any other suitable medium.

  The communication device 1004 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like.

  The input device 1005 is an input device that receives an external input (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, and the like). The output device 1006 is an output device that performs output to the outside (for example, a display, a speaker, an LED lamp, and the like). Note that the input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

  Each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured by a single bus, or may be configured by a different bus between the devices.

  The oral cancer diagnostic apparatus 10 includes hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The hardware may implement some or all of the functional blocks. For example, the processor 1001 may be implemented by at least one of these hardware.

  FIG. 4 is a diagram illustrating a functional configuration of the user terminal 20 of the present embodiment. As shown in FIG. 4, the terminal 20 includes an image acquisition unit 21, an image transmission unit 22, a determination result reception unit 23, and a determination result presentation unit 24. The terminal 20 includes an image sensor 25 and a display 29.

  The image sensor 25 is a sensor that optically acquires an image of an imaging target. The terminal 20 may include a depth sensor 26. The depth sensor 26 is a sensor that acquires a depth indicating a distance to a target by infrared rays.

  Further, the terminal 20 may include an acceleration sensor 27 and an angular velocity sensor 28. The acceleration sensor 27 is a sensor that detects an acceleration generated in the terminal 20. The angular velocity sensor 28 is a sensor that detects an angular velocity generated in the terminal 20.

  The image acquisition unit 21 acquires a partial image captured by the image sensor 25. The partial image is an image obtained by imaging a part in the oral cavity. The image acquisition unit 21 acquires a plurality of partial images.

  The image acquisition unit 21 can acquire the depth indicating the distance to the imaging target, which is detected by the depth sensor 26 when acquiring the partial image. Accordingly, the image acquisition unit 21 can acquire a partial image in which the depth is associated with each region including one or more pixels.

  Further, the terminal 20 can detect the acceleration at the time of acquiring the partial image by the acceleration sensor 27 and transmit the detected value in association with the partial image. Further, the terminal 20 can detect the angular velocity at the time of acquiring the partial image by the angular velocity sensor 28 and transmit the detected value in association with the partial image.

  Each of the plurality of partial images may be acquired as a still image, or may be extracted from each frame acquired as a moving image. Thereby, the image acquisition unit 21 can acquire a plurality of partial images of the same portion in the oral cavity. Further, the image acquisition unit 21 can acquire a plurality of partial images obtained by imaging the same portion in the oral cavity from different directions. In the present embodiment, the user can use the terminal 20 to capture an image of his own oral cavity.

  The image transmitting unit 22 transmits the plurality of partial images acquired by the image acquiring unit 21 to the oral cancer diagnostic device 10. The image transmission unit 22 can transmit a detection value detected by the sensor when acquiring each partial image in association with the partial image. The detected value can include at least one of the acceleration detected by the acceleration sensor 27 and the angular velocity detected by the angular velocity sensor 28.

  The determination result receiving unit 23 receives the determination information from the oral cancer diagnosis device 10. The determination information is information indicating a result determined by the determination unit 16 of the oral cancer diagnostic device regarding the presence / absence and state of a disease related to oral cancer. The determination of a disease related to oral cancer in the oral cavity is performed by the determination unit 16 of the oral cancer diagnostic device 10 based on the plurality of partial images transmitted by the image transmission unit 22. The result of the determination includes, for example, an intraoral photograph in which the location of the disease is marked, a comment on the found disease, and the like. The determination result receiving unit 23 may receive the determination information transmitted from the oral cancer diagnostic device 10 to the terminal 30 via the terminal 30.

  Further, the determination result receiving unit 23 receives the diagnostic information from the terminal 30. The diagnosis information is information indicating a result of a diagnosis of a disease related to oral cancer by a doctor, and is input at the terminal 30 by a doctor who refers to the determination information transmitted from the oral cancer diagnostic device 10 to the terminal 30 and the intraoral photograph. Information. The diagnosis information may include consultation necessity information regarding necessity of consultation by the user. Further, the determination result receiving unit 23 may receive the diagnostic information transmitted from the terminal 30 to the oral cancer diagnostic device 10 via the oral cancer diagnostic device 10.

  The determination result presentation unit 24 causes the display 29 to display the diagnostic information received by the determination result receiving unit 23. Further, the determination result presenting unit 24 may cause the display 29 to display the determination information.

  FIG. 5 is a diagram illustrating a functional configuration of the doctor's terminal 30 according to the present embodiment. As illustrated in FIG. 5, the terminal 30 includes a receiving unit 31, a presenting unit 32, a receiving unit 33, and a transmitting unit 34. The terminal 30 includes a display 35.

  The receiving unit 31 receives, from the oral cancer diagnosis device 10, determination information indicating a result of determination regarding a disease related to oral cancer. The determination information can include, for example, an intraoral photograph in which the location of the disease is marked, a comment on the discovered disease, and the like as the result of the determination.

  The presentation unit 32 causes the display 35 to display the determination information received by the reception unit 31. Thereby, the determination information can be presented to the doctor.

  The receiving unit 33 receives diagnostic information input to the terminal 30 by a doctor. That is, since the doctor can refer to the determination information, the intraoral photograph, and the like presented by the presentation unit 32, the doctor can input a diagnosis result based on his / her own knowledge and experience to the terminal 30 based on these references. The receiving unit 33 receives the result of the diagnosis input as described above as diagnostic information. It should be noted that the doctor can determine whether or not the user is required to consult based on the determination information and the reference to the intraoral photograph and the like. Then, the doctor can input the reception necessity information regarding the necessity of the consultation for prompting the user to perform the consultation in the diagnosis information.

  The transmitting unit 34 transmits the diagnostic information to the user terminal 20. Thus, the diagnostic information can be presented on the user terminal 20. The transmitting unit 34 may transmit the diagnostic information to the terminal 20 via the oral cancer diagnostic device 10. In this case, the output unit 17 of the oral cancer diagnostic device 10 transfers the diagnostic information from the transmitting unit 34 to the terminal 20 and causes the terminal 20 to output the diagnostic information.

  With reference to FIG. 2 again, each functional unit of the oral cancer diagnostic device 10 will be described. The acquisition unit 11 acquires a plurality of partial images obtained by imaging a portion in the oral cavity of the user. Specifically, the obtaining unit 11 obtains a partial image captured by the image sensor 25 of the terminal 20.

  FIG. 6 is a diagram schematically illustrating each part in the oral cavity to be diagnosed in the oral cancer diagnostic system 1 of the present embodiment. In the present embodiment, the parts to be diagnosed are the parts shown in the so-called intraoral standard photograph and the tongue d1, the buccal mucosa d2, the hard palate d3, the posterior triangular d4 and the gingiva d5 shown in FIG. The buccal mucosa d2 is a tissue that covers the inside of the cheek. The hard palate d3 is a ceiling portion of the oral cavity. The mortar triangle d4 is an area behind the wisdom tooth. The obtaining unit 11 obtains partial images showing these portions to be diagnosed.

  FIG. 7 is a diagram illustrating an example of a partial image and a panoramic image synthesized based on a plurality of partial images. As illustrated in FIG. 7, the acquisition unit 11 acquires, for example, a partial image A01 on the front right side of the tooth row, a partial image A02 on the front center, and a partial image A03 on the front left side. Each of these partial images may be obtained as a still image, or may be extracted from each frame obtained as a moving image.

  As will be described later, in the present embodiment, a panoramic image B is generated by synthesizing a plurality of partial images A01 to A03 based on a superimposed region in which the same portion in the oral cavity in each of the partial images A01 to A03 is represented. Is done. The panoramic image B shown in FIG. 7 constitutes a front image in an intraoral photograph. Various well-known techniques can be applied to the synthesis of the panoramic image based on the superimposed region of the partial images.

  The preprocessing unit 12 excludes a partial image satisfying a predetermined condition from partial images for forming an intraoral photograph. Specifically, when the acquiring unit 11 acquires a partial image in which at least one of the detected values of the angular velocity sensor 28 and the acceleration sensor 27 at the time of imaging is associated, the preprocessing unit 12 The partial image associated with the detection value of is not used for generating the intraoral photograph.

  If a large angular velocity or acceleration has occurred at the time of acquiring the partial image, it is highly likely that the partial image has blurred. The preprocessing unit 12 excludes the partial image associated with the detection value equal to or greater than the predetermined value from the partial images for forming the intraoral photograph, so that the blurred partial image is used for generating the intraoral photograph. Can be prevented from being used.

  The removing unit 13 acquires a partial image that does not include a reflective portion by removing a reflective portion that is a region where reflection occurs in the partial image or extracting a partial image that does not include the reflective portion. In an image obtained by imaging the inside of the oral cavity, reflection may occur due to saliva present in the oral cavity. In order to perform oral cancer diagnosis with high accuracy, it is necessary that the state of the oral cavity is correctly represented in the intraoral photograph, but the color of the image where the reflection occurs differs from the actual state. Is represented by In the present embodiment, a high-precision diagnosis is realized by acquiring a partial image that does not include a reflection portion.

  Specifically, the removing unit 13 can extract a partial image that does not include a reflection portion from the partial image acquired by the acquiring unit 11. FIG. 8 is a diagram illustrating an example of a process of extracting a partial image that does not include a reflected portion by the removing unit 13. As shown in FIG. 8, the partial image set as0 is a plurality of partial images acquired by the acquiring unit 11, and includes partial images a1, a2, a3,.

  For example, the removing unit 13 extracts a partial image having a characteristic image characteristic (for example, having a luminance equal to or more than a predetermined value) of the reflection part by a known image analysis technique. Specifically, first, the removing unit 13 performs grayscale conversion on the partial images a1, a2, a3,. Subsequently, the removing unit 13 acquires a luminance value for each pixel of each of the partial images a1, a2, a3,. Then, the removing unit 13 extracts a partial image having a pixel having a luminance value equal to or higher than a predetermined threshold value as a partial image including a reflection portion.

  In the example illustrated in FIG. 8, the removing unit 13 extracts a partial image a2 including a portion rf0 having a luminance value equal to or greater than a predetermined threshold value from among the partial images included in the set as0, as a partial image including a reflection portion. Then, the removing unit 13 extracts a set as1 of partial images including partial images a1, a3,... Other than the partial image a2 as a partial image to be used for generating an intraoral photograph. In addition, the removing unit 13 can extract a partial image that does not include a reflective portion by using various other known techniques.

  In addition, the removing unit 13 can obtain a partial image that does not include a reflective portion by removing a reflective portion that is a region where reflection occurs in the partial image. More specifically, the removing unit 13 converts the plurality of partial images so that the projection direction of the portion represented in the partial image is adjusted to a predetermined direction, and pixels representing the same portion in the oral cavity are associated with each other. Then, by averaging and synthesizing a plurality of partial images so as to obtain a partial image from which the reflection portion has been removed.

  With reference to FIG. 9 and FIG. 10, a process of removing the reflection portion by the removing unit 13 will be described in detail. FIG. 9 is a diagram schematically illustrating an example of a plurality of partial images including a reflective portion and a partial image from which the reflective portion has been removed. FIG. 10 is a flowchart showing the processing of removing the reflection portion by the removing unit 13.

  In step S21, the removing unit 13 acquires one partial image. In the example illustrated in FIG. 9, the removing unit 13 acquires the partial image A11. The partial image A11 includes a reflection part rf1.

  Subsequently, in step S22, the removing unit 13 converts the plurality of partial images so that the projection direction of the portion represented in the partial image is adjusted to a predetermined direction. Specifically, the removing unit 13 performs, for example, homography conversion on the partial image so as to conform to the standard of each image constituting the intraoral photograph. Homography transformation is a well-known image transformation technique, in which a two-dimensional image projected by projecting a three-dimensional object onto a certain plane is projected onto another plane using a projective transformation. Is a technology that generates The removing unit 13 performs homography conversion on the partial image A11 so as to conform to the standard of the front image in the intraoral photograph, and obtains a converted partial image A21.

  In step S23, the removing unit 13 determines whether or not processing of all partial images has been completed. If it is determined that the processing of all the partial images has been completed, the processing proceeds to step S24. On the other hand, if it is not determined that the processing of all the partial images has been completed, the process proceeds to step S21.

  In the example described with reference to FIG. 9, since the processing of the partial images A12 and A13 has not been completed, the processing of steps S31 and S32 for these partial images A12 and A13 is repeated. That is, the removing unit 13 performs the homography conversion on the partial image A12 including the reflection part rf2 to obtain the partial image A22. Further, the removing unit 13 performs a homography conversion on the partial image A13 including the reflection portion rf3 to obtain a partial image A23.

  In step S24, the removing unit 13 performs positioning such that pixels representing the same portion in the oral cavity are associated with each other, and performs averaging synthesis of the plurality of partial images. In the example described with reference to FIG. 9, the removing unit 13 aligns the converted partial images A21 to A23 so that pixels representing the same part in the oral cavity are associated with each other, and performs averaging. Combine. The averaging synthesis is a well-known technique in the field of image processing.

  Each of the partial images A11 to A13 is an image obtained by capturing the same portion in the oral cavity from different directions. Therefore, in the partial image A22 and the partial image A23, no reflection occurs at the position of the reflection portion rf1 in the partial image A21. In the partial image A21 and the partial image A23, no reflection occurs at the position of the reflection portion rf2 in the partial image A22. In the partial image A21 and the partial image A22, no reflection occurs at the position of the reflection portion rf3 in the partial image A23.

  Even if reflection occurs in a certain part of one partial image, the reflection is removed by adding and averaging the plurality of other partial images in which no reflection occurs in the part and the one partial image. Images can be obtained. In the example described with reference to FIG. 9, the removing unit 13 can acquire the partial image C1 from which the reflection part has been removed by aligning the partial images A21 to A23 and performing averaging synthesis.

  Referring to FIG. 2 again, correction unit 14 corrects the luminance of each region based on the depth associated with each region of the partial image. In imaging of the oral cavity, since the light amount is smaller in a deeper region, the luminance value tends to be smaller in a deeper region. The correction unit 14 corrects the luminance value to correct the difference in light amount.

  Specifically, the correction unit 14 obtains the depth of each area, and based on the correspondence between the preset depth and the brightness correction value, sets the brightness correction value associated with the obtained depth to the brightness of the area. By adding to the value, the luminance of each area is corrected. Hereinafter, an example of this correction will be described.

For example, the depth of a certain area is D, the RGB values of pixels before correction are (r ₀ , g ₀ , b ₀ ), and the pixel correction values are (r ₁ , g ₁ , b ₁ ), (r ₂ , g ₂ , It is assumed that the following correspondence between the depth D and the pixel correction value is set in advance as b ₂ ). In this example, correction for increasing the luminance value is realized by decreasing the value of each color of the RGB values of the pixels constituting each area. That is, the pixel correction value corresponds to a luminance correction value.

Correspondence relationship between depth D and pixel correction value:
When the depth D is less than n, the RGB value of the pixel is not corrected.
Depth D is corrected RGB value when a pixel less than _{n m (r 0 -r 1,} g 0 -g 1, b 0 -b 1)
Depth D is corrected RGB value when a pixel over _{m (r 0 -r 2, g} 0 -g 2, b 0 -b 2)

  The correction unit 14 corrects the RGB values of the pixels in each area according to the depth of each area with reference to the correspondence. In the above correspondence, the values of n and m and the pixel correction value may be set in design. In this way, the brightness of the pixels in each area is appropriately corrected according to the depth. As a result, it is possible to obtain an intra-oral photograph in which the state of the intra-oral can be easily recognized uniformly.

  Further, when the first region of the regions of the plurality of partial images has a depth greater than the depth of the second region different from the first region, the correction unit 14 may adjust the luminance of the first region. May be corrected to a value larger than that before correction.

  As described above, the amount of light in a region where the depth of the partial image representing the portion in the oral cavity is deep tends to be small, and the image in that region tends to be dark. Therefore, in the oral cancer diagnosis apparatus 10 of the present embodiment, the correction unit 14 corrects the luminance value of a region having a depth greater than that of another region to a larger value.

  The generation unit 15 generates an intraoral photograph by combining a plurality of partial images including the partial images acquired by the removal unit 13. The partial image used for generating the intraoral photograph may have been subjected to preprocessing and correction processing by the preprocessing unit 12 and the correction unit 14.

  Specifically, as illustrated in FIG. 7, the generation unit 15 generates a panoramic image by synthesizing a partial image based on a superimposed region in which the same portion in the oral cavity is represented, thereby generating a panoramic image. Generate multiple images according to the standard. A well-known technique for generating a so-called panoramic image can be applied to generation of an intraoral photograph by combining partial images.

  Specifically, the generation unit 15 adjusts the orientation and the arrangement of the acquired partial images. Subsequently, the generating unit 15 generates one panoramic image by superimposing and synthesizing adjacent partial images based on the superimposed region in which the same part in the oral cavity is represented. One panoramic image constitutes one of a plurality of images constituting an intraoral photograph.

  When generating the intraoral photograph based on the partial image in which the teeth are represented, the generating unit 15 generates the intraoral photograph by superimposing each partial image based on the superimposed region in which the teeth are represented. When generating an intraoral photograph in which the tongue, the buccal mucosa, the hard palate, and the posterior triangle are represented, the generating unit 15 sets each of the partial images as a superimposed region with the region in which the teeth, lips, uvula, etc. are represented. Can be superimposed to generate an intraoral photograph.

  FIG. 11 is a diagram illustrating an example of an intraoral photograph generated by the generation unit 15. The intraoral photograph MP is composed of a plurality of images representing a predetermined region in the oral cavity. The example shown in FIG. 11 is a standard intraoral photograph by the so-called five-image photographing method. The method of photographing the intraoral standard photograph includes a five-photographing method, a nine-photographing method, a twelve-photographing method, a fourteen-photographing method, and the like, depending on the number of constituent images and the imaging portion of each image. As illustrated in FIG. 11, the generating unit 15 can generate an intraoral photograph MP photographed by, for example, a five-image photographing method. The intraoral photograph MP includes a palatal image mp1, a right image mp2, a front image mp3, a left image mp4, and a lingual image mp5.

  In addition, the generation unit 15 removes the tooth region, which is the region where the teeth are represented, from the intraoral photograph by a predetermined image processing technique. The oral cancer diagnosis system of the present embodiment is a system for diagnosing oral cancer. Therefore, the tooth area is not a diagnosis target area. The generation unit 15 can generate an intra-oral photograph showing the diagnosis target region by removing the tooth region.

  Specifically, the generation unit 15 acquires an intraoral photograph generated by superimposing the partial images. Subsequently, the generation unit 15 performs a binarization process on the intraoral photograph to obtain a binarized image. In the binarized image, the tooth region is represented by, for example, white, and the region other than the tooth region is represented by black. Then, the generation unit 15 performs blob analysis on the binarized image, recognizes the tooth region and the region other than the tooth region, and removes the tooth region. The method for recognizing and removing the tooth region from the intraoral photograph is not limited to these methods, and various known image processing techniques can be applied.

  FIG. 12 is a diagram showing the intraoral photograph MP1 from which the tooth region has been removed. The intraoral photograph MP1 is obtained by removing the tooth region from the intraoral photograph MP shown in FIG. As shown in FIG. 12, the intraoral photograph MP1 includes a palatal image mp11, a right image mp12, a front image mp13, a left image mp14, and a lingual image mp15 from which the tooth region has been removed.

  The palatal-side image mp11 is an image obtained by removing the tooth region t11 from the palatal-side image mp1, and includes a determination target region r11. The determination target region r11 includes, for example, a region where gingiva is represented. The right image mp12, the front image mp13, the left image mp14, and the lingual image mp15 have tooth regions t12, t13, t14, and t15 from the right image mp2, the front image mp3, the left image mp4, and the lingual image mp5, respectively. This is an image that has been removed, and includes each of the determination target regions r12, r13, r14, and r15.

  The determination unit 16 determines the presence / absence and state of a disease related to oral cancer based on the characteristics of the image of the predetermined determination target region represented in the intraoral photograph. Specifically, the determination unit 16 performs the determination based on the intraoral photograph from which the tooth region has been removed. That is, the determination unit 16 determines the presence or absence and the state of the oral cancer as a determination target region in the region shown in the intra-oral photograph other than the tooth region. At a place where oral cancer has occurred, color changes such as vitiligo and shape changes such as convexity may occur. Therefore, the determination unit 16 determines the characteristics of an image indicating these changes in the oral cavity. Detected from inside photos.

  The determination of a disease related to oral cancer will be described with reference to FIGS. FIG. 13 is a diagram illustrating a flowchart of the determination processing of the presence / absence and state of oral cancer by the determination unit 16.

  In step S31, the determination unit 16 acquires the intraoral photograph generated by the generation unit 15. As described above, the determination unit 16 can acquire the intraoral photograph from which the tooth region has been removed.

  In step S32, the determination unit 16 inputs the intraoral photograph acquired in step S31 into the oral cancer determination model as input data. The oral cancer determination model is a model for determining the presence / absence and progress level of oral cancer using image data of an intraoral photograph as an input value, and is constructed in advance by machine learning.

  The oral cancer determination model is constructed as a classifier such as a so-called Support Vector Machine (SVM) using, as learning data, an intraoral photograph in which a cancer site is masked. The determination results output from the oral cancer determination model include, for example, information on the presence / absence of oral cancer and the classification of stages of the degree of progression, such as soundness, progress levels T1, T2, T3, and T4 (low: T1 → high: T4). It is output as information indicating the location of oral cancer.

  In addition, the oral cancer determination model may be constructed by a neural network. An oral cancer determination model including a neural network is generated by performing machine learning including error calculation (error back propagation) based on a determination value of the progress level of oral cancer obtained by inputting an intraoral photograph for learning. You. Various loss functions can be applied to the error calculation.

  In addition, the determination unit 16 may determine the region where the oral cancer has occurred and the progress level based on the threshold value set for each region with respect to the pixel value of the intraoral photograph, regardless of the model.

  In step S33, the determination unit 16 outputs the classification result by the model, that is, the information indicating the presence / absence of oral cancer and the progress level output in step S32. FIG. 14 is a diagram illustrating an example of information on the result of the determination of the progress level of oral cancer, and is a diagram schematically illustrating an intraoral photograph in which a location where there is a possibility of oral cancer is marked. As shown in FIG. 14, the intraoral photograph mp20 output by the determination unit 16 includes a mark D of a location of oral cancer. In connection with the mark D of the location of the oral cancer, the information of the progress level of the location may be output, for example, as a text such as "The marked location is suspected of the progress level T0 of the oral cancer". Good.

  Referring to FIG. 2 again, the output unit 17 outputs the determination information on the presence / absence and state of the disease related to the oral cancer, determined by the determination unit 16. Specifically, the output unit 17 causes the doctor's terminal 30 to output the intraoral photograph and the text in which the position of the oral cancer is marked, which is output as the determination information. More specifically, the output unit 17 causes the display 35 of the doctor's terminal 30 to display the determination information. The output unit 17 may output the determination information to the user terminal 20. The output unit 17 may store the determination information in a predetermined storage unit.

  Next, an oral cancer diagnosis method in the oral cancer diagnostic system 1 and the oral cancer diagnostic device 10 will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of processing contents of the oral cancer diagnosis method of the present embodiment.

  In step S1, the acquisition unit 11 acquires a plurality of partial images obtained by imaging a portion in the oral cavity of the user. Next, in step S2, the preprocessing unit 12 excludes a partial image that may have a blur from a partial image for generating an intraoral photograph based on a predetermined detection value in the partial image. Here, the detection value used for the determination is, for example, an acceleration and an angular velocity generated in the imaging device, which are acquired at the time of capturing the partial image.

  In step S3, the removing unit 13 acquires a partial image that does not include a reflective portion by removing a reflective portion that is a region where the partial image reflects, or extracting a partial image that does not include the reflective portion. The process of removing the reflection portion is, for example, as described with reference to FIG.

  In step S4, the correction unit 14 corrects the luminance of each area based on the depth associated with each area of the partial image.

  In step S5, the generation unit 15 generates an intraoral photograph by combining a plurality of partial images. Subsequently, in step S6, the generation unit 15 removes the tooth region from the intraoral photograph generated in step S5 by a predetermined image processing technique.

  In step S7, the determination unit 16 determines the presence / absence and state of a disease related to oral cancer based on the characteristics of the image of the predetermined determination target region represented in the intraoral photograph. Specifically, the determination unit 16 performs the determination based on the intraoral photograph from which the tooth region has been removed in step S6. The determination process is, for example, as described with reference to FIG.

  In step S8, the determination unit 16 determines whether there is a disease related to oral cancer in the determination result of the determination processing. If it is determined that there is a disease related to oral cancer, the process proceeds to step S9. On the other hand, if it is not determined that there is a disease related to oral cancer, the process proceeds to step S10.

  In step S9, the determination unit 16 generates, as the determination information, an intraoral photograph marked at the location of the disease related to oral cancer. The determination unit 16 may also generate and output text information indicating the progress level of the location.

  In step S10, the output unit 17 outputs the determination information regarding the presence / absence and state of the disease related to oral cancer, determined by the determination unit 16. The output unit 17 may output the determination information at the doctor's terminal 30 or at the user's terminal 20.

  Next, with reference to FIG. 16, the processing in the terminal 30 of the doctor of the oral cancer diagnosis system 1 will be described. FIG. 16 is a flowchart showing an example of the processing contents in the doctor's terminal 30.

  In step S <b> 41, the receiving unit 31 receives, from the oral cancer diagnostic device 10, determination information indicating a result of determination regarding a disease related to oral cancer in the oral cavity. The determination information may include an intraoral photograph in which the location of the oral cancer is marked, a comment on the found disease, and the like.

  In step S42, the presentation unit 32 presents the determination information received by the reception unit 31 in step S41 to the doctor. The presentation unit 32 causes the display 29 to display the determination information, for example.

  In step S43, the receiving unit 33 receives the diagnostic information input by the doctor in response to the presentation of the determination information. The diagnosis information may include consultation necessity information regarding necessity of consultation by the user.

  In step S44, the transmitting unit 34 transmits the diagnostic information received in step S43 to the user terminal 20. The transmitting unit 34 may transmit the diagnostic information to the terminal 20 via the oral cancer diagnostic device 10.

  Next, the processing in the terminal 20 of the user of the oral cancer diagnosis system 1 will be described with reference to FIG. FIG. 17 is a flowchart illustrating an example of processing contents in the user terminal 20.

  In step S51, the image sensor 25 captures a plurality of partial images. Then, the image acquiring unit 21 acquires a plurality of partial images captured by the image sensor 25.

  In step S52, the image transmitting unit 22 transmits the plurality of partial images acquired by the image acquiring unit 21 in step S51 to the oral cancer diagnostic device 10.

  In step S53, the determination result receiving unit 23 receives, from the doctor's terminal 30, diagnostic information that is the result of determining a disease related to oral cancer. The determination result receiving unit 23 may receive the diagnostic information transmitted via the oral cancer diagnostic device 10.

  In step S54, the determination result presenting unit 24 presents the diagnosis information received by the determination result receiving unit 23 in step S53 to the user. The determination result presenting unit 24 causes the display 29 to display diagnostic information, for example.

  Next, an oral cancer diagnosis program for causing a computer to function as the oral cancer diagnostic apparatus 10 of the present embodiment will be described. FIG. 18 is a diagram showing a configuration of the oral cancer diagnosis program P1.

  The oral cancer diagnosis program P1 includes a main module m10, an acquisition module m11, a preprocessing module m12, a removal module m13, a correction module m14, a generation module m15, and a determination module that comprehensively control the oral disease diagnosis processing in the oral cancer diagnosis apparatus 10. m16 and an output module m17. The modules m11 to m17 realize the functions of the acquisition unit 11, the preprocessing unit 12, the removal unit 13, the correction unit 14, the generation unit 15, the determination unit 16, and the output unit 17 in the oral cancer diagnostic device 10. Is done. The oral cancer diagnosis program P1 may be transmitted via a transmission medium such as a communication line, or may be stored in a recording medium M1 as shown in FIG. .

  In the oral cancer diagnostic system 1, the oral cancer diagnostic apparatus 10, the oral cancer diagnostic method, and the oral cancer diagnostic program P1 of the present embodiment described above, an intraoral photograph is generated based on a partial image that does not include a reflective portion. By diagnosing oral cancer based on the intraoral photograph thus generated, the possibility of erroneous diagnosis can be reduced.

  In the oral cancer diagnosis system according to another aspect, the removing unit determines whether or not the partial image includes a reflective portion based on the pixel value of each pixel included in the partial image, and the removal unit acquires the partial image. From the plurality of partial images, a partial image that does not include the reflection part may be extracted.

  According to the above embodiment, only a partial image that does not include a reflection portion can be used for generating an intraoral photograph.

  In the oral cancer diagnosis system according to another aspect, the removing unit may remove the reflection part in the partial image by a predetermined image processing technique.

  According to the above aspect, it is possible to obtain a partial image that does not include a reflection portion based on the acquired partial image.

  Further, in the oral cancer diagnosis system according to another embodiment, the acquisition unit acquires a plurality of partial images of the same portion in the oral cavity taken from different directions, and the removal unit removes a portion represented in the partial image. By converting a plurality of partial images so that the projection direction is adjusted to a predetermined direction, by performing averaging synthesis of the plurality of partial images by performing alignment so that pixels representing the same portion in the oral cavity are associated with each other, A partial image from which the reflection portion has been removed may be obtained.

  According to the above embodiment, a plurality of partial images obtained by imaging the same portion from different directions are obtained. In a plurality of partial images acquired in this way, even if reflection occurs at a specific portion in one partial image, there is a high possibility that the reflection portion and the reflection state are different in other partial images. A plurality of partial images converted so that the projection direction of a certain part is aligned in a predetermined direction are aligned and averaged and synthesized, so that it is possible to obtain a partial image from which a reflected part has been removed. Become.

  In the oral cancer diagnosis system according to another aspect, the generation unit removes, from the intraoral photograph, a tooth region that is a region where the teeth are represented by a predetermined image processing technique, and the determination unit determines that the tooth is generated by the generation unit. The presence / absence and state of a disease related to oral cancer may be determined based on the characteristics of the image of the predetermined determination target region shown in the intraoral photograph from which the region has been removed.

  According to the above aspect, since the determination regarding the oral cancer is performed based on the intraoral photograph from which the tooth region which is not the target of the diagnosis of the oral cancer is removed, the diagnosis regarding the oral cancer is appropriately performed.

  In the oral cancer diagnosis system according to another aspect, the output unit causes the determination information to be output at a first terminal of a doctor different from the user, and inputs the determination information to the first terminal according to the output of the determination information. The performed diagnosis information on the diagnosis of the disease may be output at the second terminal of the user.

  According to the above aspect, the result of the determination regarding oral cancer is output at the doctor's terminal, so that the determination result regarding the oral cancer of the user can be easily notified to the doctor. Furthermore, since the diagnostic information input by the doctor according to the notified determination result can be notified to the user, it is possible for the user to appropriately recognize the diagnostic result.

  In the oral cancer diagnosis system according to another aspect, the output unit may cause the second terminal to output the determination information.

  According to the above aspect, since the result of the determination regarding the oral cancer is output at the user's terminal, the user can easily be notified of the result of the determination regarding the oral cancer of the user himself.

  In the oral cancer diagnosis system according to another aspect, the acquisition unit associates a detection value of at least one of the angular velocity and the acceleration detected by the device that captured the partial image when capturing the partial image. The oral cancer diagnostic system further includes a preprocessing unit that acquires the partial image that is present and excludes the partial image associated with the detection value equal to or greater than the predetermined value from the partial image used for generating the intraoral photograph by the generation unit. It may be good.

  According to the embodiment, the partial image associated with the detection value equal to or greater than the predetermined value of the angular velocity sensor or the acceleration sensor is excluded from the image used for generating the intraoral photograph. As a result, the blurred partial image is not used for the synthesis of the intraoral photograph. Therefore, it is possible to obtain an intraoral photograph in which the state of the oral cavity can be easily recognized.

  In the oral cancer diagnosis system according to another aspect, the acquisition unit acquires a partial image in which a depth indicating a distance to an imaging target is associated with each region including one or more pixels, and the oral cancer diagnosis system includes: And a correction unit that corrects the luminance of each region of the partial image based on the depth.

  According to the above-described embodiment, based on the knowledge that the light from the light source is harder to hit in a deeper area in an image obtained by capturing the inside of the oral cavity, the oral cavity is synthesized by combining partial images in which the brightness of each area is corrected based on the depth. Since the internal photograph is generated, a diagnosis based on the intraoral photograph in which the state in the oral cavity can be uniformly grasped becomes possible.

  Although the present embodiment has been described in detail above, it is obvious to those skilled in the art that the present embodiment is not limited to the embodiment described in this specification. This embodiment can be implemented as a modified or changed embodiment without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description in this specification is for the purpose of illustration and description, and does not have any restrictive meaning to the present embodiment.

  Each aspect / embodiment described in the present disclosure is applicable to LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA (W-CDMA). (Registered trademark), GSM (registered trademark), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (Registered trademark), a system using other appropriate systems, and / or a next-generation system extended based on these systems.

  The processing procedure, sequence, flowchart, and the like of each aspect / embodiment described in the present disclosure may be rearranged as long as there is no inconsistency. For example, the methods described herein present elements of various steps in a sample order, and are not limited to the specific order presented.

  Information and the like can be output from an upper layer (or lower layer) to a lower layer (or upper layer). Input and output may be performed via a plurality of network nodes.

  The input and output information and the like may be stored in a specific place (for example, a memory) or may be managed by a management table. Information that is input and output can be overwritten, updated, or added. The output information or the like may be deleted. The input information or the like may be transmitted to another device.

  The determination may be made based on a value (0 or 1) represented by one bit, a Boolean value (Boolean: true or false), or a comparison of numerical values (for example, a predetermined value). Value).

  Each aspect / embodiment described in the present disclosure may be used alone, may be used in combination, or may be switched and used with execution. Further, the notification of the predetermined information (for example, the notification of “X”) is not limited to being explicitly performed, and is performed implicitly (for example, not performing the notification of the predetermined information). Is also good.

  Software, regardless of whether it is called software, firmware, middleware, microcode, a hardware description language, or any other name, instructions, instruction sets, codes, code segments, program codes, programs, subprograms, software modules , Applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.

  Also, software, instructions, and the like may be transmitted and received via a transmission medium. For example, if the software uses a wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or a wireless technology such as infrared, wireless and microwave, the website, server, or other When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

  The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., that can be referred to throughout the above description are not limited to voltages, currents, electromagnetic waves, magnetic or magnetic particles, optical or photons, or any of these. May be represented by a combination of

  Note that terms described in the present disclosure and / or terms necessary for understanding the present specification may be replaced with terms having the same or similar meaning.

  The terms "system" and "network" as used in this disclosure are used interchangeably.

  Further, the information, parameters, and the like described in the present disclosure may be represented by an absolute value, may be represented by a relative value from a predetermined value, or may be represented by other corresponding information.

  The terms "determining" and "determining" as used in the present disclosure may encompass a wide variety of operations. `` Judgment '', `` decision '', for example, judgment (judging), calculation (calculating), calculation (computing), processing (processing), derivation (deriving), investigating (investigating), searching (looking up, search, inquiry) (E.g., searching in a table, database, or another data structure), ascertaining may be considered "determined", "determined", and the like. Also, “determining” and “deciding” include receiving (eg, receiving information), transmitting (eg, transmitting information), input (input), output (output), and access. (accessing) (for example, accessing data in a memory) may be regarded as “determined” or “determined”. In addition, `` judgment '' and `` decision '' means that resolving, selecting, selecting, establishing, establishing, comparing, etc. are regarded as `` judgment '' and `` decided ''. May be included. In other words, “judgment” and “decision” may include deeming any operation as “judgment” and “determined”. “Judgment (determination)” may be read as “assuming”, “expecting”, “considering”, or the like.

  The terms "connected," "coupled," or any variation thereof, mean any direct or indirect connection or connection between two or more elements that It may include the presence of one or more intermediate elements between the two elements "connected" or "coupled." The coupling or connection between the elements may be physical, logical, or a combination thereof. For example, “connection” may be read as “access”. As used in this disclosure, two elements may be implemented using at least one of one or more wires, cables, and printed electrical connections, and as some non-limiting and non-exhaustive examples, in the radio frequency domain. , Can be considered "connected" or "coupled" to each other using electromagnetic energy having wavelengths in the microwave and optical (both visible and invisible) regions, and the like.

  Reference to “based on” as used in the present disclosure does not mean “based solely on” unless otherwise indicated. In other words, the phrase "based on" means both "based only on" and "based at least on."

  In using the designations “first,” “second,” and the like in this disclosure, any reference to the element does not generally limit the quantity or order of the element. These designations may be used herein as a convenient way to distinguish between two or more elements. Thus, a reference to a first and second element does not mean that only two elements may be employed therein, or that the first element must somehow precede the second element.

  As long as “include”, “including”, and variations thereof, are used in the present description or claims, these terms are equivalent to the term “comprising” It is intended to be comprehensive. Further, it is intended that the term "or", as used herein or in the claims, not be the exclusive OR.

  In the present disclosure, where articles are added by translation, for example, a, an and the in English, the present disclosure may include that the nouns following these articles are plural.

  This disclosure also includes a plurality of devices, unless the context or technology clearly indicates that only one device exists.

  Throughout this disclosure, the singular may include the plural unless the context clearly dictates otherwise.

  DESCRIPTION OF SYMBOLS 1 ... Oral cancer diagnostic system, 10 ... Oral cancer diagnostic apparatus, 11 ... Acquisition part, 12 ... Preprocessing part, 13 ... Removal part, 14 ... Correction part, 15 ... Generation part, 16 ... Judgment part, 17 ... Output part, 20 terminal, 21 image acquisition unit, 22 image transmission unit, 23 determination result receiving unit, 24 determination result presentation unit, 25 image sensor, 26 depth sensor, 27 acceleration sensor, 28 angular velocity sensor, 29 display, 30 terminal, 31 reception unit, 32 presentation unit, 33 reception unit, 34 transmission unit, 35 display, M1 recording medium, m10 main module, m11 acquisition module, m12 front Processing module, m13: removal module, m14: correction module, m15: generation module, m16: determination module, m17: output module, N: network, P1: oral cancer diagnosis Program.

Claims (10)

An oral cancer diagnostic system for diagnosing a disease related to oral cancer in the oral cavity of a user based on an intraoral photograph composed of a plurality of images representing a predetermined region in the oral cavity,
An acquisition unit configured to acquire a plurality of partial images obtained by imaging a portion in the oral cavity of the user,
By removing the reflection portion which is a region where reflection has occurred in the partial image or extracting the partial image not including the reflection portion, a removal unit to obtain a partial image not including the reflection portion,
A generation unit that generates the intraoral photograph by synthesizing a plurality of partial images including the partial image obtained by the removal unit,
Based on the characteristics of the image of the predetermined determination target region represented in the intraoral photograph, a determination unit that determines the presence or absence and state of a disease related to oral cancer,
An output unit that has been determined by the determination unit and outputs determination information about the presence or absence and state of a disease related to oral cancer,
An oral cancer diagnostic system comprising: The removal unit determines whether or not the partial image includes the reflection portion based on a pixel value of each pixel included in the partial image, and from the plurality of partial images acquired by the acquisition unit, Extract partial images that do not include the part,
The oral cancer diagnostic system according to claim 1.   The oral cancer diagnostic system according to claim 1, wherein the removing unit removes the reflection part in the partial image by a predetermined image processing technique. The acquisition unit acquires a plurality of partial images obtained by imaging the same portion in the oral cavity from different directions,
The removing unit converts the plurality of partial images so that the projection direction of the part represented in the partial image is aligned with a predetermined direction, so that pixels representing the same part in the oral cavity are associated with each other. By performing averaging synthesis of the plurality of partial images by performing position alignment, a partial image from which the reflection portion has been removed is obtained,
The oral cancer diagnostic system according to claim 3. From the intraoral photograph, the generation unit removes a tooth region that is a region where a tooth is represented by a predetermined image processing technique,
The determination unit is configured to determine the presence and absence of a disease related to oral cancer based on a feature of an image of a predetermined determination target region represented in the intraoral photograph from which the tooth region has been removed by the generation unit.
The oral cancer diagnostic system according to any one of claims 1 to 4. The output unit causes the determination information to be output at a first terminal of a doctor different from the user,
Diagnostic information relating to the diagnosis of a disease, which is input to the first terminal in response to the output of the determination information, is output at the second terminal of the user.
The oral cancer diagnostic system according to any one of claims 1 to 5. The output unit causes the second terminal to output the determination information.
The oral cancer diagnostic system according to claim 6. The acquisition unit, at the time of imaging the partial image, acquires the partial image associated with at least one detection value of the angular velocity and the acceleration detected in the device that has captured the partial image,
The oral cancer diagnostic system,
A preprocessing unit that excludes the partial image associated with the detection value equal to or greater than a predetermined value from the partial image used for generating the intraoral photograph by the generation unit,
The oral cancer diagnostic system according to any one of claims 1 to 7. The acquisition unit acquires the partial image associated with a depth indicating a distance to an imaging target for each region including one or more pixels,
The oral cancer diagnostic system,
A correction unit that corrects the luminance of each region of the partial image based on the depth,
The oral cancer diagnostic system according to any one of claims 1 to 8. An oral cancer diagnostic program for causing a computer to function as an oral cancer diagnostic system for diagnosing a disease related to oral cancer in the oral cavity of a user based on an intraoral photograph composed of a plurality of images representing a predetermined region in the oral cavity. hand,
On the computer,
An acquisition function of acquiring a plurality of partial images obtained by imaging a portion in the oral cavity of the user,
By removing the reflection portion which is a region where reflection occurs in the partial image or extracting the partial image not including the reflection portion, a removal function of acquiring a partial image not including the reflection portion,
A generation function of synthesizing a plurality of partial images including the partial image obtained by the removal function to generate the intraoral photograph,
Based on the characteristics of the image of the predetermined determination target area represented in the intraoral photograph, a determination function of determining the presence or absence and state of a disease related to oral cancer,
An output function of outputting determination information on the presence or absence and state of a disease related to oral cancer, determined by the determination function,
An oral cancer diagnosis program that realizes.

Images