JP2022082952A - Discrimination device, discrimination terminal device, control method and control program of discrimination device, and discrimination system - Google Patents

Abstract

To provide a discrimination device, etc. for achieving accurate discrimination of an oral cavity tumor by applying findings of skilled medical specialists to discrimination processing effectively.SOLUTION: When a captured image is discriminated to be an image that does not include an oral cavity tumor by a first discrimination part, and the captured image is discriminated to be an image that does not include an oral cavity tumor by a second discrimination part, or when the captured image is discriminated to be an image that does not include an oral cavity tumor by a third discrimination part, a discrimination device outputs health information indicating that an oral cavity tumor does not exist in the oral cavity of a patient. When the captured image is discriminated to be an image including a benign oral cavity tumor by the third discrimination part, the discrimination device outputs benign tumor information indicating that a benign oral cavity tumor exists in the oral cavity of the patient. When the captured image is discriminated to be an image including a malignant oral cavity tumor by the third discrimination part, the discrimination device outputs malignant tumor information indicating that a malignant oral cavity tumor exists in the oral cavity of the patient.SELECTED DRAWING: Figure 9

Description

The embodiments disclosed in the present specification include a discrimination device, a discrimination terminal device, a control method and control program of the discrimination device, and a discrimination for discriminating the presence or absence of an oral tumor and discriminating between benign and malignant oral tumors. Regarding the system.

Conventionally, screening for oral cancer is mainly performed by inspection and palpation by a doctor. In recent years, a fluorescence observation device has been put into practical use as a screening device used in the screening of oral cancer.

The fluorescence observation device is an optical device that utilizes the characteristics of excitation light in healthy tissues and pathological tissues in the oral cavity with respect to external light irradiation, and irradiates blue light having a wavelength of 435 to 460 nm. Since the healthy tissue in the oral cavity has collagen, the fluorescence visibility is maintained in the healthy tissue by the blue light irradiated by the fluorescence observation device. On the other hand, in the pathological tissue in the oral cavity, since collagen is broken, even if blue light is irradiated by the fluorescence observation device, the fluorescence visibility is lowered and a dark color is observed. As described above, by using the fluorescence observation device, the doctor can carry out a very simple and non-invasive screening test without contacting the affected area.

For example, Patent Document 1 describes an inspection mirror that detects fluorescence emitted from a target tissue or other weak light, and covers the target tissue to prevent intrusion of inconvenient light such as ambient room light into the observation area. Describes a cover device or a front-end device. Further, Patent Document 2 describes an oral examination management system that manages digital image data in the oral cavity of a patient taken by a camera and a fluorescence visualization device and supports the oral examination.

In Patent Document 3, a feature vector having a measured value of a specific medical examination item as an element is used as an input value, and a predicted model learned by using a measured value of a predetermined tumor marker as a teacher value is used as an input value. A machine learning device that predicts the possibility of Further, Patent Documents 4 and 5 describe a technique for discriminating between benign and malignant tumors based on a learning model learned using medical images.

Japanese Patent No. 5681162 Japanese Patent No. 6453607 Japanese Unexamined Patent Publication No. 2018-68752 Japanese Unexamined Patent Publication No. 2018-183583 Special Table 2002-530133

The use of conventional techniques related to fluorescence observation has improved the possibility of screening oral mucosal lesions from healthy mucosa, but the techniques related to fluorescence observation alone have improved the possibility of screening oral mucosal lesions. It was difficult to distinguish between them. For this reason, even if a fluorescence observation device is used in the screening of oral cancer, there is no choice but to rely on the subjective judgment of the doctor, and the accuracy of the diagnosis depends on the experience of the doctor in charge. Was there.

Further, in the conventional tumor discrimination technique using machine learning, the measurement value by the medical image or the inspection device is simply learned as input data, and the discrimination process that effectively reflects the knowledge of a skilled specialist is realized. I wasn't. In such machine learning, features input by humans may be used as teacher data, and the discrimination performance by the trained discrimination model depends on the person who set the features, which is useful and potential features. I could have missed it.

The disclosed discrimination device, discrimination terminal device, control method and control program of the discrimination device, and the discrimination system effectively apply the knowledge of a skilled specialist to the discrimination process to realize highly accurate discrimination of oral tumors. Make it possible.

The disclosed discrimination device is a discrimination device that discriminates the presence or absence of an oral tumor and the benign or malignant of an oral tumor by using a photographed image of the oral cavity of a patient irradiated with blue light. A transforming unit that generates a spatial frequency spectrum by Fourier transforming an image and a spatial frequency spectrum that has been trained for the first discrimination model are input, and the captured image corresponding to the input spatial frequency spectrum is an oral tumor. A captured image is input to the first discriminant unit that discriminates whether or not the image contains the tumor and the trained second discriminant model, and whether or not the input captured image is an image containing an intraoral tumor. When the captured image is determined to be an image containing an oral tumor by the second discrimination unit and the first discrimination unit, or when the captured image is an image containing an oral tumor by the second discrimination unit. If discriminated, a captured image is input to the trained third discrimination model, and the input captured image includes an image containing a benign oral tumor, an image containing a malignant oral tumor, and an oral tumor. The third discrimination unit that determines which of the images does not contain the tumor, and the first discrimination unit determine that the captured image is an image that does not contain an oral tumor, and the second discrimination unit determines that the captured image is an oral tumor. If it is determined that the image does not contain an oral tumor, or if the captured image is determined to be an image that does not contain an oral tumor by the third discrimination unit, it is determined that there is no oral tumor in the patient's oral cavity. When the health information shown is output and the captured image is determined by the third discrimination unit to be an image containing a benign oral tumor, the benign tumor information indicating the presence of a benign oral tumor in the patient's oral cavity. Is output, and when the captured image is determined to be an image containing a malignant oral tumor by the third discrimination unit, malignant tumor information indicating the presence of a malignant oral tumor in the patient's oral cavity is output. It has an output unit.

Further, in the disclosed discrimination device, the second discrimination unit refers to the trained second discrimination model only when the captured image is determined by the first discrimination unit to be an image that does not include an intraoral tumor. It is preferable to input a captured image and determine whether or not the captured image is an image containing an oral tumor.

Further, it is preferable that the disclosed discrimination device has a display unit that displays each information output by the output unit.

Further, the disclosed discrimination device uses the spatial frequency spectrum as input data, and uses the data indicating whether or not the captured image corresponding to the spatial frequency spectrum is an image including an oral tumor as output data, and uses the first discrimination model. The training is performed, the captured image is used as input data, the data indicating whether or not the captured image is an image containing an intraoral tumor is used as output data, the second discrimination model is trained, the captured image is used as input data, and the captured image is , An image containing a benign oral tumor, an image containing a malignant oral tumor, and an image not containing an oral tumor, as output data, and training to train the third discrimination model. It is preferable to have a portion.

Further, in the disclosed discrimination device, the input data used for learning the second discrimination model and the third discrimination model is a divided image obtained by dividing the captured image into a plurality of captured images, and an image obtained by rotating the divided image left and right. , And one or more images selected from the group consisting of left-right inverted divided images.

The disclosed discrimination terminal device is a discrimination terminal device that discriminates the presence or absence of an oral tumor and discriminates whether the oral tumor is benign or malignant by using a photographed image of the oral cavity of a patient irradiated with blue light. , The irradiation unit that irradiates blue light, the imaging unit that captures the imaging target space in a predetermined imaging direction, the conversion unit that generates a spatial frequency spectrum by Fourier transforming the captured image, and the learned first discrimination. A first discriminant unit that inputs a spatial frequency spectrum to the model and determines whether or not the captured image corresponding to the input spatial frequency spectrum is an image containing an intraoral tumor, and a trained second discriminant model. A second discriminant unit that inputs a captured image to the subject and determines whether or not the input captured image is an image containing an oral tumor, and an image captured by the first discriminant unit that includes an oral tumor. If it is determined to be present, or if the captured image is determined to be an image containing an intraoral tumor by the second discrimination unit, the captured image is input to the trained third discrimination model and input. Photographed by a third discriminating unit and a first discriminating unit that determine whether the captured image is an image containing a benign oral tumor, an image containing a malignant oral tumor, or an image not containing an oral tumor. When the image is determined to be an image that does not contain an oral tumor and the captured image is determined to be an image that does not contain an oral tumor by the second discrimination unit, or the captured image is an intraoral image by the third discrimination unit. When it is determined that the image does not contain a tumor, health information indicating that an oral tumor does not exist in the patient's oral cavity is output, and the image taken by the third discrimination unit is an image containing a benign oral tumor. If it is determined to be present, benign tumor information indicating that a benign oral tumor is present in the patient's oral cavity is output, and the image taken by the third discrimination unit is determined to be an image containing a malignant oral tumor. If so, it has an output unit that outputs malignant tumor information indicating that a malignant oral tumor is present in the oral cavity of the patient.

The disclosed control method is a control method of a discrimination device that discriminates the presence or absence of an oral tumor and the benign or malignant of an oral tumor by using a photographed image of the oral cavity of a patient irradiated with blue light. Then, a spatial frequency spectrum is generated by Fourier transforming the captured image, the spatial frequency spectrum is input to the trained first discrimination model, and the captured image corresponding to the input spatial frequency spectrum is an oral tumor. The first discrimination step for determining whether or not the image contains the tumor, and the captured image is input to the trained second discrimination model, and whether or not the input captured image is an image containing an intraoral tumor. When the captured image is determined to be an image containing an oral tumor in the second discrimination step and the first discrimination step, or when the captured image is an image containing an oral tumor in the second discrimination step. If discriminated, a captured image is input to the trained third discrimination model, and the input captured image includes an image containing a benign oral tumor, an image containing a malignant oral tumor, and an oral tumor. In the third discrimination step for determining which of the images does not contain, and in the first discrimination step, the captured image is determined to be an image that does not contain an oral tumor, and in the second discrimination step, the captured image is an oral tumor. If it is determined that the image does not contain an oral tumor, or if the captured image is determined to be an image that does not contain an oral tumor in the third discrimination step, it is determined that there is no oral tumor in the patient's oral cavity. If the captured image is determined to be an image containing a benign oral tumor in the third discrimination step, the benign tumor information indicating that a benign oral tumor is present in the patient's oral cavity is output. Is output, and when the captured image is determined to be an image containing a malignant oral tumor in the third discrimination step, malignant tumor information indicating the presence of a malignant oral tumor in the patient's oral cavity is output. Includes output steps.

The disclosed control program is a control program of a discrimination device that discriminates the presence or absence of an oral tumor and the benign or malignant of an oral tumor by using a photographed image of the oral cavity of a patient irradiated with blue light. Then, a conversion function that generates a spatial frequency spectrum by Fourier transforming the captured image and a captured image that corresponds to the input spatial frequency spectrum by inputting the spatial frequency spectrum to the trained first discrimination model. The first discrimination function for discriminating whether or not the image contains an oral tumor and the captured image are input to the trained second discrimination model, and the input captured image is an image containing an oral tumor. When the second discrimination function for discriminating whether or not the image is determined by the first discrimination function and the captured image is an image containing an oral tumor, or when the second discrimination function determines that the captured image contains an oral tumor. If it is determined to be, an image taken is input to the trained third discrimination model, and the input image is an image containing a benign oral tumor, an image containing a malignant oral tumor, and an image containing a malignant oral tumor. The third discrimination function that discriminates which of the images does not contain the oral tumor and the first discrimination function determine that the captured image is an image that does not contain the oral tumor, and the second discrimination function captures the captured image. If it is determined that the image does not contain an oral tumor, or if the captured image is determined to be an image that does not contain an oral tumor by the third discrimination function, an oral tumor is present in the patient's oral cavity. If the image taken by the third discrimination function is determined to be an image containing a benign oral tumor, it indicates that a benign oral tumor is present in the patient's oral cavity. When benign tumor information is output and the captured image is determined to be an image containing a malignant oral tumor by the third discrimination function, malignant tumor information indicating the presence of a malignant oral tumor in the patient's oral cavity. The discriminator realizes the output function to output.

The disclosed discrimination system includes a terminal device that photographs the oral cavity of a patient irradiated with blue light and outputs the captured image, and a discrimination device that determines the presence or absence of an oral tumor using the captured image. The terminal device has an irradiation unit that irradiates blue light, a shooting unit that shoots a shooting target space in a predetermined shooting direction, and a terminal output unit that outputs a shot image. The discriminator inputs the spatial frequency spectrum to the first discriminant model that has been trained and the conversion unit that generates the spatial frequency spectrum by Fourier transforming the captured image, and corresponds to the input spatial frequency spectrum. The captured image is input to the first discrimination unit that determines whether or not the captured image is an image containing an oral tumor and the trained second discrimination model, and the input captured image contains an oral tumor. When the second discrimination unit that determines whether or not the image is an image and the first discrimination unit determine that the captured image is an image containing an oral tumor, or the second discrimination unit determines that the captured image is an oral tumor. When it is determined that the image contains a benign oral tumor, a captured image is input to the trained third discrimination model, and the input captured image includes an image containing a benign oral tumor and a malignant oral tumor. The third discrimination unit that determines whether the image is an image or an image that does not contain an oral tumor, and the first discrimination unit determine that the captured image is an image that does not contain an oral tumor, and the second discrimination unit determines. If the captured image is determined to be an image that does not contain an oral tumor, or if the captured image is determined to be an image that does not contain an oral tumor by the third discrimination unit, the patient's oral cavity is intraoral. When the healthy information indicating that the tumor does not exist is output and the captured image is determined by the third discrimination unit to be an image containing a benign oral tumor, a benign oral tumor is present in the patient's oral cavity. When the image taken by the third discriminator is determined to be an image containing a malignant oral tumor, it indicates that a malignant oral tumor is present in the patient's oral cavity. It has an output unit that outputs malignant tumor information.

It is possible to effectively apply the knowledge of a skilled specialist to the discrimination process and realize highly accurate discrimination of oral tumors by using the discrimination device, discrimination terminal device, control method and control program of the discrimination device, and discrimination system. Will be.

It is a figure which shows an example of the outline structure of the discrimination system. It is a figure which shows an example of the schematic structure of a terminal device. (A) is a perspective view showing an example of a terminal device, and (b) is a diagram showing an example of how to use the terminal device. It is a figure which shows an example of the schematic structure of the discrimination device. (A) and (b) are diagrams showing an example of a blue light irradiation image, and (c) and (d) are diagrams showing a spectrum image for explaining an example of a spatial frequency spectrum. (A) is a schematic diagram for explaining an example of input data of teacher data, and (b) is a schematic diagram showing an example of a first discrimination model. (A) is a schematic diagram showing an example of a second discrimination model, and (b) is a schematic diagram showing an example of a third discrimination model. It is a schematic diagram for demonstrating the outline of the discrimination process. It is a figure which shows an example of the operation flow of the discrimination process. It is a figure which shows an example of the schematic structure of the discrimination terminal device.

Hereinafter, various embodiments of the present invention will be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to those embodiments and extends to the inventions described in the claims and their equivalents.

(Overview of discrimination system 1)
FIG. 1 is a diagram showing an example of a schematic configuration of the discrimination system 1. The discrimination system 1 photographs the oral cavity of a patient irradiated with blue light, and uses the captured images to determine the presence or absence of an oral tumor and to determine whether the oral tumor is benign or malignant. The discrimination system 1 includes a terminal device 2 operated by a medical worker such as a doctor or a nurse, and a discrimination device 3 for discriminating the presence or absence of a tumor in the oral cavity of a patient and discriminating between benign and malignant oral tumors. To prepare for. Hereinafter, a medical worker who operates the terminal device 2 may be referred to as a “user”.

The oral cavity is an organ extending from the mouth fissure to the pharynx, and is composed of parts such as the tongue, the floor of the mouth, the cheeks, the palate, the lips, the gums, and the salivary glands. Oral tumors (oral tumors) are tumors that occur in the oral cavity and are classified into malignant oral tumors and benign oral tumors. Most malignant oral tumors are squamous cell carcinoma (SCC) that develops from the epithelium of the mucous membrane, and are called tongue cancer, gingival cancer, oral floor cancer, etc., depending on the site of occurrence. A benign oral tumor is a mass of tissue in which cells are abnormally proliferated like a malignant oral tumor, but is a tumor that does not cause so-called "infiltration and metastasis" and "cachexia".

The terminal device 2 is an intraoral fluorescence observation device having a blue light irradiation function and an imaging function. The blue light irradiation function of the terminal device 2 irradiates blue light having a wavelength of 435 to 460 nm. The photographing function of the terminal device 2 photographs the space to be photographed in the oral cavity irradiated with blue light, and the photographed image is output as a blue light irradiation image.

The invasion depth of the irradiated blue light is about 400 to 600 μm, and fluorescence visualization (FVR) is maintained in healthy tissues. Therefore, in the blue light irradiation image, the pixel corresponding to the healthy tissue has blue-green (Apple-Green) color information. On the other hand, in a pathological tissue such as SCC, a decrease in fluorescence visibility occurs (Fluorescence Visualization Loss: FVL), so that the pixel corresponding to the pathological tissue has dark color information in the blue light irradiation image.

The terminal device 2 may be a combination of a mobile terminal such as a multifunctional mobile phone (so-called “smartphone”), a tablet terminal, a tablet PC (Personal Computer), a wearable computer, and a fluorescence observation device. In this case, the fluorescence observation device does not have a photographing function, and the mobile terminal captures a blue light irradiation image in the oral cavity irradiated with blue light by the fluorescence observation device. For example, as the terminal device 2, a fluorescence observation device having a storage unit capable of accommodating a smartphone, such as "Velscope (registered trademark)" or "Auralook (registered trademark)", may be used.

The discrimination device 3 is an information processing device such as a computer or a server. The discrimination device 3 determines whether the blue light irradiation image taken by the terminal device 2 is an image containing a benign oral tumor, an image containing a malignant oral tumor, or an image not containing an oral tumor. It has a discriminating function for discriminating.

Although one discriminating device 3 is shown as a component of the discriminating system 1 in FIG. 1, the discriminating device 3 may be a set of a plurality of physically separate computers. In this case, each of the plurality of computers may have the same function, or may have the functions of one discriminating device 3 in a distributed manner.

(Terminal device 2)
FIG. 2 is a diagram showing an example of a schematic configuration of the terminal device 2.

The terminal device 2 has at least a function of irradiating blue light toward a certain range in a predetermined irradiation direction and a function of photographing a shooting target space in a shooting direction substantially the same as the predetermined irradiation direction. Further, the terminal device 2 has a function of transmitting the image data of the blue light irradiation image to the discrimination device 3 by wireless communication. Therefore, the terminal device 2 includes a terminal communication unit 21, a terminal storage unit 22, a terminal display unit 23, a terminal operation unit 24, an irradiation unit 25, a photographing unit 26, and a terminal control unit 27.

The terminal communication unit 21 performs wireless communication with an access point of a wireless LAN (Local Area Network) (not shown) based on the wireless communication method of the IEEE (The Institute of Electrical and Electronics Engineers, Inc.) 802.11 standard. It is equipped with a communication interface circuit. The terminal communication unit 21 transmits the image data of the blue light irradiation image acquired from the terminal control unit 27 to the discrimination device 3 by wireless communication. Further, the terminal communication unit 21 establishes a wireless signal line with an LTE (Long Term Evolution) system, a fifth generation (5G) mobile communication system, or the like with a base station via a channel assigned by a base station (not shown). However, communication may be performed with the base station. Further, the terminal communication unit 21 has an interface circuit for performing short-range wireless communication according to a communication method such as Bluetooth (registered trademark), and may transmit radio waves to the discrimination device 3. Further, the terminal communication unit 21 may have a transmission circuit for transmitting various signals corresponding to the image data of the blue light irradiation image by infrared communication or the like. Further, the terminal communication unit 21 may include a communication interface circuit of a wired LAN. As a result, the terminal device 2 can provide the blue light irradiation image to the discrimination device 3 via the terminal communication unit 21.

The terminal device 2 may include an input / output device that detachably holds the portable storage medium in place of the terminal communication unit 21 or together with the terminal communication unit 21. In this case, the input / output device stores the image data of the blue light irradiation image acquired from the terminal control unit 27 in the portable storage medium. Thereby, the terminal device 2 can provide the blue light irradiation image to the discrimination device 3 via the portable storage medium.

The terminal storage unit 22 is, for example, a semiconductor memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The terminal storage unit 22 stores operating system programs, driver programs, application programs, data, and the like used for processing in the terminal control unit 27. The driver programs stored in the terminal storage unit 22 include a communication device driver program that controls the terminal communication unit 21, an output device driver program that controls the terminal display unit 23, an input device driver program that controls the terminal operation unit 24, and an irradiation unit. An irradiation device driver program that controls 25, a photographing device driver program that controls the photographing unit 26, and the like. The application program stored in the terminal storage unit 22 operates the irradiation unit 25 based on an instruction input in response to the operation of the terminal operation unit 24 by the user, and blue light is directed toward a certain range in a predetermined irradiation direction. The terminal control unit performs irradiation processing for irradiating the device, imaging processing for operating the photographing unit 26 to acquire image data of the blue light irradiation image, transmission processing for transmitting the acquired image data of the blue light irradiation image to the discrimination device 3, and the like. It is a control program or the like for making 27 execute. Further, the terminal storage unit 22 may temporarily store data related to a predetermined process.

The terminal display unit 23 is a liquid crystal display. The terminal display unit 23 may be an organic EL (Electro-Luminescence) display or the like. The terminal display unit 23 displays a captured blue light irradiation image or the like.

The terminal operation unit 24 is a pointing device such as a touch panel and a mouse. The terminal operation unit 24 may be an input key or the like. The touch panel may include a capacitive proximity sensor and may be configured to be capable of detecting a user's non-contact operation. The user can use the terminal operation unit 24 to input characters, numbers and symbols, or the position of the terminal display unit 23 on the display screen. When the terminal operation unit 24 is operated by the user, the terminal operation unit 24 generates a signal corresponding to the operation. Then, the terminal operation unit 24 supplies the generated signal to the terminal control unit 27 as a user's instruction.

The irradiation unit 25 is composed of a semiconductor laser such as a laser diode that emits narrow-band light. The irradiation unit 25 irradiates and outputs narrow-band blue light (for example, light having a wavelength in the wavelength band of 435 to 460 nm) as excitation light.

The photographing unit 26 has an optical lens, an image pickup device, and the like. The optical lens forms a luminous flux from the subject on the image pickup surface of the image pickup device. The image pickup device is a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and outputs an image of a subject image formed on the image pickup surface. The photographing unit 26 creates and outputs still image data in a predetermined file format from the image generated by the image sensor. When the image data is output, the photographing unit 26 may associate the acquisition time of the image data with the image data. The photographing unit 26 may continuously create and output moving image data in a predetermined file format from the images generated by the image sensor at predetermined period intervals.

The terminal control unit 27 is a processing device that loads an operating system program, a driver program, and an application program stored in the terminal storage unit 22 into a memory and executes an instruction included in the loaded program. The terminal control unit 27 is, for example, an electronic circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or a combination of various electronic circuits. In FIG. 2, the terminal control unit 27 is shown as a single component, but the terminal control unit 27 may be a set of a plurality of physically separate processors.

The terminal control unit 27 functions as an irradiation control unit 271, an imaging control unit 272, and a transmission control unit 273 by executing various commands included in the control program. Hereinafter, an example of the functions of the irradiation control unit 271, the imaging control unit 272, and the transmission control unit 273 will be described with reference to FIG.

FIG. 3A is a perspective view showing an example of the terminal device 2.

The terminal device 2 shown in FIG. 3A has a substantially plate shape, and the irradiation unit 25 and the photographing unit 26 are installed on the surface of the terminal device 2 (for example, one surface of the widest pair of surfaces). The irradiation direction of the irradiation unit 25 and the imaging direction of the imaging unit 26 are both substantially the same as the vertical direction of the surface. As a result, at least a part of the irradiation space of the blue light irradiated by the irradiation unit 25 is included in the imaging target space of the imaging unit 26.

FIG. 3B is a diagram showing an example of how to use the terminal device. The terminal display unit 23 is installed on the back surface of the terminal device 2 (for example, the other surface of the widest pair of surfaces).

The irradiation control unit 271 controls the irradiation unit 25 so as to irradiate blue light toward a certain range in a predetermined irradiation direction when an irradiation instruction is input in response to an operation of the terminal operation unit 24 by the user.

When a shooting instruction is input according to the operation of the terminal operation unit 24 by the user, the shooting unit 26 is substantially the same as the irradiation direction every predetermined time (for example, 1/30 second, 1/15 second, or 1 second). Shooting in the direction Shoot the shooting target space in the direction. The shooting control unit 272 displays the shot image taken on the terminal display unit 23 each time the shooting unit 26 takes a picture.

In the example shown in FIG. 3B, the surface of the terminal device 2 is directed by the user towards the oral cavity of patient P. As a result, the irradiation direction of the blue light emitted by the irradiation unit 25 becomes the direction of the oral cavity of the patient P. The irradiation range R1 shown in FIG. 3B is an example of the irradiation range of the blue light irradiated to the patient P.

Further, in the example shown in FIG. 3B, the imaging direction of the imaging unit 26 is substantially the same as the irradiation direction of the irradiation unit 25, so that the imaging direction of the imaging unit 26 is the direction of the oral cavity. The imaging range R2 shown in FIG. 3B is an example of an imaging range for patient P. On the terminal display unit 23, the captured images corresponding to the imaging range R2 are displayed at predetermined time intervals. This allows the user to adjust the orientation of the terminal device 2 while checking the shooting range R2 currently being shot by the shooting unit 26 in real time.

When an acquisition instruction is input in response to an operation of the terminal operation unit 24 by the user, the imaging control unit 272 acquires the captured image captured at the timing when the acquisition instruction is input as a blue light irradiation image, and blue light. The image data of the irradiation image is stored in the terminal storage unit 22. The imaging control unit 272 may acquire an image in at least a part of the captured image as a blue light irradiation image in the diagnostic range. In the acquisition process of the blue light irradiation image of the diagnostic range by the imaging control unit 272, for example, among the captured images, the image corresponding to the diagnostic range specified by the operation of the terminal operation unit 24 by the user is the diagnostic range. It is a process acquired as a blue light irradiation image of. Further, the acquisition process of the blue light irradiation image of the diagnostic range by the imaging control unit 272 corresponds to the diagnostic range automatically set by using the known image processing for extracting the detection area from the captured images. The image to be processed may be a process of being acquired as a blue light irradiation image in the diagnostic range. Further, the terminal device 2 may be provided with an operation means (for example, a switching button displayed on the touch panel) for switching between the manual diagnosis range setting and the automatic diagnosis range setting. The "blue light irradiation image" described below includes a blue light irradiation image in the diagnostic range (an image in at least a part of the captured images taken in the diagnostic range).

The photographing control unit 272 may display the acquired blue light irradiation image on the terminal display unit 23 before and / or after the storage of the image data of the blue light irradiation image.

When a transmission instruction is input in response to an operation of the terminal operation unit 24 by the user, the transmission control unit 273 acquires the image data of the blue light irradiation image stored in the terminal storage unit 22 and via the terminal communication unit 21. Is transmitted to the discrimination device 3.

The shape and size of the terminal device 2 is not limited to the example shown in FIG. For example, the terminal device 2 may have a shape and size that can be inserted into the oral cavity of patient P.

(Discrimination device 3)
FIG. 4 is a diagram showing an example of a schematic configuration of the discrimination device 3.

The discrimination device 3 has a reception function for receiving image data of the blue light irradiation image transmitted from the terminal device 2 and a conversion function for converting the blue light irradiation image into a spatial frequency spectrum. Further, the discrimination device 3 outputs a learning function for learning the discrimination model, a discrimination function for discriminating the presence or absence of the oral tumor and the benign / malignant of the oral tumor using the trained discrimination model, and an output of the discrimination result by the discrimination function. It has a function and the like. Therefore, the discrimination device 3 includes a communication unit 31, a storage unit 32, a display unit 33, an operation unit 34, and a processing unit 35.

The communication unit 31 is implemented as hardware, firmware, communication software such as a TCP / IP driver or a PPP driver, or a combination thereof. The discrimination device 3 can receive the image data of the blue light irradiation image from the terminal device 2 via the communication unit 31. Further, the communication unit 31 has an interface circuit for performing short-range wireless communication according to a communication method such as Bluetooth (registered trademark), and may receive radio waves from the terminal device 2. Further, the communication unit 31 may have a reception circuit for receiving various signals corresponding to the image data of the blue light irradiation image by infrared communication or the like. Further, the communication unit 31 may include a communication interface circuit of a wired LAN. As a result, the discrimination device 3 can acquire the blue light irradiation image from the terminal device 2 via the communication unit 31.

The discrimination device 3 may include an input / output device that detachably holds the portable storage medium in place of the communication unit 31 or together with the communication unit 31. In this case, the input / output device acquires the image data of the blue light irradiation image stored in the portable storage medium, and supplies the acquired image data to the processing unit 35. As a result, the discrimination device 3 can acquire the blue light irradiation image from the terminal device 2 via the portable storage medium.

The storage unit 32 is, for example, a semiconductor memory device such as a ROM or RAM. The storage unit 32 may be, for example, a magnetic disk, an optical disk, or various storage devices other than those capable of storing data. The storage unit 32 stores an operating system program, a driver program, an application program, data, and the like used for processing in the processing unit 35. The computer program stored in the storage unit 32 may be installed in the storage unit 32 from a computer-readable portable recording medium such as a CD-ROM or a DVD-ROM using a known setup program or the like.

The data stored in the storage unit 32 are the first discrimination model M1, the second discrimination model M2, the third discrimination model M3, the teacher data TD, and the like, which will be described later. Further, the storage unit 32 stores the image data of the blue light irradiation image acquired from the terminal device 2. The storage unit 32 may temporarily store data related to a predetermined process.

The display unit 33 is a liquid crystal display. The display unit 33 may be an organic EL display or the like. The display unit 33 displays a blue light irradiation image based on the image data supplied from the processing unit 35, result information of the discrimination process, and the like.

The operation unit 34 is a keyboard and / or a mouse. The operation unit 34 may be a pointing device such as a touch panel. The user can use the operation unit 34 to input characters, numbers, symbols, or the position of the display unit 33 on the display screen. When the operation unit 34 is operated by the user, the operation unit 34 generates a signal corresponding to the operation. Then, the generated signal is supplied to the processing unit 35 as a user's instruction.

The processing unit 35 is a processing device that loads an operating system program, a driver program, and an application program stored in the storage unit 32 into a memory and executes an instruction included in the loaded program. The processing unit 35 is, for example, an electronic circuit such as a CPU, MPU, DSP, or a combination of various electronic circuits. In FIG. 4, the processing unit 35 is shown as a single component, but the processing unit 35 may be a set of a plurality of physically separate processors.

By executing various instructions included in the application program (control program) stored in the storage unit 32, the processing unit 35 includes a reception processing unit 351 and a conversion processing unit 352, a learning processing unit 353, a discrimination processing unit 354, and a discrimination processing unit 354. It functions as an output processing unit 335. Hereinafter, an example of the functions of the reception processing unit 351, the conversion processing unit 352, the learning processing unit 353, the discrimination processing unit 354, and the output processing unit 335 will be described with reference to FIGS. 5-8.

The reception processing unit 351 receives the image data of the blue light irradiation image transmitted from the terminal device 2 via the communication unit 31, and stores the received image data of the blue light irradiation image in the storage unit 32.

5 (a) and 5 (b) are diagrams showing an example of a blue light irradiation image.

The blue light irradiation image shown in FIG. 5 (a) is an example of an image containing no oral tumor (an image containing only healthy tissue), and the blue light irradiation image shown in FIG. 5 (b) is malignant. It is an image including an intraoral tumor.

The learning processing unit 353 generates and generates a first discrimination model M1, a second discrimination model M2, and a third discrimination model M3 trained using each of the three types of teacher data TD stored in the storage unit 32. Each discriminant model M1-M3 is stored in the storage unit 32.

The first teacher data TD1 of the three types of teacher data TD uses the spatial frequency spectrum generated by Fourier transforming the blue light irradiation image as input data, and "the blue light irradiation image is an image including an oral tumor. It is teacher data using label data indicating either "there is" or "the blue light irradiation image is an image that does not include an oral tumor" as output data. Further, the second teacher data TD2 of the three types of teacher data TD uses the image data of the blue light irradiation image as input data, and "the blue light irradiation image is an image including an oral tumor" and "blue light". It is teacher data that uses label data indicating either one of "the irradiated image is an image that does not include an intraoral tumor" as output data. Further, the third teacher data TD3 of the three types of teacher data TD uses the image data of the blue light irradiation image as input data, and "the blue light irradiation image is an image including a malignant oral tumor", " With teacher data using label data as output data indicating either "the blue light irradiation image is an image containing a benign oral tumor" or "the blue light irradiation image is an image not containing an oral tumor". be.

The second teacher data TD2 and the third teacher data TD3 are data generated by a user, an administrator of the discriminating device 3, or the like. The spatial frequency spectrum, which is the input data of the first teacher data TD1, is generated by the conversion processing unit 352. Hereinafter, an example of the conversion process of the conversion process unit 352 will be described.

The conversion processing unit 352 extracts the second teacher data TD2 stored in the storage unit 32. Next, the conversion processing unit 352 performs a known Fourier transform represented by the following equation (1) on the image data of the blue light irradiation image which is the input data of the extracted second teacher data TD2, and then the equation. A spatial frequency spectrum is generated by (2). The generated spatial frequency spectrum is a component amount p (r) corresponding to the distance r from the center of the image data. Then, the conversion processing unit 352 generates the first teacher data TD1 whose input data is the generated spatial frequency spectrum data and whose output data is the label data of the extracted second teacher data TD2, and stores it in the storage unit 32. do.

5 (c) and 5 (d) are diagrams showing spectrum images for explaining an example of a spatial frequency spectrum.

The spectral image shown in FIG. 5 (c) is an F (u, v) calculated by a known Fourier transform represented by the formula (1) from the image data of the blue light irradiation image shown in FIG. 5 (a). ) Is an example diagram for explaining an image showing a power spectrum. Further, in the spectral image shown in FIG. 5 (d), the image data of the blue light irradiation image shown in FIG. 5 (b) is calculated by the known Fourier transform represented by the equation (1). , V) is an example diagram for explaining an image showing a power spectrum.

The spectral image shows a spatial frequency spectrum in which the vicinity of the center of the image is a low frequency component and the farther away from the center is the higher frequency component. For example, as shown in FIG. 5A, when the blue light irradiation image is an image that does not contain an oral tumor (an image that contains only healthy tissue), the mucosal surface is smooth in healthy tissue (healthy mucosa). , The change in brightness (pixel value) with respect to the change in pixel position is small. Therefore, the spectral image related to F (u, v) calculated by Fourier transforming the image data of the blue light irradiation image shown in FIG. 5A is a spectral image containing a large amount of low frequency components (FIG. 5). (C)).

On the other hand, as shown in FIG. 5B, when the blue light irradiation image is an image including an oral tumor, the oral tumor has irregularities on the mucosal surface, and the brightness (pixels) with respect to the change in the position of the pixels. Value) changes greatly. Therefore, the spectral image related to F (u, v) calculated by Fourier transforming the image data of the blue light irradiation image shown in FIG. 5 (b) is the spectral image shown in FIG. 5 (c). In comparison, a spectral image (FIG. 5 (d)) containing a higher frequency pixel value fluctuation component is obtained.

FIG. 6A is a schematic diagram for explaining an example of input data of the second teacher data TD2 and the third teacher data TD3.

The image data of the blue light irradiation image shown in FIG. 6A is an image having 1,600 pixels in length and width. The image size of the image data of the blue light irradiation image is not limited to that shown in FIG. 6A, and may be any size. The learning processing unit 353 divides the image data of the blue light irradiation image into 2,500 image data having a length and width of 32 pixels. Then, the learning processing unit 353 uses the divided image data as input data, and "the blue light irradiation image is an image containing an oral tumor" and "the blue light irradiation image is an image not including an oral tumor". The second teacher data TD2 is generated with the label data indicating one of the above as the output data. That is, 2,500 teacher data are generated from the image data of the blue light irradiation image shown in FIG. 6 (a). In addition, the learning processing unit 353 has a left-rotated image (2,500), a right-rotated image (2,500), and an upside-down image (upside-down image) for each of the image data of the 2,500 blue light irradiation images. 2,500) and left-right inverted images (2,500) may be generated, and the image data of a total of 12,500 of these images may be used as teacher data. The learning processing unit 353 has 2,500 blue light irradiation images, a left-handed image (2,500), a right-handed image (2,500), an upside-down image (2,500), and a left-right inverted image. Image data of one or more types of images selected from the group consisting of (2,500) may be used as teacher data. For example, image data of a total of 5,000 images, that is, 2,500 blue light irradiation images and left-rotated images (2,500) may be used as teacher data. As described above, the learning processing unit 353 makes it possible to increase the number of teacher data even when it is difficult to obtain a blue light irradiation image including an intraoral tumor.

Similarly, the learning processing unit 353 uses the divided image data as input data, and "the blue light irradiation image is an image containing a malignant oral tumor" and "the blue light irradiation image is a benign oral tumor". The third teacher data TD3 is generated using the label data indicating either "the image is an image including" or "the image irradiated with blue light is an image not including an oral tumor" as output data.

Next, the learning process by the learning process unit 353 will be described.

FIG. 6B is a schematic diagram showing an example of the first discrimination model. The first discrimination model M1 shown in FIG. 6B is a neural network model having an input layer M1-1, a hidden layer M1-2, and an output layer M1-3. Each of the so-called "neurons" of the input layer M1-1 is input with a component amount p (r) corresponding to each r of the spatial frequency spectrum of the first teacher data TD1. A spatial frequency spectrum is input to each neuron in the input layer M1-1. In addition, among the neurons of the input layer M1-1, there may be a neuron that does not correspond to r of the spatial frequency spectrum, and in this case, the neuron functions as a bias.

The number of neurons in the hidden layer M1-2 may be greater or less than the number of neurons in the input layer M1-1. In addition, the two neurons contained in the output layer M1-3 have label data indicating that "the blue light irradiation image is an image containing an oral tumor" and "an image in which the blue light irradiation image does not contain an oral tumor". Corresponds to the label data indicating "is".

FIG. 7A is a schematic diagram showing an example of the second discrimination model. The second discrimination model M2 shown in FIG. 7A is a deep neural network model including a convolutional neural network having an input layer M2-1, a convolutional layer M2-2, a pooling layer M2-3, and an output layer M2-4. Is. The number of the convolutional layer M2-2 and the pooling layer M2-3 may be two or more, respectively. The neurons of the input layer M2-1 correspond to the color information (R value, G value, B value) corresponding to each pixel of the image data of the (divided) blue light irradiation image of the second teacher data TD2. The two neurons contained in the output layer M2-4 are label data indicating that "the blue light irradiation image is an image containing an oral tumor" and "the blue light irradiation image is an image not containing an oral tumor", respectively. Corresponds to the label data indicating "that".

FIG. 7B is a schematic diagram showing an example of the third discrimination model. The third discrimination model M3 shown in FIG. 7B is a deep neural network model including a convolutional neural network having an input layer M3-1, a convolutional layer M3-2, a pooling layer M3-3, and an output layer M3-4. Is. The input layer M3-1, the convolution layer M3-2, and the pooling layer M3-3 are the same as the input layer M2-1, the convolution layer M2-2, and the pooling layer M2-3. The three neurons contained in the output layer M3-4 each include label data indicating that "the blue light irradiation image is an image containing a malignant oral tumor" and "the blue light irradiation image contains a benign oral tumor". It corresponds to the label data indicating that it is an image and the label data indicating that the blue light irradiation image is an image that does not include an intraoral tumor.

The learning processing unit 353 generates or updates a first discriminant model in which the weights of each neuron in the neural network are learned by executing known machine learning learning using the first teacher data TD1. Further, the learning processing unit 353 generates or updates a second discriminant model in which the weights of each neuron in the multi-layered neural network are learned by executing known deep learning learning using the second teacher data TD2. .. Further, the learning processing unit 353 generates or updates a third discrimination model in which the weights of each neuron in the multi-layered neural network are learned by executing known deep learning learning using the third teacher data TD3. ..

FIG. 8 is a schematic diagram for explaining the outline of the discrimination process. The discrimination process is executed by the conversion process unit 352, the discrimination process unit 354, and the output processing unit 355. In the discrimination process, when the image data of a new patient's blue light irradiation image (different from the teacher data TD) is acquired from the terminal device 2, the determination result (malignant tumor information, malignant tumor information, corresponding to the acquired blue light irradiation image, (Benign tumor information or healthy information) is output.

First, the conversion processing unit 352 generates a spatial frequency spectrum used in the first determination processing by Fourier transforming the image data of the acquired blue light irradiation image. The conversion processing unit 352 performs a first determination process by Fourier transforming the image data obtained by dividing the acquired blue light irradiation image into small sizes (32 pixels × 32 pixels) or the image data of the output image described later. The spatial frequency spectrum used may be generated.

Next, the discrimination processing unit 354 receives an image of the original size (acquired) from the image divided into small sizes (32 pixels × 32 pixels) (when the acquired blue light irradiation image is divided). (Blue light irradiation image) is reconstructed. The discrimination processing unit 354 generates a plurality of divided images including the pixels of any pixel in the oral cavity in the blue light irradiation image, and makes a determination by majority voting. The discrimination processing unit 354 outputs a final output image used in the second discrimination processing and the third discrimination processing, which will be described later, by performing the judgment processing by majority voting on all the pixels in the oral cavity region. The output image is a part of the blue light irradiation image to be discriminated.

Next, the discrimination processing unit 354 executes the first discrimination processing and the second discrimination processing. In the first discrimination process, the discrimination processing unit 354 calculates the probability of the label data estimated by using the data showing the generated spatial frequency spectrum as the input data by using the first discrimination model M1. Then, the discrimination processing unit 354 "is an image including the oral tumor in the blue light irradiation image (acquired blue light irradiation image) corresponding to the spatial frequency spectrum" or "the image including the oral tumor" based on the calculated probability of the label data. The blue light irradiation image corresponding to the spatial frequency spectrum is an image that does not include an intraoral tumor. " For example, the discrimination processing unit 354 calculates 82% as the probability of the label data indicating that "the blue light irradiation image is an image containing an oral tumor", and "the image that the blue light irradiation image does not include an oral tumor". When 18% is calculated as the probability of the label data indicating "is", it is determined that "the blue light irradiation image is an image including an oral tumor".

In the second discrimination process, the discrimination processing unit 354 uses the second discrimination model M2 to determine the probability of the label data estimated by using the image data of the output image (a part of the blue light irradiation image) as the input data. To calculate. Then, the discrimination processing unit 354 "is an image in which the blue light irradiation image contains an oral tumor" and "an image in which the blue light irradiation image does not include an oral tumor" based on the calculated probability of the label data. Either one is determined.

Next, the discrimination processing unit 354 determines whether or not the discrimination results of the first discrimination process and the second discrimination process are both "the blue light irradiation image is an image that does not include the intraoral tumor". When both the discrimination results of the first discrimination process and the second discrimination process are the results that "the blue light irradiation image is an image that does not include an intraoral tumor", the discrimination processing unit 354 outputs the discrimination result to the output processing unit 355. Send to. Then, when the output processing unit 355 acquires the result that the discrimination result of the first discrimination process and the second discrimination process is "the blue light irradiation image is an image not including the intraoral tumor", the output processing unit 355 outputs the health information. The healthy information is information indicating that there is no oral tumor in the oral cavity of the patient from which the acquired blue light irradiation image was taken. Further, the output of the health information is the display of the health information on the display unit 33 and / or the transmission of the health information to another device.

When the discrimination results of the first discrimination process and the second discrimination process are not the result that "the blue light irradiation image is an image that does not include the intraoral tumor" (the judgment result of the first discrimination process and the judgment result of the second discrimination process). When at least one of the above is the result that "the blue light irradiation image is an image including an oral tumor"), the discrimination processing unit 354 executes the third discrimination processing.

In the third discrimination process, the discrimination processing unit 354 uses the third discrimination model M3 to determine the probability of the label data estimated by using the image data of the output image (a part of the blue light irradiation image) as the input data. To calculate. Then, the discrimination processing unit 354 "is an image containing a malignant oral tumor in the blue light irradiation image" and "an image including a benign oral tumor in the blue light irradiation image" based on the calculated probability of the label data. It is determined whether "the image is an image that does not contain an oral tumor" or "the image irradiated with blue light is an image that does not include an oral tumor". For example, the discrimination processing unit 354 calculates that the probability of the label data indicating that "the blue light irradiation image is an image containing a malignant oral tumor" is 68%, and "the blue light irradiation image is a benign oral tumor". When the probability of the label data indicating "the image contains" is calculated to be 23%, and the probability of the label data indicating "the image irradiated with blue light is an image containing no oral tumor" is calculated to be 9%. , "The blue light irradiation image is an image containing a malignant oral tumor".

Next, the discrimination processing unit 354 transmits the discrimination result of the third discrimination processing to the output processing unit 355. When the output processing unit 355 obtains the result that the discrimination result of the third discrimination process is "the blue light irradiation image is an image including a malignant oral tumor", the output processing unit 355 outputs the malignant tumor information. When the output processing unit 355 obtains the result that the discrimination result of the third discrimination process is "the blue light irradiation image is an image including a benign oral tumor", the output processing unit 355 outputs the benign tumor information. When the output processing unit 355 obtains the result that the discrimination result of the third discrimination process is "the blue light irradiation image is an image that does not include the intraoral tumor", the output processing unit 355 outputs the healthy information. The malignant tumor information is information indicating that a malignant oral tumor is present in the oral cavity of the patient from which the acquired blue light irradiation image was taken. The benign tumor information is information indicating that a benign oral tumor is present in the oral cavity of the patient from which the acquired blue light irradiation image was taken. Further, the output of the malignant tumor information is the display of the malignant tumor information on the display unit 33 and / or the transmission of the malignant tumor information to another device, and the output of the benign tumor information is the benign tumor information to the display unit 33. Displaying tumor information and / or transmitting benign tumor information to other devices.

FIG. 9 is a diagram showing an example of an operation flow of discrimination processing by the conversion processing unit 352, the discrimination processing unit 354, and the output processing unit 355.

First, when the image data of the blue light irradiation image is acquired from the terminal device 2, the conversion processing unit 352 generates a spatial frequency spectrum by Fourier transforming the acquired image data of the blue light irradiation image (step). S101).

Next, the discrimination processing unit 354 executes the first discrimination processing (step S102).

Next, the discrimination processing unit 354 determines whether or not the discrimination result of the first discrimination process is the result that "the blue light irradiation image is an image including an oral tumor" (step S103).

When the discrimination result of the first discrimination process is the result that "the blue light irradiation image is an image including an intraoral tumor" (step S103-Yes), the discrimination processing unit 354 proceeds to the process to step S106.

When the discrimination result of the first discrimination process is the result that "the blue light irradiation image is an image that does not include the intraoral tumor" (step S103-No), the discrimination processing unit 354 executes the second discrimination process (step S103-No). Step S104)

Next, the discrimination processing unit 354 determines whether or not the discrimination result of the second discrimination process is the result that "the blue light irradiation image is an image including an oral tumor" (step S105).

When the discrimination result of the first discrimination process is the result that "the blue light irradiation image is an image including an intraoral tumor" (step S105-Yes), the discrimination processing unit 354 proceeds to the process in step S106.

When the discrimination result of the first discrimination process is the result that "the blue light irradiation image is an image that does not include the intraoral tumor" (step S105-No), the discrimination processing unit 354 proceeds to the process to step S111.

In step S106, the discrimination processing unit 354 executes the third discrimination processing.

Next, the discrimination processing unit 354 determines whether or not the discrimination result of the third discrimination process is the result that "the blue light irradiation image is an image including an oral tumor" (step S107).

When the discrimination result of the third discrimination process is the result that "the blue light irradiation image is an image that does not include the intraoral tumor" (step S107-No), the discrimination processing unit 354 proceeds to the process to step S111.

When the discrimination result of the third discrimination process is "the blue light irradiation image is an image containing an oral tumor" (step S107-Yes), the discrimination result of the third discrimination process is "the blue light irradiation image is malignant". It is determined whether or not the result is "an image including an intraoral tumor" (step S108).

When the discrimination result of the third discrimination process is the result that "the blue light irradiation image is an image including a malignant oral tumor" (step S108-Yes), the output processing unit 355 outputs the malignant tumor information (step S108-Yes). Step S109), the discrimination process is terminated.

When the discrimination result of the third discrimination process is the result that "the blue light irradiation image is an image including a benign oral tumor" (step S108-No), the output processing unit 355 outputs the benign tumor information (step S108-No). Step S110), the discrimination process is terminated.

In step S111, the output processing unit 355 outputs healthy information and ends the discrimination process.

As described in detail above, the discrimination system 1 of the present embodiment executes the second and third discrimination processes using the second and third discrimination models that have been trained using the blue light irradiation image as the teacher data, and also performs the oral cavity. The first discrimination process using the first discrimination model focusing on the difference in brightness in the blue light irradiation image between the mucosal surface of the tumor inside and the mucosal surface of the healthy tissue in the oral cavity is executed. As a result, in the discrimination system 1 of the present embodiment, the knowledge of a skilled specialist can be effectively applied to the discrimination process, and it becomes possible to realize highly accurate discrimination of oral tumors.

Further, in the discrimination system 1 of the present embodiment, the first, second, and third discrimination processes using the first, second, and third discrimination models, which are different from each other, are sequentially executed to obtain a blue light irradiation image. Are classified as malignant tumors, benign tumors and healthy tumors. Therefore, in the discrimination system 1 of the present embodiment, even if the number of teacher data of the second and third discrimination models is not sufficient, the discrimination processing with higher accuracy can be performed as compared with the case of classifying with only one type of discrimination model. It will be possible. Further, by using a plurality of types of image data processed (divided, rotated, and / or inverted) from one type of image data as the teacher data of the second and third discrimination models, a blue light irradiation image can be obtained. Even if it is difficult, it is possible to improve the accuracy of the discrimination process. In this way, the number of teacher data used in the discrimination model can be reduced, the processing load of the discrimination device 3 and / or the communication load between the terminal device 2 and the discrimination device 3 can be reduced, and the processing of the discrimination device 3 can be reduced. It is possible to realize highly accurate discrimination of oral tumors without consuming excessive resources.

(Modification 1)
The present invention is not limited to the present embodiment. For example, instead of using the discriminating device 3, a discriminating terminal device 4 having the discriminating device 3 having a conversion function, a learning function, a discriminating function, and an output function may be used.

FIG. 10 is a diagram showing an example of a schematic configuration of the discrimination terminal device 4. The discrimination terminal device 4 is a terminal device 2 having a conversion function, a learning function, a discrimination function, and an output function of the discrimination device 3. In FIG. 10, the same reference numerals are given to the same configurations as those in FIGS. 2 and 4, and the description thereof will be omitted.

The conversion processing unit 352, the learning processing unit 353, and the discrimination processing unit 354 of the discrimination terminal device 4 use the blue light irradiation image (blue light irradiation image stored by the photographing control unit 272) stored in the terminal storage unit 22. , Perform conversion processing, learning processing, and discrimination processing, respectively.

Further, the output processing unit 355 displays and outputs the discrimination result (healthy information, malignant tumor information, benign tumor information) to the terminal display unit 23.

As described above, in the discrimination terminal device 4, one device executes the shooting process of the blue light irradiation image, the discrimination process, and the display of the discrimination result, so that the data transmission process to other devices is unnecessary. At the same time, the user can confirm the discrimination result immediately after taking the blue light irradiation image. As described above, the discrimination terminal device 4 makes it possible to improve the discrimination accuracy of the oral tumor and shorten the time required for screening.

(Modification 2)
The first discrimination model used in the first discrimination process may be a support vector machine (SVM) or the like. In this case, the frequency spectrum is used as a feature amount, and the "blue light irradiation image" is classified into either "an image containing an oral tumor" or "an image not including an oral tumor".

1 Discrimination system 2 Terminal device 21 Terminal communication unit 22 Terminal storage unit 23 Terminal display unit 24 Terminal operation unit 25 Irradiation unit 26 Imaging unit 27 Terminal control unit 271 Irradiation control unit 272 Imaging control unit 273 Transmission control unit 3 Discrimination device 31 Communication unit 32 Storage unit 33 Display unit 34 Operation unit 35 Processing unit 351 Reception processing unit 352 Conversion processing unit 353 Learning processing unit 354 Discrimination processing unit 355 Output processing unit 4 Discrimination terminal device M1 First discrimination model M2 Second discrimination model M3 Third discrimination Model TD teacher data

Claims (9)

It is a discrimination device that discriminates the presence or absence of an oral tumor and the benign or malignant of an oral tumor using a photographed image of the oral cavity of a patient irradiated with blue light.
A conversion unit that generates a spatial frequency spectrum by Fourier transforming the captured image, and
The first discrimination model in which the spatial frequency spectrum is input to the trained first discrimination model and whether or not the captured image corresponding to the input spatial frequency spectrum is an image including the oral tumor is determined. Department and
A second discrimination unit that inputs the captured image to the trained second discrimination model and determines whether or not the input captured image is an image including the oral tumor.
When the captured image is determined to be an image containing the oral tumor by the first discrimination unit, or when the captured image is determined to be an image containing the oral tumor by the second discrimination unit. In the case, the captured image is input to the trained third discrimination model, and the input captured image includes the benign oral tumor, the malignant oral tumor, and the oral cavity. A third discriminating unit that discriminates which of the images does not contain an internal tumor,
When the captured image is determined to be an image that does not contain the oral tumor by the first discrimination unit, and the captured image is determined to be an image that does not contain the oral tumor by the second discrimination unit. Alternatively, when the captured image is determined by the third discrimination unit to be an image that does not include the oral tumor, health information indicating that the oral tumor does not exist in the oral cavity of the patient is output.
When the captured image is determined by the third discrimination unit to be an image containing the benign oral tumor, benign tumor information indicating that the benign oral tumor is present in the oral cavity of the patient is output. death,
When the photographed image is determined by the third discrimination unit to be an image containing the malignant oral tumor, malignant tumor information indicating that the malignant oral tumor is present in the oral cavity of the patient is output. A discriminant device characterized by having an output unit and an output unit. The second discrimination unit inputs the captured image to the trained second discrimination model only when the captured image is determined by the first discrimination unit to be an image that does not include the intraoral tumor. The discriminating device according to claim 1, wherein the captured image is discriminated as to whether or not the image includes the intraoral tumor. The discrimination device according to claim 1 or 2, further comprising a display unit that displays each information output by the output unit. The first discrimination model is trained using the spatial frequency spectrum as input data and data indicating whether or not the captured image corresponding to the spatial frequency spectrum is an image containing the oral tumor as output data.
The second discriminant model is trained using the captured image as input data and data indicating whether or not the captured image is an image containing the oral tumor as output data.
Using the captured image as input data, the captured image is either an image containing the benign oral tumor, an image containing the malignant oral tumor, or an image not containing the oral tumor. The discriminant device according to any one of claims 1-3, which has a learning unit for learning the third discriminant model using the indicated data as output data. The input data used for training the second discriminant model and the third discriminant model are a divided image obtained by dividing the captured image into a plurality of captured images, an image obtained by rotating the divided image left and right, and the divided image. The discrimination device according to claim 4, further comprising one or more images selected from a group consisting of left-right inverted images. It is a discrimination terminal device that discriminates the presence or absence of an oral tumor and the benign or malignant of an oral tumor using a photographed image of the oral cavity of a patient irradiated with blue light.
The irradiation unit that irradiates the blue light and
A shooting unit that shoots the shooting target space in a predetermined shooting direction,
A conversion unit that generates a spatial frequency spectrum by Fourier transforming the captured image, and
The first discrimination model in which the spatial frequency spectrum is input to the trained first discrimination model and whether or not the captured image corresponding to the input spatial frequency spectrum is an image including the oral tumor is determined. Department and
A second discrimination unit that inputs the captured image to the trained second discrimination model and determines whether or not the input captured image is an image including the oral tumor.
When the captured image is determined to be an image containing the oral tumor by the first discrimination unit, or when the captured image is determined to be an image containing the oral tumor by the second discrimination unit. In the case, the captured image is input to the trained third discrimination model, and the input captured image includes the benign oral tumor, the malignant oral tumor, and the oral cavity. A third discriminating unit that discriminates which of the images does not contain an internal tumor,
When the captured image is determined to be an image that does not contain the oral tumor by the first discrimination unit, and the captured image is determined to be an image that does not contain the oral tumor by the second discrimination unit. Alternatively, when the captured image is determined by the third discrimination unit to be an image that does not include the oral tumor, health information indicating that the oral tumor does not exist in the oral cavity of the patient is output.
When the captured image is determined by the third discrimination unit to be an image containing the benign oral tumor, benign tumor information indicating that the benign oral tumor is present in the oral cavity of the patient is output. death,
When the photographed image is determined by the third discrimination unit to be an image containing the malignant oral tumor, malignant tumor information indicating that the malignant oral tumor is present in the oral cavity of the patient is output. A discriminant terminal device characterized by having an output unit. It is a control method of a discriminator that discriminates the presence or absence of an oral tumor and whether the oral tumor is benign or malignant by using a photographed image of the oral cavity of a patient irradiated with blue light.
By Fourier transforming the captured image, a spatial frequency spectrum is generated.
The first discrimination model in which the spatial frequency spectrum is input to the trained first discrimination model and whether or not the captured image corresponding to the input spatial frequency spectrum is an image including the oral tumor is determined. Steps and
A second discrimination step in which the captured image is input to the trained second discrimination model and it is determined whether or not the input captured image is an image containing the oral tumor.
When the captured image was determined to be an image containing the oral tumor in the first discrimination step, or when the captured image was determined to be an image containing the oral tumor in the second discrimination step. In the case, the captured image is input to the trained third discrimination model, and the input captured image includes the benign oral tumor, the malignant oral tumor, and the oral cavity. A third discrimination step for determining which of the images does not contain an internal tumor, and
When the captured image is determined to be an image that does not contain the oral tumor in the first discrimination step, and the captured image is determined to be an image that does not contain the oral tumor in the second discrimination step. Alternatively, when the captured image is determined to be an image that does not include the oral tumor in the third discrimination step, health information indicating that the oral tumor does not exist in the oral cavity of the patient is output.
When the captured image is determined to be an image containing the benign oral tumor in the third discrimination step, benign tumor information indicating that the benign oral tumor is present in the oral cavity of the patient is output. death,
When the captured image is determined to be an image containing the malignant oral tumor in the third discrimination step, malignant tumor information indicating that the malignant oral tumor is present in the oral cavity of the patient is output. A control method characterized by including an output step to be performed. It is a control program of a discriminator that discriminates the presence or absence of an oral tumor and the benign or malignant of an oral tumor using a photographed image of the oral cavity of a patient irradiated with blue light.
A conversion function that generates a spatial frequency spectrum by Fourier transforming the captured image,
The first discrimination model in which the spatial frequency spectrum is input to the trained first discrimination model and whether or not the captured image corresponding to the input spatial frequency spectrum is an image including the oral tumor is determined. Functions and
A second discrimination function that inputs the captured image to the trained second discrimination model and determines whether or not the input captured image is an image containing the oral tumor.
When the captured image is determined to be an image containing the oral tumor in the first discrimination function, or when the captured image is determined to be an image containing the oral tumor in the second discrimination function. In the case, the captured image is input to the trained third discrimination model, and the input captured image includes the benign oral tumor, the malignant oral tumor, and the oral cavity. The third discrimination function that discriminates which of the images does not include the internal tumor, and
When the captured image is determined to be an image that does not contain the oral tumor in the first discrimination function, and the captured image is determined to be an image that does not contain the oral tumor in the second discrimination function. Alternatively, when the captured image is determined by the third discrimination function to be an image that does not include the oral tumor, health information indicating that the oral tumor does not exist in the oral cavity of the patient is output.
When the captured image is determined to be an image containing the benign oral tumor by the third discrimination function, benign tumor information indicating that the benign oral tumor is present in the oral cavity of the patient is output. death,
When the captured image is determined to be an image containing the malignant oral tumor by the third discrimination function, malignant tumor information indicating that the malignant oral tumor is present in the oral cavity of the patient is output. A control program for realizing the output function to be performed by the discrimination device. A terminal device that photographs the oral cavity of a patient irradiated with blue light and outputs the captured image, and the photographed image is used to determine the presence or absence of an oral tumor and to determine whether the oral tumor is benign or malignant. A discrimination system having a discrimination device,
The terminal device is
The irradiation unit that irradiates the blue light and
A shooting unit that shoots the shooting target space in a predetermined shooting direction,
It has a terminal output unit that outputs the captured image, and has a terminal output unit.
The discrimination device is
A conversion unit that generates a spatial frequency spectrum by Fourier transforming the captured image, and
The first discrimination model in which the spatial frequency spectrum is input to the trained first discrimination model and whether or not the captured image corresponding to the input spatial frequency spectrum is an image including the oral tumor is determined. Department and
A second discrimination unit that inputs the captured image to the trained second discrimination model and determines whether or not the input captured image is an image including the oral tumor.
When the captured image is determined to be an image containing the oral tumor by the first discrimination unit, or when the captured image is determined to be an image containing the oral tumor by the second discrimination unit. In the case, the captured image is input to the trained third discrimination model, and the input captured image includes the benign oral tumor, the malignant oral tumor, and the oral cavity. A third discriminating unit that discriminates which of the images does not contain an internal tumor,
When the captured image is determined to be an image that does not contain the oral tumor by the first discrimination unit, and the captured image is determined to be an image that does not contain the oral tumor by the second discrimination unit. Alternatively, when the captured image is determined by the third discrimination unit to be an image that does not include the oral tumor, health information indicating that the oral tumor does not exist in the oral cavity of the patient is output.
When the captured image is determined by the third discrimination unit to be an image containing the benign oral tumor, benign tumor information indicating that the benign oral tumor is present in the oral cavity of the patient is output. death,
When the photographed image is determined by the third discrimination unit to be an image containing the malignant oral tumor, malignant tumor information indicating that the malignant oral tumor is present in the oral cavity of the patient is output. A discrimination system characterized by having an output unit.

Images