WO2020017211A1 - Dispositif d'apprentissage pour image médicale, procédé d'apprentissage pour image médicale et programme - Google Patents

Dispositif d'apprentissage pour image médicale, procédé d'apprentissage pour image médicale et programme Download PDF

Info

Publication number
WO2020017211A1
WO2020017211A1 PCT/JP2019/023882 JP2019023882W WO2020017211A1 WO 2020017211 A1 WO2020017211 A1 WO 2020017211A1 JP 2019023882 W JP2019023882 W JP 2019023882W WO 2020017211 A1 WO2020017211 A1 WO 2020017211A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
learning
band
group
light
Prior art date
Application number
PCT/JP2019/023882
Other languages
English (en)
Japanese (ja)
Inventor
駿平 加門
Original Assignee
富士フイルム株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 富士フイルム株式会社 filed Critical 富士フイルム株式会社
Priority to JP2020530944A priority Critical patent/JP6931425B2/ja
Publication of WO2020017211A1 publication Critical patent/WO2020017211A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing

Definitions

  • the present invention relates to a medical image learning device, a medical image learning method, and a program, and particularly to learning of a special light observation image.
  • AI technology applying deep learning is expected.
  • AI technology there is an automatic detection of a lesion and an automatic identification of a lesion.
  • AI is an abbreviation for Artificial Intelligence.
  • Patent Literature 1 describes an electronic endoscope apparatus that captures a wavelength region of visible light and generates a normal image that is a color image representing the visible light region and a diagnostic image that is a narrow-band spectral image. .
  • the document describes that a diagnostic image is generated by applying an arithmetic process to a normal image to obtain an image equivalent to a narrowband image obtained using a narrowband bandpass filter. .
  • Patent Document 2 describes a machine learning support device that supports machine learning of an artificial intelligence device and verification of the operation of the artificial intelligence device for multi-sensor information.
  • the document describes that virtual data for learning is generated by randomly changing other waveform portions while retaining the characteristic spectrum portion of the spectral data. Further, the document describes that an artificial intelligence device is learned in advance using a virtual data set.
  • the correct answer data set means an image and a set of image recognition results.
  • collecting correct answer data sets is a barrier to the development of learning models.
  • the observation using the endoscope apparatus includes not only observation using normal light but also observation using special light such as BLI (Blue LAZER Imaging).
  • BLI Bluetooth LAZER Imaging
  • White light may include light in multiple wavelength bands.
  • An example of the special light is light in a band narrower than the wavelength band of white light.
  • the special light may include light of a plurality of wavelength bands.
  • Patent Document 1 has no description about learning. Further, although Patent Literature 1 describes the adjustment of luminance and the like, it does not describe switching of a process that focuses on a difference between band components in a spectral distribution.
  • Patent Document 2 describes a learning virtual data set generated by randomly changing other wavelength components while retaining the spectral components of the spectral data.
  • Patent Literature 2 does not disclose that a band component in a corresponding band is used for learning in a plurality of image groups having different spectral distributions.
  • the present invention has been made in view of such circumstances, and can realize learning corresponding to a plurality of images having different spectral distributions without collecting a large amount of correct images for images having different spectral distributions. It is an object to provide an image learning device, a medical image learning method, and a program.
  • the medical image learning device includes an image acquisition unit that acquires a first image and a second image having different spectral distributions from each other, and an image processing unit that performs image processing on the first image to generate a third image.
  • the signal or the signal of the band having the same characteristic as the corresponding band of the second image is used.
  • a third image is generated by enhancing a signal in a similar band or suppressing a signal in a band having different characteristics from a corresponding band of the second image.
  • An example of an image having a different spectral distribution is an image to which a different observation mode is applied.
  • the observation mode there is a mode in which the illumination light is different.
  • the image processing unit may perform a process of converting a luminance variance on the first image.
  • the third image can be generated from the first image by applying the process of converting the luminance variance to the first image.
  • the image processing unit may be configured to perform a process of changing the brightness of the first image to a specified brightness value.
  • the third aspect it is possible to generate a third image from the first image by applying the process of changing the luminance to the specified luminance value to the first image.
  • the image processing unit may perform a process of changing a luminance to a random luminance value on the first image.
  • the fourth aspect it is possible to generate a third image from the first image by applying the process of changing the luminance to a random luminance value to the first image.
  • a fifth aspect is the medical image learning device according to the first aspect, wherein the first image is a normal light image captured using normal light, and the second image is captured using special light having a narrower band than the normal light. It may be configured to be a special light image.
  • the first image can be generated by applying the normal light observation mode.
  • the second image can be generated by applying the special light observation mode.
  • the image processing unit may perform a process of suppressing a B channel component on the normal light image.
  • the third image can be generated by suppressing the B channel component of the normal light image.
  • the image processing unit may perform a process of enhancing at least one of the R channel component and the G channel component on the normal light image.
  • At least one of the R channel component and the G channel component of the normal light image can be enhanced to generate the third image.
  • the image processing unit may be configured to perform processing for suppressing the R channel component on the normal light image.
  • the third image can be generated by suppressing the R channel component of the normal light image.
  • the image processing unit may perform a process of enhancing a B-channel component and a G-channel component on the normal light image.
  • the third image can be generated by enhancing the B channel component and the G channel component of the normal light image.
  • a tenth aspect is the medical image learning device according to the first aspect, wherein the image processing unit is configured to suppress a specified frequency component of a spatial frequency in a processing target band of the first image, or to perform spatial processing in a processing target band of the first image.
  • a configuration may be adopted in which processing for emphasizing a prescribed frequency component of the frequency is performed.
  • An eleventh aspect is the medical image learning device according to the tenth aspect, wherein the first image is a normal light image captured using normal light, and the second image is captured using special light having a narrower band than the normal light. It is a special light image, and the image processing unit may be configured to perform processing for enhancing the high frequency component of the B channel component of the normal light image.
  • the high frequency component among the spatial frequencies in the B channel component of the normal light image can be emphasized to generate the third image from the normal light image.
  • the learning unit may switch a learning method between learning of the first image group and learning of the second image group. Good.
  • a thirteenth aspect is a medical image learning device according to any one of the first to eleventh aspects, wherein the learning unit switches a learning method between learning of the second image group and learning of the third image group. It may be.
  • the learning unit may be configured to apply a learning method for suppressing or enhancing the effect of the third image on the learning.
  • the effect of the third image on learning can be suppressed or emphasized. Thereby, learning accuracy can be improved.
  • a fifteenth aspect is the medical image learning device according to the twelfth aspect or the thirteenth aspect, wherein the learning unit is configured to convert the first image group, the third image group, the correct data of the first image group, and the correct data of the third image group.
  • the learning using the correct data of the second image group and the second image group is performed on the learned recognizer that has been used or the learning device that has learned using the correct data of the third image group and the third image group. It may be configured to be implemented.
  • the correctness of the second image group and the second image group which are true data having higher reliability than the third image group, for at least the learned recognizer using the third image group. Re-learn using data. Thereby, the accuracy of learning can be improved, and the accuracy of the recognizer can be improved.
  • the medical image learning method includes an image acquisition step of acquiring a first image and a second image having different spectral distributions from each other, and an image processing step of performing image processing on the first image to generate a third image.
  • a program provides a computer with an image acquisition function of acquiring a first image and a second image having different spectral distributions, an image processing function of performing image processing on a first image to generate a third image, and A first image group including a plurality of first images, a second image group including a plurality of second images, and a third image group including a plurality of third images, and a first image group, a second image group, and a second image group.
  • Image processing for suppressing a signal in a band having a characteristic different from the corresponding band of the two images, and a signal of a band having the same characteristic as the corresponding band of the second image or a signal of a similar band among the bands included in the first image Image processing that emphasizes At least subjected to one, it is a program that generates a third image from the first image.
  • matters similar to the matters specified in the second to fifteenth aspects can be appropriately combined.
  • the component that performs the process or function specified in the medical image learning device can be grasped as the component of the program that performs the corresponding process or function.
  • the present invention for the first image and the second image having different spectral distributions, among the bands included in the spectral distribution of the first image, signals or similar signals having the same characteristic as the corresponding band of the second image are used.
  • the third image is generated by emphasizing the signal of the band to be processed or suppressing the signal of the band having the characteristic different from the corresponding band of the second image.
  • FIG. 1 is a block diagram illustrating an example of a hardware configuration of the endoscope image learning device according to the embodiment.
  • FIG. 2 is a functional block diagram of the endoscope image learning device according to the embodiment.
  • FIG. 3 is a functional block diagram of the learning unit shown in FIG.
  • FIG. 4 is a flowchart illustrating a procedure of the endoscopic image learning method according to the embodiment.
  • FIG. 5 is an explanatory diagram of the spectral sensitivity of the image sensor.
  • FIG. 6 is an explanatory diagram showing the relationship between the intensity distribution of the illumination light and the spectral distribution of the endoscope image.
  • FIG. 7 is an explanatory diagram showing the relationship between the spectral distribution of the normal light image and the spectral distribution of the special light image in the NBI.
  • FIG. 8 is an overall configuration diagram of the endoscope system.
  • FIG. 9 is a functional block diagram of the endoscope system.
  • FIG. 10 is a graph showing the light intensity distribution.
  • the endoscopic image learning device extends the number of learning data by combining images acquired in different observation modes. By using the images acquired in different observation modes for learning, highly accurate learning can be realized even when the number of images acquired in a specific observation mode is not sufficient. This makes it possible to realize highly accurate AI technology.
  • the endoscope image learning device according to the embodiment is an example of a medical image learning device.
  • LCI Linked Color Imaging
  • FIG. 1 is a block diagram illustrating an example of a hardware configuration of the endoscope image learning device according to the embodiment.
  • the endoscope image learning device 10 can apply a personal computer or a workstation.
  • the endoscope image learning device 10 includes a communication unit 12, a first storage device 14, a second storage device 16, a third storage device 17, an operation unit 18, a CPU 20, a RAM 22, a ROM 24, and a display unit 26.
  • CPU is an abbreviation of Central Processing Unit.
  • RAM is an abbreviation for Random @ Access @ Memory.
  • ROM is an abbreviation for Read ⁇ Only ⁇ Memory.
  • the communication unit 12 is an interface that processes communication with an external device.
  • the communication unit 12 can apply a general-purpose communication interface.
  • the communication unit 12 can apply either wired or wireless communication.
  • the first storage device 14 has a first data set including a normal light image group including a plurality of normal light images captured by applying the normal light observation mode and a correct data set indicating correct image recognition results of each normal light image. It is memorized.
  • the second storage device 16 stores a special light image group including a plurality of special light images by applying the special light observation mode and a second data set including correct data indicating a correct image recognition result of each special light image. .
  • the third storage device 17 is a third data set including a processed image group including a plurality of processed images obtained by performing image processing on the normal light image or the special light image, and correct data indicating a correct image recognition result of each processed image group. Is stored.
  • a processed image is generated from a normal light image and used for learning.
  • Correct data representing a correct recognition result of the processed image may be extracted from the first data set.
  • a processed image may be generated from the special light image, and correct data of the processed image may be extracted from the second data set.
  • the first storage device 14, the second storage device 16, and the third storage device 17 can apply a large-capacity storage device.
  • a storage device arranged outside the endoscope image learning device 10 can be applied.
  • the first storage device 14, the second storage device 16, and the third storage device 17 may be communicably connected to the endoscope image learning device 10 via a network.
  • the normal light image shown in the embodiment is an example of the first image.
  • the normal light image group described in the embodiment is an example of a first image group.
  • the special light image shown in the embodiment is an example of the second image.
  • the special light image group described in the embodiment is an example of a second image group.
  • the normal light image is a color image obtained by irradiating normal light and imaging in observation of the inside of the body cavity using the endoscope apparatus.
  • the special light image is a color image obtained by irradiating special light and capturing an image in observation of a body cavity using an endoscope apparatus.
  • Normal light is light in which light in all wavelength bands of visible light is mixed almost evenly.
  • the normal light image is used for normal observation.
  • One example of the ordinary light is white light.
  • a group of ordinary light images can be collected in a relatively large number.
  • Special light is light in various wavelength bands according to the purpose of observation, which is light in one specific wavelength band or light in a plurality of specific wavelength bands.
  • the special light has a band narrower than the white wavelength band, and is used for NBI (Narrow Band Imaging), LCI, FICE (Flexible Spectral Imaging Color), and the like.
  • NBI means narrowband observation.
  • the first example of the specific wavelength band is, for example, a blue band or a green band in a visible region.
  • the wavelength band of the first example includes a wavelength band of 390 nm to 450 nm, or 530 nm to 550 nm.
  • the light of the first example has a peak wavelength in a wavelength band of 390 nm to 450 nm, or 530 nm to 550 nm.
  • the second example of the specific wavelength band is, for example, a red band in a visible region.
  • the wavelength band of the second example includes a wavelength band of 585 nm to 615 nm, or a wavelength band of 610 nm to 730 nm.
  • the light of the second example has a peak wavelength in a wavelength band from 585 nm to 615 nm or from 610 nm to 730 nm.
  • the third example of the specific wavelength band includes a wavelength band having a different absorption coefficient between oxyhemoglobin and reduced hemoglobin, and the light of the third example has a peak wavelength at a wavelength band having a different absorption coefficient between oxyhemoglobin and reduced hemoglobin.
  • the third example wavelength band includes a wavelength band of 400 ⁇ 10 nm, 440 ⁇ 10 nm, 470 ⁇ 10 nm, or 600 nm or more and 750 nm or less.
  • the light of the third example has a peak wavelength in a wavelength band of 400 ⁇ 10 nm, 440 ⁇ 10 nm, 470 ⁇ 10 nm, or 600 nm or more and 750 nm or less.
  • the fourth example of the specific wavelength band is used for fluorescence observation which is observation of fluorescence emitted from a fluorescent substance in a living body.
  • the wavelength band of the fourth example is a wavelength band of the excitation light for exciting the fluorescent substance, for example, a wavelength band from 390 nm to 470 nm.
  • the fifth example of the specific wavelength band is a wavelength band of infrared light.
  • the fifth example wavelength band includes a wavelength band of 790 nm to 820 nm, or 905 nm to 970 nm.
  • the light of the fifth example has a peak wavelength in a wavelength band of 790 nm to 820 nm, or 905 nm to 970 nm.
  • the special light image captured under the special light having the specific wavelength band is for obtaining an easy-to-view image according to the observation purpose of the lesion, for example, an observation purpose such as wanting to observe a surface structure. Is used only when the number of data items is small.
  • the first data set of the normal light image group stored in the first storage device 14 is prepared more than the second data set of the special light image group stored in the second storage device 16. It is assumed that A number is prepared for the third data set to compensate for the shortage of the second data set.
  • the second storage device 16, and the third storage device 17 as an example of the correct answer data stored in association with each normal light image, each special light image, and each processed image, Examples include the type of lesion shown in the light image and the special light image, data indicating the position of the lesion, and identification information unique to the case.
  • the classification of the lesion there are two classifications, neoplastic or non-neoplastic, or NICE classification.
  • the data indicating the position of the lesion may be rectangular information surrounding the lesion or mask data that covers the lesion.
  • NICE is an abbreviation of NBI International Colorectal Endoscopic Classification.
  • the first storage device 14, the second storage device 16, and the third storage device 17 are provided in the endoscope image learning device 10, the first storage device 14, the second storage device 16, and the third storage device are provided. 17 may be arranged outside the endoscope image learning device 10. In this case, a data set for learning can be acquired from a database external to the endoscope image learning device 10 via the communication unit 12.
  • the operation unit 18 uses a keyboard, a mouse, and the like that are connected to the computer applied to the endoscope image learning apparatus 10 by wire or wirelessly.
  • the operation unit 18 receives various operation inputs for learning. That is, the operation unit 18 is used as a part of the user interface.
  • the CPU 20 reads various programs stored in the ROM 24 or a hard disk device (not shown) and executes various processes.
  • An example of the program to be read is an endoscope image learning program.
  • the RAM 22 is used as a work area of the CPU 20.
  • the RAM 22 is used as a storage unit that temporarily stores the read programs and various data.
  • the display unit 26 various monitors such as a liquid crystal monitor that can be connected to a computer applied to the endoscope image learning device 10 are used.
  • the display unit 26 is used as a part of a user interface.
  • the display unit 26 and the operation unit 18 may be integrally formed by applying a monitor device of a touch panel type to the display unit 26.
  • the CPU 20 reads the endoscope image learning program stored in the ROM 24, the hard disk device, or the like based on the instruction signal transmitted from the operation unit 18.
  • the CPU 20 executes an endoscope image learning program.
  • FIG. 2 is a functional block diagram of the endoscope image learning device according to the embodiment.
  • FIG. 2 is a functional block diagram showing main functions of the endoscope image learning device 10 shown in FIG.
  • the endoscope image learning device 10 includes a learning unit 30 and an image generation unit 40.
  • the learning unit 30 uses the first data set stored in the first storage device 14, the second data set stored in the second storage device 16, and the third data set stored in the third storage device 17. Learn and generate a learning model for image recognition.
  • a convolutional neural network which is one of the learning models, is constructed.
  • CNN Convolution Neural Network
  • CNN represents a convolutional neural network.
  • the image generation unit 40 generates a processed image from the normal light image. Specifically, for a corresponding wavelength band between the spectral distribution of the normal light image and the spectral distribution of the special light image, a process of emphasizing the band component when the band components are the same or similar, or When the band components are different, a process of suppressing the band components is performed. Details of the image processing in the image generation unit 40 will be described later.
  • the learning unit 30 and the image generation unit 40 may have a function of an image acquisition unit that acquires a normal light image and a special light image.
  • FIG. 3 is a functional block diagram of the learning unit shown in FIG.
  • the learning unit 30 includes a CNN 32, an error calculating unit 34, and a parameter updating unit 36.
  • the CNN 32 is, for example, a part corresponding to a recognizer that recognizes the type of a lesion appearing in an endoscope image.
  • the CNN 32 has a plurality of layer structures and holds a plurality of weight parameters.
  • the CNN 32 may change from an unlearned model to a learned model by updating the weight parameter from an initial value to an optimal value.
  • the CNN 32 includes an input layer 32a, an intermediate layer 32b, and an output layer 32c.
  • the intermediate layer 32b includes a set 32f composed of a convolutional layer 32d and a pooling layer 32e, and a fully connected layer 32g.
  • Each layer of the learning unit 30 has a structure in which a plurality of nodes are connected using edges.
  • the normal light image 14a, the special light image 16a, and the processed image 17a to be learned are input to the input layer 32a.
  • the normal light image 14a, the special light image 16a, and the processed image 17a are transmitted to the CNN 32.
  • the intermediate layer 32b is a part that extracts features from an image input using the input layer 32a.
  • the convolutional layer 32d performs a filtering process on a nearby node in the previous layer, and acquires a feature map. For the filter processing, a convolution operation using a filter is applied.
  • the pooling layer 32e reduces the feature map output from the convolutional layer 32d to a new feature map.
  • the convolution layer 32d plays a role of feature extraction such as edge extraction from an image.
  • the pooling layer 32e plays a role in providing robustness so that the extracted features are not affected by translation or the like.
  • the intermediate layer 32b is not limited to the case where the convolutional layer 32d and the pooling layer 32e constitute one set.
  • the convolutional layer 32d may be continuous or may include a normalization layer (not shown).
  • the output layer 32c outputs a recognition result for classifying the type of lesion shown in the endoscopic image based on the features extracted using the intermediate layer 32b.
  • the trained CNN 32 may, for example, classify endoscopic images into three categories: neoplastic, non-neoplastic, and other.
  • the learned CNN 32 may output a score corresponding to neoplasm, a score corresponding to non-tumor, and a score corresponding to others as a recognition result. Note that the sum of the three scores is 100%.
  • Arbitrary initial values are set for the filter coefficient applied to each convolutional layer 32d of the CNN 32 before learning, the offset value, and the connection weight with the next layer in a fully connected layer (not shown).
  • the error calculator 34 acquires the recognition result output from the output layer 32c of the CNN 32, the correct answer data 14b for the normal light image 14a, and the correct answer data 16b for the special light image 16a.
  • the correct data corresponding to the processed image 17a may be the correct data 14b for the normal light image 14a.
  • the error calculator 34 calculates an error between the recognition result and the correct answer data. Examples of a method for calculating the error include soft max cross entropy and sigmoid.
  • the parameter updating unit 36 adjusts the weight parameter of the CNN 32 by applying the error back propagation method based on the error calculated by using the error calculating unit 34.
  • the adjustment process of the weight parameter of the CNN 32 is repeatedly performed, and the learning is repeatedly performed until the difference between the output of the CNN 32 and the correct data becomes small.
  • the learning unit 30 performs learning to optimize each parameter of the CNN 32 by using all data sets of the normal light image group, all data sets of the special light image group, and all data sets of the processed image group. Generate a trained model.
  • the endoscope image shown in the embodiment is an example of a medical image.
  • FIG. 4 is a flowchart illustrating a procedure of the endoscopic image learning method according to the embodiment.
  • the endoscope image learning method shown in FIG. 4 includes a normal light image acquisition step S10, a special light image acquisition step S12, an image processing step S14, a processed image storage step S16, a learning step S18, and a recognizer update step S20.
  • the endoscope image learning method described in the embodiment is an example of a medical image learning method.
  • the learning unit 30 and the image generation unit 40 illustrated in FIG. 2 read the normal light image 14a from the first storage device 14.
  • the learning unit 30 reads the special light image 16a from the second storage device 16.
  • the special light image acquisition step S12 may be performed before the learning step S18.
  • the image generator 40 performs image processing on the normal light image 14a to generate a processed image.
  • the image generation unit 40 stores the processed image 17a in the third storage device 17.
  • the learning unit 30 reads the processed image 17a from the third storage device 17, and performs learning using the normal light image 14a, the special light image 16a, and the processed image 17a.
  • the recognizer updating step S20 the learning unit 30 updates the recognizer.
  • the image generation unit 40 generates a processed image 17a by performing a process of enhancing or suppressing a signal in a wavelength band in the spectral distribution of the normal light image 14a.
  • the image generation unit 40 suppresses the signal of the corresponding wavelength band in the normal light image 14a. I do.
  • the band components in the corresponding wavelength band between the normal light image 14a and the special light image 16a are the same or similar between the normal light image 14a and the special light image 16a, the corresponding wavelength band in the normal light image 14a May be emphasized. This makes it possible to increase the number of pieces of learning data applicable to learning of the special light image 16a.
  • the image generation unit 40 can perform post-processing such as correction to which inverse processing is applied to an image difference that is different from the wavelength dependence and is caused by a difference in processing between observation modes.
  • the learning unit 30 can apply the corrected image to learning.
  • a linear matrix, a lookup table, and the like are given.
  • FIG. 5 is an explanatory diagram of the spectral sensitivity of the image sensor.
  • the imaging device is provided in an endoscope that acquires an endoscope image.
  • the imaging device is shown in FIG.
  • the endoscope is shown in FIG.
  • the horizontal axis of the spectral sensitivity characteristic 100 shown in FIG. 5 represents the wavelength.
  • the vertical axis of the spectral sensitivity characteristic 100 represents the sensitivity.
  • the spectral sensitivity characteristic 100 has a B-channel component 102 in the B-channel wavelength band.
  • the spectral sensitivity characteristic 100 has a G channel component 104 in a G channel wavelength band.
  • the spectral sensitivity characteristic 100 has an R channel component 106 in an R channel wavelength band.
  • B represents blue.
  • G represents green.
  • R represents red.
  • FIG. 6 is an explanatory diagram showing the relationship between the intensity distribution of the illumination light and the spectral distribution of the endoscope image.
  • Reference numeral 120 shown in FIG. 6 represents the intensity distribution of normal light.
  • Reference numeral 160 represents the intensity distribution of the illumination light in LCI.
  • the horizontal axis of the normal light intensity distribution 120 and the illumination light intensity distribution 160 in the LCI represent wavelength.
  • the vertical axis represents the intensity of the illumination light. Note that the illumination light in LCI is an example of special light.
  • the symbol 140 indicates the spectral distribution of the normal light image 14a.
  • the horizontal axis of the spectral distribution 140 of the normal light image 14a represents a wavelength.
  • the vertical axis represents signal strength.
  • the spectral distribution 140 of the normal light image 14a has a B channel component 142 in the B channel wavelength band.
  • the spectral distribution 140 of the normal light image 14a has a G channel component 144 in the G channel wavelength band.
  • the spectral distribution 140 of the normal light image 14a has an R channel component 146 in the wavelength band of the R channel.
  • the symbol 180 represents the spectral distribution of the special light image 16a in LCI.
  • the horizontal axis of the spectral distribution 180 of the special light image 16a in LCI represents a wavelength.
  • the vertical axis represents signal strength.
  • the spectral distribution 180 of the special light image 16a in LCI has a B channel component 182 in the B channel wavelength band.
  • the spectral distribution 180 of the special light image 16a in LCI has a G channel component 184 in a G channel wavelength band.
  • the spectral distribution 180 of the special light image 16a in LCI has an R channel component 186 in the wavelength band of the R channel.
  • the spectral distribution 140 of the normal light image 14a and the spectral distribution 180 of the special light image 16a in LCI are different in the characteristics of the B channel component. That is, the B channel component of the normal light image 14a and the B channel component of the special light image 16a in LCI each have two peaks, but have different sensitivities at the two peaks. In other words, the normal light image 14a and the special light image 16a in the LCI have different B channel components.
  • the G channel component and the R channel component in both are equal except for the difference in absolute value. That is, the difference between the normal light image 14a and the special light image 16a in the LCI is smaller in the image including only the G channel component and the R channel component than in the image including only the B channel component. In other words, the characteristics of the G channel component and the R channel component are similar between the normal light image 14a and the special light image 16a in LCI.
  • the image information of the G channel component in the normal light image 14a and the image information of the R channel component in the normal light image 14a are also useful in learning the special light image 16a in LCI.
  • the image information of the B channel component in the normal light image 14a is considered to have unnecessary or adverse effects in learning the special light image 16a in LCI.
  • the image generation unit 40 performs processing for suppressing the image information of the B channel component in the normal light image 14a to generate the processed image 17a.
  • the learning unit 30 learns the special light image 16a in LCI by adding the processed image 17a to the normal light image 14a and the special light image 16a.
  • ⁇ As an example of the process of suppressing the image information of the B channel component in the normal light image 14a, there is a process of multiplying the luminance value in the normal light image 14a by 1 / ⁇ .
  • ⁇ is an arbitrary constant exceeding 1.
  • An example of the prescribed value is a luminance value that minimizes luminance.
  • the process of suppressing the image information of the B channel component may be a process of randomly giving the luminance value of each pixel.
  • the luminance value in the normal light image 14a may be shifted together with the process of multiplying the luminance value in the normal light image 14a by 1 / ⁇ .
  • the process of multiplying the luminance value by 1 / ⁇ described in the embodiment is an example of a process of converting the luminance variance.
  • the process of converting the luminance variance may include a process of shifting the luminance value in the normal light image 14a.
  • ⁇ is an arbitrary constant exceeding 1.
  • the luminance value in the normal light image 14a may be shifted together with the process of multiplying the luminance value in the normal light image 14a by ⁇ .
  • the process of multiplying the luminance value by ⁇ shown in the embodiment is an example of a process of converting the luminance variance.
  • the process of converting the luminance variance may include a process of shifting the luminance value in the normal light image 14a.
  • the normal light image 14a is subjected to the process of suppressing the B channel component in the spectral distribution or the process of enhancing at least one of the G channel component and the R channel component. This makes it possible to expand learning data applicable to learning of the special light image 16a in LCI.
  • ⁇ Second embodiment> The image processing according to the second embodiment is applied to the case where the normal light image 14a is used for learning the special light image 16a in LCI, similarly to the image processing according to the first embodiment. In the second embodiment, image processing is performed focusing on the difference between the B channel components in the normal light image 14a and the special light image 16a.
  • the B-channel component of the special light image 16a in the ILCI includes a larger number of short wavelength components than the B-channel component of the normal light image 14a.
  • the special light image 16a in the LCI has a property of representing a minute blood vessel structure or the like with higher contrast than the normal light image 14a.
  • the minute blood vessel structure corresponds to the high frequency component of the spatial frequency of the special light image 16a in LCI. Therefore, the image generation unit 40 generates a processed image 17a obtained by performing a process of emphasizing a high frequency component of a spatial frequency on the image of the B channel component of the normal light image 14a. This makes it possible to bring the B-channel component image of the normal light image 14a closer to the B-channel component image of the special light image 16a. Therefore, it is possible to expand the learning data of the special light image 16a in the LCI.
  • a high-frequency component there is an example in which the frequency of a special light image 16a containing many high-frequency components is analyzed and a frequency or a frequency band to be emphasized is defined based on a frequency distribution. For example, a reference frequency is calculated, and a frequency component equal to or higher than the reference frequency among the spatial frequency components of the processing target image may be set as a high frequency component.
  • the reference frequency can be calculated as the lower limit of a range in which a certain percentage or more of the frequency components of the frequency components included in the processing target image are included.
  • the process of enhancing the high frequency component of the spatial frequency there is a mask process using an unsharp mask.
  • a filter process using a low-pass filter As an example of the process of suppressing the high-frequency component of the spatial frequency, there is a filter process using a low-pass filter.
  • the high-frequency component of the spatial frequency shown in the embodiment is an example of a prescribed frequency component of the spatial frequency.
  • the image of the B channel component shown in the embodiment is an example of the processing target band.
  • the process of emphasizing the high frequency component of the spatial frequency is performed on the image of the B channel component of the normal light image 14a.
  • the processed image 17a shows a minute blood vessel structure or the like with a higher contrast than the normal light image 14a. This makes it possible to expand learning data applicable to learning of the special light image 16a in LCI.
  • ⁇ Third embodiment> The image processing according to the third embodiment is applied to a case where a special light image 16a in NBI is used in learning applied to an automatic recognition system for a region of interest using the normal light image 14a.
  • FIG. 7 is an explanatory diagram showing the relationship between the spectral distribution of a normal light image and the spectral distribution of a special light image in NBI.
  • Reference numeral 200 indicates the spectral distribution of the normal light image 14a.
  • the horizontal axis of the spectral distribution 200 of the normal light image 14a represents a wavelength.
  • the vertical axis represents signal strength.
  • the spectral distribution 200 of the normal light image 14a has a B channel component 202 in the B channel wavelength band.
  • the spectral distribution 200 of the normal light image 14a has a G channel component 204 in a G channel wavelength band.
  • the spectral distribution 200 of the normal light image 14a has an R channel component 206 in an R channel wavelength band.
  • Reference numeral 220 indicates a spectral distribution characteristic of the special light image 16a in NBI.
  • the horizontal axis of the spectral distribution 220 of the special light image 16a in NBI represents a wavelength.
  • the vertical axis represents signal strength.
  • the special light image 16a in the NBI has a B channel component 222 in the wavelength band of the B channel.
  • the special light image 16a in the NBI has a G channel component 224 in the G channel wavelength band.
  • the R channel component is lost when receiving light.
  • the spectral distribution 220 of the special light image 16a in the NBI shown in FIG. 7 indicates the missing R channel component using a dashed line. That is, in the normal light image 14a and the special light image 16a in the NBI, the R channel components are different.
  • the B channel component 222 and the G channel component 224 of the special light image 16a in the NBI have different bandwidths from the B channel component 202 and the G channel component 204 of the normal light image 14a, but have overlapping bands.
  • the B channel component and the G channel component have properties closer to each other than the R channel components. That is, in the normal light image 14a and the special light image 16a in the BI, the B channel components and the G channel components are similar.
  • the image generation unit 40 performs a process of suppressing the R channel component 206 in the normal light image 14a to generate the processed image 17a.
  • the learning unit 30 can apply the processed image to the learning of the special light image 16a in the NBI.
  • the image generation unit 40 may perform processing for enhancing at least one of the B channel component 202 and the G channel component 204 of the normal light image 14a to generate the processed image 17a.
  • the learning unit 30 can apply the processed image 17a to learning of the special light image 16a in NBI.
  • the processing of suppressing the processing target band component of the normal light image 14a and the processing of emphasizing the processing target band component are the same as in the first embodiment. The description here is omitted.
  • the normal light image 14a is subjected to the suppression processing of the R channel component or the enhancement processing of at least one of the G channel component and the B channel component. This makes it possible to expand the learning data applicable to learning the special light image 16a in the NBI.
  • the B channel component 222 or the G channel component 224 of the special light image 16a in the NBI has a large band component in which the hemoglobin absorption coefficient is large compared to the B channel component 222 or the G channel component 224 of the normal light image 14a.
  • the special light image 16a has a property of representing a blood vessel structure or the like with a higher contrast than the normal light image 14a. Therefore, similarly to the image processing according to the second embodiment, the image generation unit 40 performs a process of enhancing the high-frequency component in the spatial frequency of the image of the B-channel component 222 or the image of the G-channel component 224 of the normal light image 14a. To generate the processed image 17a.
  • the processed image 17a shows a minute blood vessel structure or the like with a higher contrast than the normal light image 14a. This makes it possible to expand the learning data applicable to learning the special light image 16a in the NBI.
  • the image processing according to the fourth embodiment is applied when learning is performed by applying a plurality of image groups acquired by applying different observation modes.
  • the image processing shown in the first embodiment to the third embodiment is an endoscope image obtained by applying an observation mode different from the original, an endoscope image acquired by applying the original observation mode in a pseudo manner. This is a process for approaching.
  • the learning method applied to the learning unit 30 is devised to adjust the balance of the plurality of image groups. Specifically, the learning method for learning the normal light image group and the learning method for the special light image group is switched. Switching between the learning method for learning the normal light image group and the learning method for the special light image group includes switching between the learning method for learning the processed image 17a generated from the normal light image 14a and the learning method for learning the special light image 16a. .
  • switching between the learning method for learning the normal light image group and the learning method for the special light image group includes switching between the learning method for learning the processed image 17a generated from the normal light image 14a and the learning method for learning the special light image 16a.
  • Equation 1 L_maim + ⁇ ⁇ L_sub Equation 1
  • ⁇ in Equation 1 is a design parameter.
  • An arbitrary constant less than 1 is applied to the design parameter ⁇ .
  • the effect of the second term representing the influence of an observation mode different from the original term on the objective function L can be further suppressed with respect to the first term representing the influence of the original observation mode.
  • the case where learning is performed using the image group of the normal light image 14a and the image group of the special light image 16a in LCI is as follows.
  • An objective function calculated from only the image group of the special light image 16a is defined as L_maim.
  • the objective function calculated from only the image group of the normal light image 14a is L_sub.
  • the endoscope image learning device 10 shown in FIG. 2 may include a design parameter setting unit for setting the design parameter ⁇ .
  • the endoscope image learning device 10 may include an objective function storage unit that stores the objective function L.
  • the learning unit 30 can read out the objective function L and the design parameter ⁇ and perform learning.
  • a CNN is learned using a plurality of image groups having different observation modes. Thereafter, the CNN is re-learned using only the image group to which the original observation mode is applied, using the learned parameters as initial values.
  • relearning is performed using the special light image 16a for the CNN that has been learned using the normal light image 14a and the processed image 17a or the CNN that has been learned using the processed image 17a.
  • the learning unit 30 includes a first learning unit that learns the CNN using a plurality of image groups having different observation modes, and a second learning unit that re-learns the CNN using only the image group to which the original observation mode is applied. Can be provided.
  • the influence of the image group to which the observation mode different from the original is applied is suppressed while using the information of the image group to which the observation mode different from the original is applied. Learning can be performed.
  • FIG. 8 is an overall configuration diagram of the endoscope system.
  • the endoscope system 300 shown in FIG. 8 includes an endoscope 302, a light source device 311, a processor device 312, a display device 313, an image processing device 314, an input device 315, and a monitor device 316.
  • the endoscope 302 is an electronic endoscope.
  • the endoscope 302 is a flexible endoscope.
  • the endoscope 302 includes an insertion section 320, an operation section 321 and a universal cord 322.
  • the insertion section 320 is inserted into the subject.
  • the insertion section 320 is formed in a small diameter and a long shape as a whole.
  • the insertion section 320 includes a flexible section 325, a curved section 326, and a tip section 327.
  • the insertion section 320 is configured by connecting a flexible section 325, a bending section 326, and a tip section 327 in a row.
  • the flexible portion 325 has flexibility in order from the proximal end to the distal end of the insertion portion 320.
  • the bending section 326 has a structure that can be bent when the operation section 321 is operated.
  • the distal end portion 327 has a built-in image pickup optical system and image pickup element 328 (not shown).
  • CMOS is an abbreviation for Complementary ⁇ Metal ⁇ Oxide ⁇ Semiconductor.
  • CCD is an abbreviation for Charge ⁇ Coupled ⁇ Device.
  • An observation window (not shown) is arranged on the distal end surface 327a of the distal end portion 327.
  • the observation window is an opening formed on the distal end surface 327a of the distal end portion 327.
  • a cover (not shown) is attached to the observation window.
  • An imaging optical system (not shown) is arranged behind the observation window.
  • the image plane of the observation site enters the imaging surface of the imaging element 328 via an observation window, an imaging optical system, and the like.
  • the image sensor 328 captures the image light of the observed region that has entered the image capturing surface of the image sensor 328 and outputs an image signal.
  • imaging as used herein means that reflected light from the region to be observed is converted into an electric signal.
  • the operation unit 321 is continuously provided on the base end side of the insertion unit 320.
  • the operation unit 321 includes various operation members operated by an operator.
  • the operation unit 321 includes two types of bending operation knobs 329.
  • the bending operation knob 329 is used when performing a bending operation of the bending portion 326. Note that the surgeon may be called a doctor, an operator, an observer, a user, or the like.
  • the operation unit 321 includes an air / water supply button 330 and a suction button 331.
  • the air / water button 330 is used when the operator performs an air / water operation.
  • the suction button 331 is used when an operator performs a suction operation.
  • the operation unit 321 includes a still image capturing instruction unit 332 and a treatment instrument introduction port 333.
  • the still image capturing instruction unit 332 is operated by the operator when capturing a still image of the observed region.
  • the treatment instrument introduction port 333 is an opening for inserting the treatment instrument into the treatment instrument insertion passage that passes through the inside of the insertion section 320. The illustration of the treatment tool insertion passage and the treatment tool is omitted.
  • the universal cord 322 is a connection cord for connecting the endoscope 302 to the light source device 311.
  • the universal cord 322 includes a light guide 335, a signal cable 336, and a fluid tube (not shown) that pass through the inside of the insertion section 320.
  • the distal end of the universal cord 322 includes a connector 337a connected to the light source device 311 and a connector 337b branched from the connector 337a and connected to the processor device 312.
  • the connector 337 a When the connector 337 a is connected to the light source device 311, the light guide 335 and a fluid tube (not shown) are inserted into the light source device 311. Thereby, necessary illumination light, water, and gas are supplied from the light source device 311 to the endoscope 302 via the light guide 335 and a fluid tube (not shown).
  • the illumination light is emitted from the illumination window (not shown) of the distal end surface 327a of the distal end portion 327 toward the observation target site.
  • gas or water is jetted from an air / water supply nozzle (not shown) on the distal end surface 327a of the distal end portion 327 toward an observation window (not shown) on the distal end surface 327a.
  • the signal cable 336 and the processor device 312 are electrically connected.
  • an imaging signal of the observed region is output from the imaging element 328 of the endoscope 302 to the processor device 312 via the signal cable 336, and a control signal is output from the processor device 312 to the endoscope 302.
  • a flexible endoscope has been described as an example of the endoscope 302.
  • various types of electronic endoscopes capable of capturing moving images of a site to be observed such as a hard endoscope, are described.
  • An endoscope may be used.
  • the light source device 311 supplies illumination light to the light guide 335 of the endoscope 302 via the connector 337a.
  • the illumination light white light or light in a specific wavelength band can be applied.
  • the illumination light may be a combination of white light and light of a specific wavelength band.
  • the light source device 311 is configured to be able to appropriately select light in a wavelength band according to an observation purpose as illumination light.
  • the white light may be light in a white wavelength band or light in a plurality of wavelength bands.
  • the specific wavelength band is a band narrower than the white wavelength band.
  • light of a specific wavelength band light of one type of wavelength band may be applied, or light of a plurality of wavelength bands may be applied.
  • a specific wavelength band may be called special light.
  • the processor device 312 controls the operation of the endoscope 302 via the connector 337b and the signal cable 336. In addition, the processor device 312 acquires an image signal from the image sensor 328 of the endoscope 302 via the connector 337b and the signal cable 336. The processor device 312 acquires an image signal output from the endoscope 302 by applying a specified frame rate.
  • the processor device 312 generates an endoscope image which is an observation image of the observed part based on the imaging signal acquired from the endoscope 302.
  • the endoscope image 338 here includes a moving image.
  • the endoscope image 338 may include a still image 339.
  • the processor device 312 When the still image imaging instruction unit 332 of the operation unit 321 is operated, the processor device 312 generates a still image 339 of the observed part based on the imaging signal acquired from the imaging element 328 in parallel with the generation of the moving image. .
  • the still image 339 may be generated at a higher resolution than the resolution of the moving image.
  • the processor device 312 performs image quality correction using digital signal processing such as white balance adjustment and shading correction.
  • the processor device 312 may add additional information specified by the DICOM standard to the endoscope image 338.
  • DICOM is an abbreviation for Digital Imaging and Communications in Medicine.
  • the endoscope image 338 is an in-vivo image of the inside of the subject, that is, the inside of the living body.
  • the processor device 312 outputs the generated endoscope image 338 to each of the display device 313 and the image processing device 314.
  • the processor device 312 may output the endoscope image 338 to a storage device (not shown) via a network (not shown) according to a communication protocol conforming to the DICOM standard.
  • the display device 313 is connected to the processor device 312.
  • the display device 313 displays the endoscope image 338 transmitted from the processor device 312.
  • the operator can perform the operation of moving the insertion section 320 forward and backward while checking the endoscope image 338 displayed on the display device 313.
  • the surgeon can operate the still image imaging instruction unit 332 to capture a still image of the observed region.
  • a computer is used for the image processing device 314.
  • the input device 315 a keyboard, a mouse, and the like connectable to a computer are used.
  • the connection between the input device 315 and the computer may be either a wired connection or a wireless connection.
  • the monitor device 316 various monitors that can be connected to a computer are used.
  • a diagnosis support device such as a workstation and a server device may be used.
  • the input device 315 and the monitor device 316 are provided for each of a plurality of terminals connected to a workstation or the like.
  • a medical service support device that supports creation of a medical report or the like may be used.
  • the image processing device 314 acquires the endoscope image 338 and stores the endoscope image 338.
  • the image processing device 314 controls the reproduction of the monitor device 316.
  • the image processing device 314 illustrated in FIG. 8 can have the function of the endoscope image learning device 10 described with reference to FIGS. 1 to 7.
  • the input device 315 illustrated in FIG. 8 corresponds to the operation unit 18 illustrated in FIG.
  • the monitor device 316 shown in FIG. 8 corresponds to the display unit 26 shown in FIG.
  • image in this specification includes the meaning of a still image 339 such as an electric signal representing an image and information representing an image.
  • image in this specification means at least one of the image itself and image data.
  • image storage can be read as image storage.
  • Image storage here means non-temporary storage of an image.
  • the image processing device 314 may include a temporary storage memory for temporarily storing an image.
  • the input device 315 is used to input an operation instruction to the image processing device 314.
  • the monitor device 316 displays the endoscope image 338 under the control of the image processing device 314.
  • the monitor device 316 may function as a display unit of various information in the image processing device 314.
  • the image processing device 314 can be connected to a storage device (not shown) via a network (not shown).
  • a network not shown
  • DICOM standard For communication between the devices via the image storage format and the network, the DICOM standard, a protocol based on the DICOM standard, and the like can be applied.
  • a storage device (not shown) to which data is temporarily stored can be applied.
  • the storage device may be managed using a server device (not shown).
  • a computer that stores and manages various data can be applied to the server device.
  • FIG. 9 is a functional block diagram of the endoscope system.
  • the endoscope system 300 is configured to be switchable between a normal light observation mode and a special light observation mode.
  • the operator can switch between a normal light observation mode and a special light observation mode by operating an observation mode switching button (not shown).
  • the light source device 311 includes a first laser light source 400, a second laser light source 402, and a light source control unit 404.
  • the first laser light source 400 is a blue laser light source having a center wavelength of 445 nm.
  • the second laser light source 402 is a violet laser light source having a center wavelength of 405 nm.
  • laser diodes can be used as the first laser light source 400 and the second laser light source 402.
  • Light emission of the first laser light source 400 and the second laser light source 402 is individually controlled using the light source control unit 404.
  • the emission intensity ratio between the first laser light source 400 and the second laser light source 402 can be changed freely.
  • the endoscope 302 includes a first optical fiber 410, a second optical fiber 412, a phosphor 414, a diffusion member 416, an imaging lens 418, an imaging device 328, and an analog-to-digital converter 420.
  • An irradiation unit is configured using the first laser light source 400, the second laser light source 402, the first optical fiber 410, the second optical fiber 412, the phosphor 414, and the diffusion member 416.
  • Laser light emitted from the first laser light source 400 is applied to the phosphor 414 disposed at the distal end 327 of the endoscope 302 via the first optical fiber 410.
  • the phosphor 414 is configured to include a plurality of types of phosphors that absorb part of the blue laser light from the first laser light source 400 and excite and emit light in a range from green to yellow. Accordingly, the light emitted from the phosphor 414 is blue which transmits the blue laser light from green to excitation light not absorbed by the excitation light L 11 and the phosphor 414 in the range up to the yellow from the first laser light source 400 the laser beam L 12 is combined, the light L 1 of the white or pseudo white.
  • the white light mentioned here is not limited to a light that strictly includes all wavelength components of visible light.
  • any light containing a specific wavelength band such as R, G, and B, may be used.
  • Light including a wavelength component from green to red, or light including a wavelength component from blue to green may be broadly defined. Shall be included.
  • the laser light emitted from the second laser light source 402 is applied to the diffusion member 416 disposed at the distal end 327 of the endoscope 302 via the second optical fiber 412.
  • a resin material or the like having a light-transmitting property can be used for the diffusion member 416. Light emitted from the diffusion member 416, a light L 2 of the narrow band wavelength light amount is uniform in the exposure area.
  • FIG. 10 is a graph showing the light intensity distribution.
  • the light source control unit 404 changes the light amount ratio between the first laser light source 400 and the second laser light source 402.
  • the light quantity ratio between the light L 1 and the light L 2 is changed
  • the changed wavelength pattern of the light L 1 and the irradiation light L 0 is a composite light of the light L 2 is, wavelengths different patterns depending on the observation mode it can be irradiated with the irradiation light L 0 of.
  • the endoscope system 300 shown in FIG. 9 includes an imaging unit including an imaging lens 418, an imaging element 328, and an analog-to-digital conversion unit 420. As shown in FIG. 8, the imaging unit is disposed at the distal end 327 of the endoscope 302.
  • the imaging lens 418 shown in FIG. 9 focuses the incident light on the imaging element 328.
  • the image sensor 328 generates an analog signal corresponding to the received light.
  • An analog signal output from the image sensor 328 is converted to a digital signal using the analog-to-digital converter 420 and input to the processor device 312.
  • the processor device 312 includes an imaging control unit 440, an image processing unit 442, an image acquisition unit 444, and an image recognition unit 446.
  • the imaging control unit 440 controls the light source control unit 404 of the light source device 311, the imaging element 328 and the analog-to-digital conversion unit 420 of the endoscope 302, and the image processing unit 442 of the processor device 312, and uses the endoscope system 300. It controls the imaging of the moving image and the still image.
  • the image processing unit 442 performs image processing on the digital signal input from the analog-to-digital conversion unit 420 of the endoscope 302 to generate an image.
  • the image processing unit 442 performs image processing according to the wavelength pattern of irradiation light at the time of imaging.
  • the image acquisition unit 444 acquires the image generated by the image processing unit 442. That is, the image acquisition unit 444 sequentially acquires a plurality of images that are taken in chronological order by applying a fixed frame rate to the inside of the body cavity of the subject. Note that the image acquisition unit 444 may acquire an image input from the input unit 447 or an image stored in the storage unit 468. Further, an image may be obtained from an external device such as a server connected to a network (not shown). The images in these cases are also preferably a plurality of images taken in time series.
  • the image recognition unit 446 performs image recognition of the image acquired by the image acquisition unit 444 using the learning model learned by the endoscope image learning device 10.
  • a lesion is recognized from the image acquired by the image acquisition unit 444.
  • the lesion is not limited to the one caused by the disease but includes an area having a state different from a normal state in appearance.
  • Examples of lesions include polyps, cancer, colonic diverticulum, inflammation, treatment marks, clipping points, bleeding points, perforations, and vascular atypia.
  • Examples of the treatment scar include an EMR scar and an ESD scar.
  • EMR is an abbreviation for Endoscopic ⁇ Mucosal ⁇ Resection.
  • ESD is an abbreviation for Endoscopic ⁇ Submucosal ⁇ Dissection.
  • the display control unit 450 causes the display device 313 to display the image generated using the image processing unit 442.
  • the display control unit 450 may superimpose the lesion recognized using the image recognition unit 446 on the image so as to be recognizable.
  • the storage control unit 452 causes the storage unit 468 to store the image generated using the image processing unit 442. For example, and stores the information of the wavelength pattern of the irradiation light L 0 at the time of capturing a captured image and the image according to the acquired instruction of a still image in the storage unit 468.
  • An example of the storage unit 468 is a storage device such as a hard disk. Note that the storage unit 468 is not limited to the one built in the processor device 312. For example, an external storage device (not shown) connected to the processor device 312 may be used. The external storage device may be connected via a network (not shown).
  • the endoscope system 300 configured as described above normally captures a moving image at a fixed frame rate, and displays the captured image on the display device 313. Further, a lesion is detected from the captured moving image, and the detected lesion is superimposed on the moving image so as to be recognizable and displayed on the display device 313.
  • the image recognition unit 446 using the learning model trained by the endoscope image learning device 10 is applied to automatically recognize the endoscope image and perform image recognition of the special light image. Can be done properly.
  • the endoscope system 300 described with reference to FIGS. 8 to 10 can acquire the normal light image 14a and the special light image 16a.
  • the processor device 312 may have the function of the image processing device 314. That is, the processor device 312 may be configured integrally with the image processing device 314. In such an embodiment, the display device 313 can also serve as the monitor device 316.
  • the processor device 312 may include a connection terminal to which the input device 315 is connected.
  • Example of generating feature image As a medical image, based on at least one of a white band light, a normal image obtained by irradiating light of a plurality of wavelength bands as white band light, and a special light image obtained by irradiating light of a specific wavelength band The calculation may be used to generate a feature image.
  • the above-described endoscopic image learning method can be configured as a program that realizes a function corresponding to each step in the endoscopic image learning method using a computer.
  • a computer has an image acquisition function of acquiring first and second images having different spectral distributions from each other, an image processing function of performing image processing on the first image to generate a third image, and a plurality of first images.
  • Each of the first image group, the second image group including the plurality of second images, and the third image group including the plurality of third images, and the first image group, the second image group, and the third image group A program for realizing a learning function of learning a recognizer to be applied to automatic recognition using correct answer data that is a correct recognition result, wherein the image processing function is configured to perform a second function of a band included in the spectral distribution of the first image.
  • Less image processing to emphasize Also subjected to either a can realize a program for generating a third image from the first image.
  • a program for causing a computer to realize the above-described endoscopic image learning function is stored in a computer-readable information storage medium, which is a non-transitory information storage medium that is a tangible entity, and the program is provided through the information storage medium. Is possible.

Abstract

L'invention concerne un dispositif d'apprentissage pour image médicale, un procédé d'apprentissage pour image médicale, et un programme par lequel l'apprentissage pour manipuler une pluralité d'images ayant différentes distributions spectrales peut être effectué sans nécessiter de nombreuses images correctes à collecter pour chacune des images ayant des distributions spectrales différentes. Le dispositif d'apprentissage pour image médicale comprend : des unités d'acquisition d'images (30, 40) qui acquièrent une première image et une seconde image ayant différentes distributions spectrales; une unité de traitement d'image (40) qui génère une troisième image par réalisation d'un traitement d'image sur la première image; et une unité d'apprentissage (30) qui est amené à apprendre un dispositif de reconnaissance à adopter pour une reconnaissance automatique à l'aide d'un premier groupe d'images, d'un deuxième groupe d'images, d'un troisième groupe d'images et de données correctes pour les images respectives. L'unité de traitement d'image génère la troisième image à partir de la première image par suppression, dans la première image, de signaux dans une bande passante ayant des caractéristiques différentes de celles de la seconde image et/ou par amélioration, dans la première image, de signaux dans une bande passante ayant des caractéristiques identiques ou similaires à celles de la seconde image.
PCT/JP2019/023882 2018-07-20 2019-06-17 Dispositif d'apprentissage pour image médicale, procédé d'apprentissage pour image médicale et programme WO2020017211A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2020530944A JP6931425B2 (ja) 2018-07-20 2019-06-17 医用画像学習装置、医用画像学習方法、及びプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018136922 2018-07-20
JP2018-136922 2018-07-20

Publications (1)

Publication Number Publication Date
WO2020017211A1 true WO2020017211A1 (fr) 2020-01-23

Family

ID=69164325

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/023882 WO2020017211A1 (fr) 2018-07-20 2019-06-17 Dispositif d'apprentissage pour image médicale, procédé d'apprentissage pour image médicale et programme

Country Status (2)

Country Link
JP (1) JP6931425B2 (fr)
WO (1) WO2020017211A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20230093127A (ko) * 2021-12-17 2023-06-27 가천대학교 산학협력단 의료영상의 깊이영상 예측 시스템 및 방법

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8903167B2 (en) * 2011-05-12 2014-12-02 Microsoft Corporation Synthesizing training samples for object recognition
WO2016208016A1 (fr) * 2015-06-24 2016-12-29 オリンパス株式会社 Dispositif de traitement d'image, procédé de traitement d'image et programme de traitement d'image
WO2017109904A1 (fr) * 2015-12-24 2017-06-29 オリンパス株式会社 Dispositif de traitement d'image, procédé de traitement d'image, et programme
WO2017175282A1 (fr) * 2016-04-04 2017-10-12 オリンパス株式会社 Procédé d'apprentissage, dispositif de reconnaissance d'image et programme

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8903167B2 (en) * 2011-05-12 2014-12-02 Microsoft Corporation Synthesizing training samples for object recognition
WO2016208016A1 (fr) * 2015-06-24 2016-12-29 オリンパス株式会社 Dispositif de traitement d'image, procédé de traitement d'image et programme de traitement d'image
WO2017109904A1 (fr) * 2015-12-24 2017-06-29 オリンパス株式会社 Dispositif de traitement d'image, procédé de traitement d'image, et programme
WO2017175282A1 (fr) * 2016-04-04 2017-10-12 オリンパス株式会社 Procédé d'apprentissage, dispositif de reconnaissance d'image et programme

Also Published As

Publication number Publication date
JP6931425B2 (ja) 2021-09-01
JPWO2020017211A1 (ja) 2021-08-02

Similar Documents

Publication Publication Date Title
JP7346285B2 (ja) 医療画像処理装置、内視鏡システム、医療画像処理装置の作動方法及びプログラム
JP5438571B2 (ja) 電子内視鏡システム
US10335014B2 (en) Endoscope system, processor device, and method for operating endoscope system
WO2013005533A1 (fr) Système d'endoscope, processeur de système d'endoscope et procédé d'affichage d'image
JP7005767B2 (ja) 内視鏡画像認識装置、内視鏡画像学習装置、内視鏡画像学習方法及びプログラム
JP5222934B2 (ja) 内視鏡システム、内視鏡システムのプロセッサ装置、及び内視鏡システムの作動方法
JP2011194151A (ja) 電子内視鏡システム
JP5757891B2 (ja) 電子内視鏡システム、画像処理装置、画像処理装置の作動方法及び画像処理プログラム
WO2020012872A1 (fr) Dispositif de traitement d'image médicale, système de traitement d'image médicale, procédé de traitement d'image médicale et programme
CN107105987B (zh) 图像处理装置及其工作方法、记录介质和内窥镜装置
JP2022186872A (ja) 医用画像処理装置、プロセッサ装置、内視鏡システム、医用画像処理装置の作動方法、及びプログラム
JP2023115352A (ja) 医療画像処理装置、医療画像処理システム、医療画像処理装置の作動方法及び医療画像処理プログラム
US11978550B2 (en) Endoscopic image learning device, endoscopic image learning method, endoscopic image learning program, and endoscopic image recognition device
JP7289373B2 (ja) 医療画像処理装置、内視鏡システム、診断支援方法及びプログラム
JPWO2020174747A1 (ja) 医用画像処理装置、プロセッサ装置、内視鏡システム、医用画像処理方法、及びプログラム
WO2020184257A1 (fr) Appareil et procédé de traitement d'image médicale
WO2020054255A1 (fr) Dispositif d'endoscope, processeur d'endoscope et procédé de fonctionnement de dispositif d'endoscope
CN109152520A (zh) 图像信号处理装置、图像信号处理方法以及图像信号处理程序
WO2020017211A1 (fr) Dispositif d'apprentissage pour image médicale, procédé d'apprentissage pour image médicale et programme
US10863149B2 (en) Image processing apparatus, image processing method, and computer readable recording medium
JP7387859B2 (ja) 医用画像処理装置、プロセッサ装置、内視鏡システム、医用画像処理装置の作動方法及びプログラム
JP7122328B2 (ja) 画像処理装置、プロセッサ装置、画像処理方法、及びプログラム
WO2018079217A1 (fr) Système d'endoscopie et son procédé de fonctionnement
WO2020067100A1 (fr) Dispositif de traitement d'image médicale, dispositif processeur, méthode de traitement d'image médicale et programme
WO2022049901A1 (fr) Dispositif d'apprentissage, procédé d'apprentissage, appareil de traitement d'images, système d'endoscope et programme

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19837171

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020530944

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19837171

Country of ref document: EP

Kind code of ref document: A1