WO2024079785A1 - 内視鏡システム、画像生成装置および画像生成方法 - Google Patents

内視鏡システム、画像生成装置および画像生成方法 Download PDF

Info

Publication number
WO2024079785A1
WO2024079785A1 PCT/JP2022/037862 JP2022037862W WO2024079785A1 WO 2024079785 A1 WO2024079785 A1 WO 2024079785A1 JP 2022037862 W JP2022037862 W JP 2022037862W WO 2024079785 A1 WO2024079785 A1 WO 2024079785A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
captured image
captured
light
imaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/037862
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
延好 浅岡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Olympus Medical Systems Corp
Original Assignee
Olympus Medical Systems Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Olympus Medical Systems Corp filed Critical Olympus Medical Systems Corp
Priority to PCT/JP2022/037862 priority Critical patent/WO2024079785A1/ja
Priority to JP2024550941A priority patent/JPWO2024079785A1/ja
Priority to CN202280100911.4A priority patent/CN120018808A/zh
Publication of WO2024079785A1 publication Critical patent/WO2024079785A1/ja
Priority to US19/175,263 priority patent/US20250235090A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000095—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope for image enhancement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0638—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements providing two or more wavelengths
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0655—Control therefor

Definitions

  • the present invention relates to an endoscope system, an image generating device, and an image generating method.
  • the endoscope system is provided with an illumination unit and an imaging unit at the tip, and the illumination unit irradiates the subject with illumination light, and the imaging unit captures the subject. It is desirable that the captured image be bright enough to allow the subject to be visually recognized in detail.
  • an upper limit on the amount of illumination light is set so that the temperature of the tip does not exceed an acceptable range.
  • image signal processing such as gain adjustment
  • noise resulting from the image signal processing will exceed an acceptable range.
  • Patent Document 1 describes an endoscope system that suppresses the amount of illumination light emitted to an upper limit value except when capturing still images, thereby suppressing temperature increases at the tip of the endoscope.
  • the temperature of the tip of the endoscope may still exceed the allowable range when capturing still images.
  • the present invention aims to provide an endoscope system, an image generating device, and an image generating method that can provide sufficiently bright captured images while suppressing temperature rise at the tip of the endoscope, even when capturing still images.
  • An endoscopic system comprises an endoscope having an illumination unit that irradiates illumination light onto a subject and an imaging unit that images the subject, a control device that performs imaging signal processing based on imaging parameters on an imaging signal acquired from the imaging unit to generate an image, and an image generating device, wherein the control device generates a first captured image, which is an image captured of the subject illuminated with an amount of light equal to or less than an upper limit value, and a second captured image, which is an image captured of the subject illuminated with an amount of light equal to or less than the upper limit value and is brighter than the first captured image, and the image generating device generates an estimated image by estimating an image captured of the subject illuminated with an amount of light greater than the upper limit value from the first captured image and the second captured image.
  • the image generating device is an image generating device that acquires an imaged image by performing image signal processing based on imaging parameters on an imaging signal acquired from an imaging unit that images a subject, and acquires a first captured image in which the subject is captured while illuminated with an amount of light equal to or less than an upper limit, and a second captured image in which the subject is captured while illuminated with an amount of light equal to or less than the upper limit and is brighter than the first captured image, and generates an estimated image by estimating an image of the subject captured while illuminated with an amount of light greater than the upper limit from the first captured image and the second captured image.
  • the image generating method obtains a first captured image generated by performing image signal processing on an image signal obtained by capturing an image of a subject illuminated with an amount of light equal to or less than an upper limit, and a second captured image generated by performing image signal processing on an image signal obtained by capturing an image of the subject illuminated with an amount of light equal to or less than the upper limit, the second captured image being brighter than the first captured image, and generates an estimated image by estimating an image captured of the subject illuminated with an amount of light greater than the upper limit from the first captured image and the second captured image.
  • the endoscope system, image generating device, and image generating method of the present invention can provide sufficiently bright captured images while suppressing temperature rise at the tip of the endoscope, even when capturing still images.
  • FIG. 1 is a diagram showing an endoscope system according to a first embodiment.
  • FIG. 2 is a functional block diagram of the endoscope system.
  • FIG. 2 is a function block diagram of an imaging control unit 21 of the endoscope system.
  • FIG. 2 is a diagram illustrating a hardware configuration of an image generating device of the endoscope system.
  • FIG. 2 is a functional block diagram of the image generating device.
  • 4A and 4B are diagrams illustrating a relationship between a first captured image and a second captured image.
  • FIG. 13 is a diagram illustrating the generation of a trained model of the endoscope system.
  • FIG. 13 is a diagram showing training data used to generate the trained model.
  • 3 is a control flowchart of the endoscope system.
  • FIG. 1 is a diagram showing an endoscope system according to a first embodiment.
  • FIG. 2 is a functional block diagram of the endoscope system.
  • FIG. 2 is a function block diagram of an imaging control unit 21
  • FIG. 13 is a diagram showing a modified example of the trained model.
  • FIG. 13 is a diagram showing another modified example of the trained model.
  • 3A to 3C are diagrams showing moving images generated by the image generating unit of FIG. 13A to 13C are diagrams showing an image generated by an imaging control unit and a still image generated by an image generating unit in an endoscope system according to a second embodiment.
  • 3 is a control flowchart of the endoscope system.
  • FIG. 11 is a functional block diagram of an endoscope system according to a third embodiment.
  • FIG. 2 is a diagram showing an image generating unit of the endoscope system.
  • 13A and 13B are diagrams illustrating a relationship between an image captured with a first special light and an image captured with a second special light.
  • FIG. 13 is a diagram showing a modified example of the trained model of the endoscope system.
  • FIG. 13 is a diagram illustrating the generation of the trained model.
  • FIG. 13 is a diagram showing training data used to generate the trained model.
  • FIG. 13 is a diagram showing an image generating unit of an endoscope system according to a fourth embodiment.
  • FIG. 13 is a diagram illustrating the generation of a trained model of the endoscope system.
  • FIG. 1 An endoscope system 100 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 9.
  • FIG. 1 An endoscope system 100 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 9.
  • FIG. 1 An endoscope system 100 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 9.
  • FIG. 1 An endoscope system 100 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 9.
  • FIG. 1 is a diagram showing an endoscope system 100 .
  • the endoscope system (image generating system) 100 includes an endoscope 1, a control device 2, an image generating device 3, and a display device 4.
  • the control device 2 and the image generating device 3 may be an integrated device (image control device).
  • the display device 4 is a device that displays images generated by the control device 2 and the image generating device 3, various information related to the endoscope system 100, and the like.
  • the endoscope 1 is a device for observing and treating the inside of the body of a patient lying on, for example, an operating table T.
  • the endoscope 1 includes an elongated insertion section 10 that is inserted into the body of the patient, an operating section 18 that is connected to the base end of the insertion section 10, and a universal cord 19 that extends from the operating section 18.
  • the insertion section 10 has a tip section 11, a freely bendable bending section 12, and a long, flexible flexible tube section 13.
  • the tip section 11, the bending section 12, and the flexible tube section 13 are connected in that order from the tip side.
  • the flexible tube section 13 is connected to the operation section 18.
  • FIG. 2 is a functional block diagram of the endoscope system 100.
  • the tip portion 11 has an imaging unit 14 , an illumination unit 15 , and a temperature sensor 17 .
  • the imaging unit 14 has an optical system 141 and an imaging element 142 such as a CCD image sensor or a CMOS image sensor (see FIG. 3).
  • the imaging unit 14 captures an image of a subject based on imaging parameters P transmitted from the imaging control unit 21 via an imaging control cable 143, and generates an imaging signal.
  • the imaging signal is transmitted to the control device 2 via an imaging signal cable 144.
  • the illumination unit (white light illumination unit) 15 irradiates the subject with illumination light (white light) transmitted by the light guide 151.
  • the light guide 151 is connected to the control device 2 by inserting the insertion unit 10, the operation unit 18, and the universal cord 19.
  • the illumination unit 15 may have a light source such as an LED, or an optical element such as a phosphor with a wavelength conversion function.
  • the temperature sensor 17 is a sensor that detects the temperature of the lighting unit 15.
  • the temperature sensor 17 includes, for example, a thermocouple, a thermistor, a temperature resistor, a thermal ferrite, a thermal expansion thermometer, etc.
  • the detected temperature of the lighting unit 15 is acquired by the control device 2.
  • the operation unit 18 accepts operations for the endoscope 1.
  • the operation unit 18 has an ankle knob 181 that controls the bending portion 12, an air/water supply button 182, a suction button 183, a release button 184, and an observation mode switching button 185.
  • Operations input to the air/water supply button 182, the suction button 183, the release button 184, and the observation mode switching button 185 are acquired by the control device 2.
  • the release button 184 is a push button that inputs an operation to save the captured image acquired from the imaging unit 14.
  • the universal cord 19 connects the endoscope 1 and the control device 2.
  • the universal cord 19 is a cable through which the imaging control cable 143, the imaging signal cable 144, the light guide 151, etc. are inserted.
  • the control device 2 is a device that controls the entire endoscope system 100.
  • the control device 2 includes an imaging control unit 21, an illumination control unit 22, a control unit 24, and a recording unit 25.
  • FIG. 3 is a functional block diagram of the imaging control unit 21. As shown in FIG. The imaging control unit 21 performs imaging signal processing based on imaging parameters P on the imaging signal acquired from the imaging unit 14 to generate an imaged image (imaged video) DW .
  • the imaging control unit 21 has an imaging drive unit 211, an analog signal processing unit 212, an AD conversion unit 213, and a digital signal processing unit 214.
  • the imaging driver 211 drives the imaging element 142 of the imaging unit 14 based on the imaging parameter P from the control unit 24.
  • the imaging driver 211 controls the exposure of the imaging element 142.
  • the analog signal processing unit 212 performs analog signal processing, including noise removal and amplification, on the imaging signal acquired from the imaging element of the imaging unit 14 based on the imaging parameters P (analog gain, ISO sensitivity, etc.) from the control unit 24.
  • the AD conversion unit 213 converts the image signal that has been subjected to analog signal processing by the analog signal processing unit 212 into a digital signal (e.g., RAW data) based on instructions from the control unit 24.
  • the digital signal processing unit 214 performs digital signal processing on the digital signal (e.g., RAW data) converted by the AD conversion unit 213 based on the imaging parameter P from the control unit 24 to generate a captured image.
  • the digital signal processing unit 214 performs digital signal processing on the captured image to adjust brightness, contrast, etc. as necessary.
  • the lighting control unit 22 has a light source such as an LED, and controls the light source based on the imaging parameters P (such as light intensity) from the control unit 24 to control the amount of illumination light transmitted to the lighting unit 15 via the light guide 151.
  • a light source such as an LED
  • the imaging parameters P such as light intensity
  • the control unit 24 is a programmable processing circuit (computer) that has a processor and a memory into which a program can be read.
  • the control unit 24 controls the endoscope system 100 by executing an endoscope control program.
  • the control unit 24 may include a dedicated circuit.
  • the dedicated circuit is a processor separate from the processor in the control unit 24, a logic circuit implemented in an ASIC or FPGA, or a combination of these.
  • the control unit 24 controls the imaging control unit 21 and the illumination control unit 22 by specifying operation parameters including the imaging parameter P to the imaging control unit 21 and the illumination control unit 22 to generate the captured image DW .
  • the control unit 24 also instructs the image generation device 3 to generate an image and instructs the display device 4 to display the image.
  • the recording unit 25 is a non-volatile recording medium that stores the above-mentioned programs and necessary data.
  • the recording unit 25 is composed of, for example, a flexible disk, a magneto-optical disk, a writable non-volatile memory such as a ROM or a flash memory, a portable medium such as a CD-ROM, or a storage device such as a hard disk built into a computer system.
  • the recording unit 25 may also be a storage device provided on a cloud server connected to the control device 2 via the Internet.
  • the image generating device 3 is a device that generates a new estimated image EW based on the captured image (captured video) DW output by the imaging control unit 21 of the control device 2, and displays the generated estimated image EW on the display device 4. When there is no need to generate a new estimated image EW , the image generating device 3 displays the captured image output by the imaging control unit 21 of the control device 2 on the display device 4.
  • FIG. 4 is a diagram showing the hardware configuration of the image generating device 3.
  • the image generating device 3 includes a processor 301, a memory 302 capable of reading programs, a storage unit 303, and an input/output control unit 304.
  • the image generating device 3 is a computer capable of executing programs.
  • the functions of the image generating device 3 are realized by the processor 301 executing the programs. At least a part of the functions of the image generating device 3 may be realized by a dedicated logic circuit implemented in an ASIC or an FPGA.
  • the storage unit 303 is a non-volatile recording medium that stores the above-mentioned programs and necessary data.
  • the storage unit 303 is composed of, for example, a ROM or a hard disk.
  • the programs recorded in the storage unit 303 are read into the memory 302 and executed by the processor 301.
  • the input/output control unit 304 is connected to the control device 2, the display device 4, an input device (not shown), and a network device (not shown). Based on the control of the processor 301, the input/output control unit 304 transmits and receives data and control signals to and from the connected devices.
  • the image generating device 3 is not limited to being an integrated hardware device.
  • the image generating device 3 may be configured by separating a portion of it into a separate hardware device and connecting the separated hardware device via a communication line.
  • the image generating device 3 may be a cloud system in which the separated storage unit 303 is connected via a communication line.
  • the image generating device 3 may further include components other than the processor 301, the memory 302, the storage unit 303, and the input/output control unit 304.
  • the image generating device 3 may further include an image calculation unit that performs part or all of the image processing and image recognition processing. By further including the image calculation unit, the image generating device 3 can execute specific image processing and image recognition processing at high speed.
  • the image generating device 3 may further include an inference calculation unit that performs part or all of the inference processing of the estimated image EW described below. By further including the inference calculation unit, the image generating device 3 can execute the inference processing of the estimated image EW at high speed.
  • the image calculation unit and the inference calculation unit may be mounted on separate hardware devices connected via a communication line.
  • FIG. 5 is a functional block diagram of the image generating device 3.
  • the image generating device 3 includes, as functional blocks, an image generating unit 31 , an image display unit 32 , and an image storage unit 33 .
  • the image generation unit 31 generates an estimated image (white light estimated image) EW from the first captured image (first white light captured image) DW1 and the second captured image (second white light captured image) DW2 input from the control device 2 based on the trained model M.
  • FIG. 6 is a diagram showing the relationship between the first captured image DW1 and the second captured image DW2 .
  • the first captured image DW1 is an image of an object illuminated with a light amount equal to or less than the upper limit value U.
  • the second captured image DW2 is an image of an object illuminated with a light amount equal to or less than the upper limit value U, and is an image with increased brightness compared to the first captured image DW1 , although it is accompanied by increased noise due to imaging signal processing based on the imaging parameter P (analog signal processing by the analog signal processing unit 212 and/or digital signal processing by the digital signal processing unit 214).
  • the "brightness of the captured image” refers to, for example, “brightness", “luminance”, and “luminosity”.
  • the brightness of the first captured image DW1 is also referred to as "first brightness B1”
  • the brightness of the second captured image DW2 is also referred to as "second brightness B2”.
  • the first captured image DW1 and the second captured image DW2 are captured images of a subject illuminated with approximately the same amount of light. Note that the first captured image DW1 and the second captured image DW2 do not necessarily have to be captured images of a subject illuminated with approximately the same amount of light.
  • the first captured image DW1 and the second captured image DW2 are captured images of a subject illuminated with an amount of light that is equal to or less than the upper limit value U and as close as possible to the upper limit value U.
  • the first captured image DW1 and the second captured image DW2 become brighter and have less noise.
  • the estimation accuracy of the estimated image EW can be improved.
  • the estimation accuracy of the estimated image EW can be efficiently improved.
  • the second captured image DW2 is captured after a period T1 has elapsed since the first captured image DW1 was captured. Since the period T1 is very short, the first captured image DW1 and the second captured image DW2 capture substantially the same subject. The order in which the first captured image DW1 and the second captured image DW2 are captured does not matter.
  • the first captured image DW1 and the second captured image DW2 are input to the trained model M.
  • the difference between the information included in the first captured image DW1 and the information included in the second captured image DW2 is large. Therefore, it is desirable that the imaging parameters P for generating the first captured image DW1 and the imaging parameters P for generating the second captured image DW2 are more different from each other.
  • the image signal processing includes, for example, amplification of an analog signal or amplification of a digital signal.
  • the gain of the image capturing parameters P for generating the second captured image DW2 is greater than the gain of the image capturing parameters P for generating the first captured image DW1 .
  • amplification of an analog signal is more suitable than amplification of a digital signal. This is because amplification of a digital signal is performed after AD conversion, in which part of the information of the imaging signal is lost, and is therefore disadvantageous in terms of increasing the difference between the information contained in the first captured image DW1 and the information contained in the second captured image DW2 .
  • amplification of a digital signal is more suitable than amplification of an analog signal in order to perform diverse and flexible imaging signal processing.
  • imaging signal processing based on gains of two types of imaging parameters P can be performed on one type of imaging signal acquired from the imaging unit 14 to generate the first captured image DW1 and the second captured image DW2 .
  • the estimated image EW is an image generated based on the trained model M from the first captured image DW1 and the second captured image DW2 , and is an estimate of an image captured of a subject illuminated with an amount of light greater than the upper limit value U.
  • the image display unit 32 outputs the estimated image EW generated by the image generating unit 31 to the display device 4 and displays it on the display device 4 .
  • the image storage unit 33 records the estimated image EW generated by the image generation unit 31. If there is no need to record the estimated image EW , the image storage unit 33 is not necessary.
  • the trained model M is a machine learning model that learns the relationship between an input image and an output image, and is a machine learning model suitable for image generation, such as a neural network, a simple perceptron, or a multilayer perceptron.
  • the trained model M is generated by prior supervised learning based on teacher data.
  • the trained model M may be generated by the image generation device 3, or may be generated by using another computer with a higher computing power than the image generation device 3.
  • FIG. 7 is a diagram illustrating the generation of the trained model M.
  • the teacher data is a set of images including a first learning image DTW1 , a second learning image DTW2 , and a third learning image DTW3 . It is desirable for the teacher data to include as many images as possible captured under a variety of conditions.
  • the trained model M generates an estimated image EW based on the input first captured image for learning DTW1 and second captured image for learning DTW2 .
  • the trained model M trains parameters of the trained model M so that the difference between the estimated image EW to be output and the third captured image for learning DTW3 becomes small.
  • FIG. 8 is a diagram showing training data used to generate the trained model M.
  • the learning first captured image DTW1 is a first captured image DW1 prepared for learning
  • the learning second captured image DTW2 is a second captured image DW2 prepared for learning.
  • the third learning image DTW3 is an image obtained by actually capturing an image of a subject illuminated with a light amount greater than the upper limit value U.
  • the third learning image DTW3 is an image in which noise generated by imaging signal processing based on imaging parameters P (analog signal processing by the analog signal processing unit 212 and/or digital signal processing by the digital signal processing unit 214) is comparable to noise generated by imaging signal processing in the first learning image DTW1 .
  • the third learning image DTW3 is an image in which the imaging parameters P other than the light amount are substantially the same as the imaging parameters P of the first learning image DTW1 .
  • the third learning image DTW3 is an image captured after a period T2 has elapsed since the first learning image DTW1 and the second learning image DTW2 were captured. Since the period T2 is very short, the subjects captured in the first learning image DTW1 , the second learning image DTW2 , and the third learning image DTW3 are in substantially the same state. The order in which the first learning image DTW1 , the second learning image DTW2 , and the third learning image DTW3 are captured does not matter.
  • the first learning image DTW1 , the second learning image DTW2 , and the third learning image DTW3 may be images of a stationary subject. In this case, the subjects captured in the first learning image DTW1 , the second learning image DTW2 , and the third learning image DTW3 are in the same state.
  • the brightness of the first captured image DTW1 for learning and the brightness of the second captured image DTW2 for learning are substantially equal to the first brightness B1 and the second brightness B2, because this allows the trained model M to easily and efficiently learn the relationship between the first captured image DW1 and the second captured image DW2 .
  • the second captured image for learning DTW2 is an image captured approximately a period T1 after the first captured image for learning DTW1 is captured, because this allows the trained model M to easily and efficiently learn the relationship between the first captured image DW1 and the second captured image DW2 .
  • a trained model M is generated by previously learning by machine learning the relationship between the first captured image DW1 and the second captured image DW2 and the estimated image EW through supervised learning using the above-mentioned teacher data.
  • the noise contained in the estimated image EW output by the trained model M to which the first captured image DW1 and the second captured image DW2 are input is about the same as the noise generated in the first captured image DW1 by image signal processing.
  • Step S110 Imaging parameter adjustment process>
  • the control unit 24 of the control device 2 calculates optimal imaging parameters P (including light amount and analog gain) for imaging the subject based on the captured image captured by the imaging control unit.
  • a known imaging adjustment technique is used to calculate the optimal imaging parameters P.
  • the endoscope system 100 then executes step S120.
  • Step S120 Light amount determination process>
  • the control unit 24 of the control device 2 determines whether the calculated light amount exceeds the upper limit value U. If the calculated light amount exceeds the upper limit value U, the control unit 24 of the control device 2 next executes step S130. If the calculated light amount is equal to or less than the upper limit value U, the endoscope system 100 next executes step S160.
  • the upper limit value U is not a fixed value, but may be a value that varies based on the temperature of the illumination unit 15 acquired by the temperature sensor 17 of the endoscope 1. For example, if the temperature of the illumination unit 15 is lower than a predetermined temperature, the control unit 24 may increase the upper limit value U. Also, if the temperature of the illumination unit 15 is higher than the predetermined temperature, the control unit 24 may decrease the upper limit value U. Note that, if the endoscope 1 does not have a temperature sensor 17, the control unit 24 may use a temperature estimated from the accumulated light amount obtained by multiplying the light amount and the light emission time.
  • Step S130 First captured image generating process>
  • the control unit 24 of the control device 2 sets the light amount of the imaging parameter P to be equal to or less than the upper limit value U, and generates the first captured image DW1 .
  • the endoscope system 100 then executes step S140.
  • Step S140 Second captured image generating process>
  • the control unit 24 of the control device 2 changes the imaging parameters P other than the light amount to generate a second captured image DW2 that is brighter but has increased noise compared to the first captured image DW1 .
  • the control unit 24 increases the analog gain of the imaging parameter P to generate the second captured image DW2 .
  • the control unit 24 of the control device 2 generates the second captured image DW2 after a period T1 has elapsed since capturing the first captured image DW1 .
  • the endoscope system 100 then executes step S150.
  • Step S150 Estimated image generation process>
  • the image generating unit 31 of the image generating device 3 generates an estimated image EW from the first captured image DW1 and the second captured image DW2 input from the imaging control unit 21 of the control device 2 based on the trained model M.
  • the image storage unit 33 records the estimated image EW as necessary.
  • the endoscope system 100 then executes step S170.
  • Step S160 First captured image generating process>
  • the control unit 24 and the imaging control unit 21 of the control device 2 generate a first captured image DW1 . Since the light amount calculated in step S110 is equal to or less than the upper limit value U, the control unit 24 of the control device 2 sets the light amount to the calculated light amount and generates the first captured image DW1 .
  • the endoscope system 100 then executes step S170.
  • Step S170 Image display process>
  • the image display unit 32 of the image generating device 3 displays the estimated image EW generated in step S150 or the first captured image DW1 captured in step S160 on the display device 4.
  • the endoscope system 100 then executes step S180.
  • step S180 the control unit 24 of the control device 2 determines whether or not the release button 184 has been pressed by the surgeon or the like. If the release button 184 has been pressed, the control unit 24 of the control device 2 performs step S110 again.
  • the endoscope system 100 of this embodiment even when a still image is acquired, a sufficiently bright captured image can be provided while suppressing a temperature rise of the tip 11 of the endoscope 1.
  • the endoscope system 100 uses only the first captured image DW1 and the second captured image DW2 captured of an object illuminated with a light amount equal to or less than the upper limit U, the amount of illumination light irradiated from the illumination unit 15 can be suppressed to the upper limit U or less, and the temperature of the tip 11 of the endoscope 1 can be suppressed to a predetermined temperature or less.
  • the image generating device 3 of the endoscope system 100 can infer an estimated image EW that estimates an image captured of an object illuminated with a light amount greater than the upper limit U from the first captured image DW1 and the second captured image DW2 captured of an object illuminated with a light amount equal to or less than the upper limit U. That is, the endoscope system 100 can present to the user an estimated image EW that is brighter than the first captured image DW1 and contains noise equivalent to that generated by image signal processing in the first captured image DW1 while suppressing the amount of illumination light irradiated from the illumination unit 15 to the upper limit U or less.
  • FIG. 10 is a diagram showing a trained model M1, which is a modified example of the trained model M.
  • the trained model M1 uses at least a part of the imaging parameters P (e.g., analog gain) as input data.
  • the trained model M1 further has, as inputs, a first imaging parameter PW1 which is the imaging parameter P when the first captured image DW1 is captured, and a second imaging parameter PW2 which is the imaging parameter P when the second captured image DW2 is captured.
  • the imaging parameters P are also added as inputs to teacher data used for training the trained model M1.
  • FIG. 11 is a diagram showing a trained model M2, which is another modified example of the trained model M.
  • the trained model M2 uses the additional auxiliary captured image as input data.
  • the trained model M2 further has as inputs a first auxiliary captured image S W1 captured immediately after capturing the first captured image D W1 with the amount of light set to zero without changing the imaging parameters P other than the amount of light, and a second auxiliary captured image S W2 captured immediately after capturing the second captured image D W2 with the amount of light set to zero without changing the imaging parameters P other than the amount of light.
  • an estimated image E W that is less susceptible to the pattern noise of the imaging element 142 of the imaging unit 14 can be generated.
  • the estimated image E W can be generated with higher accuracy by adding the auxiliary captured image captured with the amount of light set to zero as an input.
  • the auxiliary captured image is also added as an input to the teacher data used for learning the trained model M2.
  • FIG. 12 is a diagram showing a moving image (continuous estimated images EW ) generated by the image generating unit 31.
  • the endoscope system 100 generates one estimated image EW by executing a series of steps from step S110 to step S170, and generates a moving image (continuous estimated images EW ) by repeatedly executing the series of steps. It is desirable that the endoscope system 100 has a processing performance for generating the estimated images EW at, for example, 30 FPS in order to generate a smooth moving image.
  • the endoscope system 100 adjusts the amount of light in step S110 every time an estimated image EW is generated. Therefore, the amount of light fluctuates during generation of a moving image. It is desirable that the amount of light is equal to or less than the upper limit value U and close to the upper limit value U, but it may be set to a value lower than the upper limit value U based on the adjustment result in step S110. Note that the generation sequence of a moving image shown in FIG. 12 is a generation sequence of a moving image when the amount of light calculated in step S120 exceeds the upper limit value U.
  • the endoscope system 100 When the amount of light calculated during generation of a moving image is equal to or less than the upper limit value U, the endoscope system 100 generates a first captured image DW1 in step S160, and the generated first captured image DW1 is set as one frame of the moving image.
  • the endoscopic system 100 can present to the user a moving image including a series of estimated images EW, which are brighter than the first captured image DW1 and contain noise equivalent to that generated by image signal processing in the first captured image DW1 , while keeping the amount of illumination light irradiated from the illumination unit 15 below an upper limit value U.
  • the illumination unit irradiates the subject with white light, but the illumination light irradiated by the illumination unit is not limited to this.
  • the illumination unit may irradiate the subject with special light, which will be described later.
  • FIG. 13 A second embodiment of the present invention will be described with reference to Fig. 13 and Fig. 14.
  • An endoscope system 100B according to the second embodiment differs from the endoscope system 100 according to the first embodiment only in the control flow.
  • components common to those already described will be denoted by the same reference numerals, and duplicated description will be omitted.
  • the endoscope system 100B displays captured video (moving images) on the display device 4, and generates a captured image (still image) when the control unit 24 of the control device 2 detects the operator pressing (ON) the release button 184.
  • Fig. 13 is a diagram showing a captured image generated by the imaging control unit 21 and a still image (estimated image E W ) generated by the image generation unit 31. The description will be given along with the control flowchart of the endoscope system 100B shown in Fig. 14. When the endoscope system 100B is started up, the endoscope system 100B executes step S210.
  • Step S210 Video Generation Process>
  • the control unit 24 of the control device 2 calculates optimal imaging parameters P (including light amount and analog gain) for imaging the subject, and continuously generates second captured images DW2 .
  • the control unit 24 of the control device 2 generates second captured images DW2 at 30 FPS.
  • the control unit 24 of the control device 2 adjusts the light amount within a range equal to or less than an upper limit value U.
  • the control unit 24 of the control device 2 causes the display device 4 to display the continuously generated second captured images DW2 as captured video (moving image).
  • Step S220 Release Detection Process>
  • the control unit 24 of the control device 2 detects a pressing operation of the release button 184.
  • the control unit 24 of the control device 2 continues step S210 unless a pressing operation of the release button 184 is detected.
  • the endoscope system 100B next executes step S230.
  • Step S230 First captured image generating process>
  • the control unit 24 of the control device 2 sets the light amount of the imaging parameter P to be equal to or less than the upper limit value U, as in step S130 of the first embodiment, and generates a first captured image DW1 .
  • the control unit 24 temporarily reduces the analog gain of the imaging parameter P to generate the first captured image DW1 .
  • control unit 24 After generating one first captured image DW1 , the control unit 24 causes the imaging control unit 21 to continuously generate second captured images DW2 , and causes the display device 4 to display the continuously generated second captured images DW2 as captured video (moving images).
  • the endoscope system 100B then executes step S240.
  • Step S240 Estimated image generation process>
  • the image generating unit 31 of the image generating device 3 generates an estimated image EW from the first captured image DW1 generated in step S230 and the second captured image DW2 generated before and after the generation of the first captured image DW1 , based on the trained model M.
  • the image storage unit 33 records the estimated image EW as necessary.
  • the endoscope system 100B then executes step S250.
  • Step S250 Image display process>
  • the image display unit 32 of the image generating device 3 displays the estimated image EW generated in step S240 on the display device 4.
  • the image display unit 32 of the image generating device 3 may simultaneously display the second captured image DW2 displayed as a moving image and the estimated image EW on the display device 4. Furthermore, the image display unit 32 of the image generating device 3 may display the estimated image EW on the display device 4 only for a predetermined period of time, replacing the second captured image DW2 displayed as a moving image.
  • the endoscope system 100 then executes step S260.
  • Step S260 End Determination Process> The control unit 24 of the control device 2 determines whether to end the moving image display in step S180. If the moving image display is not to be ended, the control unit 24 of the control device 2 executes step S210 again.
  • the endoscope system 100B may generate the estimated images EW successively, as in the third modification, and generate a moving image having the successive estimated images EW as frames.
  • the endoscope system 100B can provide a sufficiently bright captured image (still image) while suppressing the temperature rise of the tip 11 of the endoscope 1, even when a still image is captured in the middle of acquiring a captured video (moving image).
  • the endoscope system 100B generates an estimated image EW as a captured image (still image) using the continuously generated second captured image DW2 as a captured video (moving image) and one first captured image DW1 generated in response to a pressing operation of the release button 184.
  • the manner of generating the captured video (moving image) and the still image is not limited to this.
  • the endoscope system 100B may generate an estimated image EW as a captured image (still image) using the continuously generated first captured image DW1 as a captured video (moving image) and one second captured image DW2 generated in response to a pressing operation of the release button 184.
  • the endoscope system 100C according to the third embodiment can use two types of observation modes.
  • components common to those already described will be denoted by the same reference numerals, and duplicated description will be omitted.
  • FIG. 15 is a functional block diagram of an endoscope system 100C.
  • the endoscope system 100C includes an endoscope 1C, a control device 2C, an image generating device 3C, and a display device 4.
  • the endoscope 1C is the same as the endoscope 1 of the first embodiment except for the tip portion 11C.
  • the tip portion 11C of the endoscope 1C has an imaging section 14, an illumination section 15, a special light illumination section 16, and a temperature sensor 17.
  • the special light illumination unit 16 illuminates the subject with special light used in NBI (registered trademark, Narrow Band Imaging) and RDI (registered trademark, Red Dichromatic Imaging).
  • NBI registered trademark, Narrow Band Imaging
  • RDI registered trademark, Red Dichromatic Imaging
  • the special light illuminated by the special light illumination unit 16 is light with a different wavelength range from white light.
  • the special light used in NBI is light with two narrow wavelength ranges: blue (wavelengths of 390 to 445 nm) and green (wavelengths of 530 to 550 nm).
  • the special light used in RDI is light with three narrow wavelength ranges: red, amber, and green.
  • the control device 2C is a device that controls the entire endoscope system 100C.
  • the control device 2C includes an imaging control unit 21C, an illumination control unit 22, a special light illumination control unit 23, a control unit 24C, and a recording unit 25.
  • the imaging control unit 21C has the same functions as the imaging control unit 21 of the first embodiment, and can further perform imaging signal processing based on imaging parameters P on an imaging signal obtained by imaging a subject illuminated with special light, to generate a special light imaging image (special light imaging video) D S.
  • the special light illumination control unit 23 controls the light source of the special light based on the imaging parameters P (such as the amount of special light) from the control unit 24, and controls the amount of special light transmitted to the special light illumination unit 16.
  • the control unit 24C has the same functions as the control unit 24 in the first embodiment, and can further switch between two types of observation modes (white light observation mode and special light observation mode) based on an operation input to the observation mode switching button 185, etc.
  • the control unit 24C In the white light observation mode, the control unit 24C generates an image (white light captured image) DW from an imaging signal obtained by capturing an image of a subject illuminated with white light from the illumination unit 15.
  • the control unit 24C In the special light observation mode, the control unit 24C generates a special light captured image DS from an imaging signal obtained by capturing an image of a subject illuminated with special light from the special light illumination unit 16.
  • the image generating device 3C has the following functional blocks: an image generating unit 31C, an image display unit 32, and an image storage unit 33.
  • FIG. 16 is a diagram showing the image generating unit 31C.
  • the image generator 31C generates an estimated image EW from the first captured image DW1 , the second captured image DW2 , the first special light captured image DS1 and the second special light captured image DS2 input from the control device 2C based on the trained model MC .
  • the first special light captured image D S1 is a special light captured image captured of a subject illuminated with special light with an amount of special light not greater than the upper limit U.
  • the second special light captured image D S2 is a special light captured image captured of a subject illuminated with special light with an amount of special light not greater than the upper limit U, and is a special light captured image that, compared to the first special light captured image D S1 , is brighter but with increased noise due to imaging signal processing based on the imaging parameter P (analog signal processing by the analog signal processing unit 212 and/or digital signal processing by the digital signal processing unit 214).
  • the first special light captured image D S1 and the second special light captured image D S2 are captured images of a subject illuminated with an amount of light that is equal to or less than the upper limit U and as close as possible to the upper limit U.
  • the first special light captured image D S1 and the second special light captured image D S2 become brighter and have less noise.
  • the estimation accuracy of the estimated image E W can be improved.
  • the estimation accuracy of the estimated image E W can be efficiently improved.
  • the first special light captured image DS1 is a special light captured image captured a period T3 after the first captured image DW1 . Because the period T3 is very short, the subject captured in the first special light captured image DW1 and the first special light captured image DS1 are in substantially the same state. The order in which the first special light captured image DW1 and the first special light captured image DS1 are captured does not matter.
  • the second special light captured image D S2 is a special light captured image captured a period T4 after the second special light captured image D W2 was captured. Because the period T4 is very short, the subject captured in the second special light captured image D W2 and the second special light captured image D S2 are in substantially the same state. The order in which the second special light captured image D W2 and the second special light captured image D S2 are captured does not matter.
  • the trained model MC further has, as inputs, a first special light captured image D S1 and a second special light captured image D S2 .
  • the special light captured images are also added as inputs to the teacher data used for training the trained model MC.
  • the endoscopic system 100C When the control unit 24C of the control device 2C detects the surgeon pressing the release button 184, the endoscopic system 100C generates a first captured image DW1 , a second captured image DW2 , a first special light captured image DS1 and a second special light captured image DS2 , and generates an estimated image EW from these four captured images.
  • the endoscope system 100C may continuously generate estimated images EW and generate a moving image having the continuous estimated images EW as frames. Also, as in the second embodiment, the endoscope system 100C may use the continuously generated second captured image DW2 as a captured video (moving image) and generate an estimated image EW as a captured image (still image) using the first captured image DW1 , the first special light captured image DS1 and the second special light captured image DS2 generated in response to the pressing operation of the release button 184 .
  • the endoscope system 100C of this embodiment while keeping the amount of illumination light irradiated from the illumination unit 15 and the special light illumination unit 16 below the upper limit value U, it is possible to present to the user an estimated image EW that is brighter than the first captured image DW1 and contains noise that is comparable to the noise generated in the first captured image DW1 by image signal processing.
  • a special light captured image obtained from an imaging signal capturing an image of a subject illuminated with special light is added as an input to the trained model MC. Since the two types of captured images captured by capturing an image of a subject illuminated with two types of illumination light (white light and special light) have different characteristics, the estimation accuracy of the estimated image EW by the image generator 31C can be suitably improved.
  • FIG. 18 is a diagram showing a trained model MC1, which is a modified example of the trained model MC.
  • the trained model MC1 outputs a special light estimated image E S.
  • the special light estimated image E S is an image that is brighter than the first special light captured image D S1 and contains noise that is comparable to the noise generated by image signal processing in the first special light captured image D S1 .
  • the trained model MC1 may output both the estimated image E W and the special light estimated image E S.
  • FIG. 19 is a diagram illustrating the generation of the trained model MC1.
  • the teacher data is a group of images including a first learning image DTW1 , a first special light learning image DTS1 , a second learning image DTW2 , a second special light learning image DTS2 , and a third special light learning image DTS3 .
  • FIG. 20 is a diagram showing training data used to generate the trained model MC1.
  • the first special light captured image for learning DTS1 is the first special light captured image DTS1 prepared for learning purposes
  • the second special light captured image for learning DTS2 is the second special light captured image DTS2 prepared for learning purposes.
  • the third special light learning image DTS3 is an image actually captured of a subject illuminated with special light with an amount of special light greater than the upper limit U.
  • the third special light learning image DTS3 is an image in which noise generated by imaging signal processing based on imaging parameter P (analog signal processing by the analog signal processing unit 212 and/or digital signal processing by the digital signal processing unit 214) is comparable to the noise generated by imaging signal processing in the first special light learning image DTS1 .
  • the third special light learning image DTS3 is an image in which the imaging parameters P other than the amount of light are substantially the same as the imaging parameters P of the first special light learning image DTS1 .
  • the endoscope 1C has two types of observation modes. If the endoscope 1C has three or more types of observation modes, the trained model MC may be capable of inputting three or more types of captured images.
  • FIG. 21 A fourth embodiment of the present invention will be described with reference to Fig. 21 and Fig. 22.
  • the endoscope system 100D according to the fourth embodiment can use two types of observation modes, similar to the endoscope system 100C according to the third embodiment.
  • components common to those already described will be denoted by the same reference numerals, and duplicated description will be omitted.
  • the endoscope system 100D has an image generating unit 31D instead of the image generating unit 31C.
  • FIG. 21 is a diagram showing the image generating unit 31D.
  • the image generator 31D generates a special light estimated image E S from the first special light captured image D S1 and the second special light captured image D W2 input from the control device 2C based on the trained model MD.
  • FIG. 22 is a diagram illustrating the generation of the trained model MD.
  • the training data is a set of images including a first special light image for learning DTS1 , a second special light image for learning DTW2 , and a third special light image for learning DTS3 . It is desirable for the training data to include as many images as possible captured under a variety of conditions.
  • the trained model MD generates a special light estimated image E S based on the input first special light captured image D TS1 and second special light captured image D TW2 for training.
  • the trained model MD trains parameters of the trained model MD so that the difference between the output special light estimated image E S and the output third special light captured image D TS3 for training is small.
  • the third special light learning image DTS3 is an image captured a period T5 after the first special light learning image DTS1 and the second special light learning image DTW2 are captured. Because the period T5 is very short, the first special light learning image DTS1 , the second special light learning image DTW2 , and the third special light learning image DTS3 capture subjects in substantially the same state. The order in which the first special light learning image DTS1 , the second special light learning image DTW2 , and the third special light learning image DTS3 are captured does not matter.
  • the endoscope system 100D When the control unit 24C of the control device 2C detects the operator pressing the release button 184, the endoscope system 100D generates a first special light captured image D S1 and a second captured image D W2 , and generates a special light estimated image E S from these two captured images.
  • the endoscope system 100D may generate special light estimated images E S continuously and generate a moving image having the continuous special light estimated images E S as frames, as in Modification 3.
  • the endoscope system 100D may also generate special light estimated images E S as captured images (moving images) using the continuously generated second captured images DW2 , as in the second embodiment, and generate special light estimated images E S as captured images (still images) using the first special light captured images D S1 generated in response to the pressing operation of the release button 184.
  • the endoscope system 100D can present to a user a special light estimated image E S that is brighter than the first special light captured image D S1 and contains noise comparable to that generated by image signal processing in the first special light captured image D S1 , while keeping the amount of illumination light irradiated from the illumination unit 15 and the special light illumination unit 16 below the upper limit U.
  • the endoscope system 100D requires fewer captured images to generate an estimated image compared to the endoscope system 100C of the third embodiment. This allows for a higher frame rate when generating a moving image using estimated images as frames, as in Modification 3.
  • the special light captured image may be darker than the white light captured image depending on the type of special light.
  • the special light captured image acquired by NBI is darker than the white light captured image.
  • the special light used in NBI is more easily absorbed by the subject, that is, human tissue, than white light, and is less likely to be reflected by the human tissue. Therefore, when the irradiated light intensity is the same, the light intensity received by the image sensor 142 is weaker when the subject is irradiated with special light than when the subject is irradiated with white light.
  • the endoscope system 100D can generate and provide a special light estimated image E S brighter than the first special light captured image D S1 from the first special light captured image D S1 and the second captured image D W2 .
  • the imaging section 14 that generates an imaging signal is provided in the endoscope 1, and the image generating device 3, 3C generates an estimated image E based on an image acquired from the imaging section 14 of the endoscope 1.
  • the source of the imaging signal of the image generating device is not limited to the endoscope 1.
  • the image generating device may generate an estimated image E based on an image acquired from another imaging device, such as a camera, a video camera, an industrial endoscope, a microscope, a robot having an image recognition function, or a mobile device such as a smartphone, a mobile phone, a smartwatch, a tablet terminal, or a notebook PC.
  • LEDs are exemplified as light sources provided in the illumination unit 15 and the illumination control unit 22, but the light source is not limited to LEDs.
  • the light source may be a laser light source including a laser diode, an organic electroluminescence (EL), a light bulb such as a xenon or halogen bulb, a combination of these, or a combination of these with an optical element such as a phosphor having a wavelength conversion function.
  • EL organic electroluminescence
  • a light bulb such as a xenon or halogen bulb
  • an optical element such as a phosphor having a wavelength conversion function.
  • the imaging parameter P is the amount of light and the gain, but the imaging parameter P is not limited to these.
  • the imaging parameter P may be a spectral distribution, an aperture value (F value), a shutter speed, a frame rate of a moving image, an optical magnification, a digital magnification, a gradation, a pixel size, a resolution (DPI), etc., in a process in which an imaged image (imaged video) is generated in the imaging control unit 21 from an imaging signal captured by the imaging element 142.
  • the imaging parameter P may also include an organ type (esophagus, stomach, duodenum, etc.) of the subject. Note that the organ type may be determined from the captured image by a model such as a machine learning model, may be determined from the insertion distance of the tip portion 11 of the endoscope 1 into the body, or may be input to the image generating device 3 by the surgeon.
  • the programs in each embodiment and each modified example thereof may be recorded on a computer-readable recording medium, and the programs recorded on the recording medium may be read into a computer system and executed.
  • the term "computer system” includes hardware such as an OS and peripheral devices.
  • the term "computer-readable recording medium” refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system.
  • the term "computer-readable recording medium” may also include devices that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, or devices that hold a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in such a case.
  • the above-mentioned program may be for realizing part of the above-mentioned functions, or may be capable of realizing the above-mentioned functions in combination with a program already recorded in the computer system.
  • Endoscope system image generating system
  • Endoscope 14 Imaging unit 15 Illumination unit (white light illumination unit)
  • Special light illumination unit 17 Temperature sensor 2
  • Image generating device 4 Display device P Imaging parameter U Upper limit value D W1 First captured image (first white light captured image)
  • D W2 second captured image second white light captured image
  • EW estimated image white light estimated image
  • D S1 First captured image first special light captured image
  • D S2 Second captured image second special light captured image
  • ES estimated image special light estimated image

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Public Health (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Endoscopes (AREA)
PCT/JP2022/037862 2022-10-11 2022-10-11 内視鏡システム、画像生成装置および画像生成方法 Ceased WO2024079785A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/JP2022/037862 WO2024079785A1 (ja) 2022-10-11 2022-10-11 内視鏡システム、画像生成装置および画像生成方法
JP2024550941A JPWO2024079785A1 (https=) 2022-10-11 2022-10-11
CN202280100911.4A CN120018808A (zh) 2022-10-11 2022-10-11 内窥镜系统、图像生成装置以及图像生成方法
US19/175,263 US20250235090A1 (en) 2022-10-11 2025-04-10 Endoscope system, image generation device, and image generation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/037862 WO2024079785A1 (ja) 2022-10-11 2022-10-11 内視鏡システム、画像生成装置および画像生成方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/175,263 Continuation US20250235090A1 (en) 2022-10-11 2025-04-10 Endoscope system, image generation device, and image generation method

Publications (1)

Publication Number Publication Date
WO2024079785A1 true WO2024079785A1 (ja) 2024-04-18

Family

ID=90668986

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/037862 Ceased WO2024079785A1 (ja) 2022-10-11 2022-10-11 内視鏡システム、画像生成装置および画像生成方法

Country Status (4)

Country Link
US (1) US20250235090A1 (https=)
JP (1) JPWO2024079785A1 (https=)
CN (1) CN120018808A (https=)
WO (1) WO2024079785A1 (https=)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0564628A (ja) * 1991-09-09 1993-03-19 Olympus Optical Co Ltd 内視鏡用写真撮影装置
JP2010004319A (ja) * 2008-06-20 2010-01-07 Hoya Corp 撮像装置
WO2010131620A1 (ja) * 2009-05-14 2010-11-18 オリンパスメディカルシステムズ株式会社 撮像装置
WO2014156253A1 (ja) * 2013-03-25 2014-10-02 オリンパスメディカルシステムズ株式会社 内視鏡装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0564628A (ja) * 1991-09-09 1993-03-19 Olympus Optical Co Ltd 内視鏡用写真撮影装置
JP2010004319A (ja) * 2008-06-20 2010-01-07 Hoya Corp 撮像装置
WO2010131620A1 (ja) * 2009-05-14 2010-11-18 オリンパスメディカルシステムズ株式会社 撮像装置
WO2014156253A1 (ja) * 2013-03-25 2014-10-02 オリンパスメディカルシステムズ株式会社 内視鏡装置

Also Published As

Publication number Publication date
JPWO2024079785A1 (https=) 2024-04-18
CN120018808A (zh) 2025-05-16
US20250235090A1 (en) 2025-07-24

Similar Documents

Publication Publication Date Title
US10335014B2 (en) Endoscope system, processor device, and method for operating endoscope system
JP5271062B2 (ja) 内視鏡装置およびその作動方法
JPWO2013175908A1 (ja) 撮像システム
JP2015195845A (ja) 内視鏡システム、内視鏡システムの作動方法、プロセッサ装置、プロセッサ装置の作動方法
CN107072506B (zh) 摄像装置和处理装置
US12426771B2 (en) Endoscope system, image processing device, total processing time detection method, and processing device
JP6430880B2 (ja) 内視鏡システム、及び、内視鏡システムの作動方法
CN112118778A (zh) 内窥镜系统
JP2012239757A (ja) 医療機器及び医療用プロセッサ
JP5747362B2 (ja) 内視鏡装置
WO2016104386A1 (ja) 調光装置、撮像システム、調光装置の作動方法および調光装置の作動プログラム
US9838610B2 (en) Imaging system
JP5422180B2 (ja) 内視鏡装置およびその作動方法
US10462440B2 (en) Image processing apparatus
JP6285373B2 (ja) 内視鏡システム、プロセッサ装置、及び内視鏡システムの作動方法
CN114269221B (zh) 医疗图像处理装置、内窥镜系统、医疗图像处理方法以及记录介质
WO2024079785A1 (ja) 内視鏡システム、画像生成装置および画像生成方法
JP6120758B2 (ja) 医用システム
US12035052B2 (en) Image processing apparatus and image processing method
JP6396717B2 (ja) 感度調整方法および撮像装置
JP2010042133A (ja) 画像取得方法および装置
JP7524307B2 (ja) 内視鏡システム、制御方法、及び制御プログラム
WO2022210508A1 (ja) プロセッサ装置、医療画像処理装置、及び医療画像処理システム、並びに内視鏡システム
JP2010124921A (ja) 画像取得方法および装置
WO2021065286A1 (ja) 電子内視鏡システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22962006

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024550941

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202280100911.4

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 202280100911.4

Country of ref document: CN

122 Ep: pct application non-entry in european phase

Ref document number: 22962006

Country of ref document: EP

Kind code of ref document: A1