WO2021033216A1 - 内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法 - Google Patents

内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法 Download PDF

Info

Publication number
WO2021033216A1
WO2021033216A1 PCT/JP2019/032134 JP2019032134W WO2021033216A1 WO 2021033216 A1 WO2021033216 A1 WO 2021033216A1 JP 2019032134 W JP2019032134 W JP 2019032134W WO 2021033216 A1 WO2021033216 A1 WO 2021033216A1
Authority
WO
WIPO (PCT)
Prior art keywords
system information
information
learning model
output
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/032134
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
健太 小杉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hoya Corp
Original Assignee
Hoya Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hoya Corp filed Critical Hoya Corp
Priority to ES19941927T priority Critical patent/ES3024141T3/es
Priority to EP19941927.6A priority patent/EP4014831B1/en
Priority to US17/634,389 priority patent/US12390089B2/en
Priority to PCT/JP2019/032134 priority patent/WO2021033216A1/ja
Priority to JP2021541340A priority patent/JP7116849B2/ja
Publication of WO2021033216A1 publication Critical patent/WO2021033216A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00057—Operational features of endoscopes provided with means for testing or calibration
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00055—Operational features of endoscopes provided with output arrangements for alerting the user
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/40—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30028—Colon; Small intestine
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30092—Stomach; Gastric

Definitions

  • the present invention relates to an endoscope processor, an endoscope system, an information processing device, a program, and an information processing method.
  • Patent Document 1 discloses an image processing device that prevents misalignment between a mark indicating a lesion portion attached to an endoscopic image and an endoscopic image.
  • Patent Document 1 when a system abnormality that affects the image quality occurs, the cause of the abnormality (for example, image setting, optical system, light source, electric circuit, etc.) may not be correctly identified. There is.
  • One aspect is to provide an endoscope processor or the like that can identify the cause of a system abnormality using a learned learning model.
  • the endoscope processor has an image acquisition unit that acquires an endoscopic image taken by using the first system information and parameters based on the endoscopic image acquired by the image acquisition unit.
  • a calculation unit that calculates, a first learning model that outputs an identification result that identifies a part of a subject when a parameter calculated by the calculation unit is input, a parameter calculated by the calculation unit, and the first learning. Determine the difference between the second learning model that outputs the second system information when the identification result output by the model is input, the second system information output by the second learning model, and the first system information. It is characterized by including a determination unit.
  • FIG. 1 is a schematic view showing a configuration example of an endoscope system.
  • the system shown in FIG. 1 is inserted into the body of a subject to take an image, and the endoscope 1 outputs an image signal to be observed, and the image signal output by the endoscope 1 is converted into an endoscope image.
  • It includes a processor 2 for an endoscope and a display device 3 for displaying an endoscopic image or the like. Each device transmits and receives electric signals, video signals, etc. via a connector.
  • the endoscope 1 is an instrument for diagnosing or treating by inserting an insertion part having an image sensor at the tip into the body of a subject.
  • the endoscope 1 transfers a captured image captured by the image sensor at the tip to the processor 2.
  • the endoscope processor 2 is an information processing device that performs image processing on an image captured from an imaging element at the tip of the endoscope 1, generates an endoscope image, and outputs the endoscopic image to the display device 3. is there. Further, in the following, for the sake of brevity, the endoscope processor 2 will be read as the processor 2.
  • the display device 3 is a liquid crystal display, an organic EL (electroluminescence) display, or the like, and displays an endoscopic image or the like output from the processor 2.
  • FIG. 2 is an external view of the endoscope 1.
  • the endoscope 1 includes an image sensor 11, a treatment tool insertion channel 12, an operation unit 13, and a connector 14.
  • the image sensor 11 is, for example, a CCD (Charge Coupled Device) image sensor, a CMD (Charge Modulation Device) image sensor, or a CMOS (Complementary Metal Oxide Semiconductor) image sensor installed at the tip of the endoscope 1, and is an incident light. Is photoelectrically converted. The electric signal generated by the photoelectric conversion is subjected to signal processing such as A / D conversion and noise removal by a signal processing circuit (not shown), and is output to the processor 2.
  • signal processing such as A / D conversion and noise removal by a signal processing circuit (not shown), and is output to the processor 2.
  • the treatment tool insertion channel 12 is a channel for passing the treatment tool.
  • Treatment tools include, for example, grippers, biopsy needles, forceps, snares, clamps, scissors, scalpels, incision instruments, endoscopic staplers, tissue loops, clip pliers, suture delivery instruments, or energy-based tissue coagulation instruments or tissue cutting. It is an instrument.
  • the operation unit 13 is provided with a release button, an angle knob for bending the tip of the endoscope, and the like, and receives input of operation instruction signals of peripheral devices such as air supply, water supply, and gas supply.
  • the connector 14 is connected to the processor 2.
  • FIG. 3 is a block diagram showing a configuration example of the processor 2.
  • the processor 2 includes a control unit 21, a storage unit 22, an operation input unit 23, an output unit 24, a light source control unit 25, a reading unit 26, a large-capacity storage unit 27, a light source 28, and a communication unit 29. Each configuration is connected by bus B.
  • the control unit 21 includes arithmetic processing units such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit), and reads and executes a control program 2P stored in the storage unit 22. Performs various information processing, control processing, and the like related to the processor 2. Although the control unit 21 is described as a single processor in FIG. 3, it may be a multiprocessor.
  • arithmetic processing units such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit)
  • CPU Central Processing Unit
  • MPU Micro-Processing Unit
  • GPU Graphics Processing Unit
  • the storage unit 22 includes memory elements such as RAM (RandomAccessMemory) and ROM (ReadOnlyMemory), and stores the control program 2P or data required for the control unit 21 to execute the process. In addition, the storage unit 22 temporarily stores data and the like necessary for the control unit 21 to execute arithmetic processing.
  • the operation input unit 23 is composed of input devices such as a touch panel and various switches, and inputs an input signal generated in response to an external operation on these input devices to the control unit 21. Under the control of the control unit 21, the output unit 24 outputs an image signal for display and various types of information to the display device 3 to display the image and information.
  • the light source control unit 25 controls the amount of light emitted from the illumination light by turning on / off the LED and the like, adjusting the drive current and the drive voltage of the LED and the like. Further, the light source control unit 25 controls the wavelength band of the illumination light by changing the optical filter or the like. The light source control unit 25 adjusts the emission timing, emission period, light amount, and spectral spectrum of the illumination light by independently controlling the lighting and extinguishing of each LED and the amount of light emitted at the time of lighting.
  • the reading unit 26 reads a portable storage medium 2a including a CD (Compact Disc) -ROM or a DVD (Digital Versatile Disc) -ROM.
  • the control unit 21 may read the control program 2P from the portable storage medium 2a via the reading unit 26 and store it in the large-capacity storage unit 27. Further, the control unit 21 may download the control program 2P from another computer via the network N or the like and store it in the large-capacity storage unit 27. Furthermore, the control unit 21 may read the control program 2P from the semiconductor memory 2b.
  • the large-capacity storage unit 27 includes a recording medium such as an HDD (Hard disk drive) or SSD (Solid State Drive).
  • the large-capacity storage unit 27 stores a site identification model (first learning model) 271, a system information output model (second learning model) 272, a system information DB (database) 273, and a threshold value DB 274.
  • the site identification model 271 is a site classifier that identifies the site of the subject, and is a trained model generated by machine learning.
  • the site of the subject may be, for example, the mouth, esophagus, stomach, small intestine, large intestine, or the like.
  • the system information output model 272 is an output device that outputs system information, and is a trained model generated by machine learning.
  • the system information DB 273 stores various system information for setting the system.
  • the threshold value DB 274 stores threshold values of various system information.
  • the site identification model 271 and the system information output model 272 may be arranged and used in a cloud computing system connected via a network.
  • the storage unit 22 and the large-capacity storage unit 27 may be configured as an integrated storage device. Further, the large-capacity storage unit 27 may be composed of a plurality of storage devices. Furthermore, the large-capacity storage unit 27 may be an external storage device connected to the processor 2.
  • the light source 28 includes a light source that emits illumination light used for illuminating the observation target.
  • the light source 28 is, for example, a semiconductor light source such as a plurality of colors of LEDs (Light Emitting Diodes) having different wavelength ranges, a combination of a laser diode and a phosphor, a xenon lamp, a halogen lamp, or the like.
  • the light used to illuminate the observation target is guided to the tip of the endoscope 1 by an optical fiber.
  • the light source may be installed at the tip of the endoscope.
  • the light source 28 adjusts the brightness and the like according to the control from the light source control unit 25 of the processor 2.
  • the processor 2 is a light source integrated type, but the present invention is not limited to this.
  • the processor 2 may be a light source separation type that is separated from the light source device.
  • the communication unit 29 is a communication module for performing processing related to communication, and transmits / receives information to / from an external information processing device or the like via a network N.
  • FIG. 4 is an explanatory diagram showing an example of the record layout of the system information DB 273.
  • the system information DB 273 is a database that stores the management ID and the system information in association with each other.
  • the system information includes setting information such as strength, brightness (luminance) or enhancement mode of a color (for example, red or blue) for setting an endoscopic image. Further, the system information includes setting information of the lamp diaphragm for controlling the brightness of the illumination light and voltage or current to the lamp. These are examples of system information.
  • the system information DB 273 includes a management ID column, an image setting column, a lamp diaphragm array, and a voltage / current array.
  • the management ID column stores the ID of the management number uniquely specified in order to identify the management number for managing each system information.
  • the image setting column includes a red column, a blue column, a brightness column, and a highlight column.
  • the red column stores the values set for the intensity of red in the endoscopic image.
  • the blue column stores the values set for the intensity of blue in the endoscopic image.
  • the luminance column stores information that sets the luminance (brightness) of the endoscopic image. For example, when the brightness is set to 5 levels, "level1", “level2", “level3”, "level4" or "level5" may be stored in the brightness sequence.
  • the emphasis column stores the setting mode for performing the enhancement process of the endoscopic image for the structure, color, etc.
  • the setting mode may be "Off", “Low”, “Med”, “High”, or the like.
  • the lamp diaphragm array stores information for controlling the brightness of the illumination light.
  • the voltage / current sequence stores the voltage or current to the lamp.
  • FIG. 5 is an explanatory diagram showing an example of the record layout of the threshold value DB 274.
  • the threshold value DB 274 includes an item ID column, a category column, an item column, and a threshold value column.
  • the item ID column stores the ID of the item uniquely specified in order to identify each item.
  • the category column stores item type information.
  • the item column stores the name of the item.
  • the threshold value column stores the threshold value of the item.
  • FIG. 6 is an explanatory diagram illustrating a process of monitoring a system abnormality.
  • the control unit 21 of the processor 2 acquires an endoscope image taken by using the first system information from the endoscope 1.
  • the first system information is the system information set in the endoscopic system in use (currently).
  • the user can change the first system information by operating a keyboard or the like connected to the processor 2.
  • the control unit 21 of the processor 2 stores the first system information in the system information DB 273 of the large-capacity storage unit 27. Since the items included in the first system information are the same as the items included in the system information described above, the description thereof will be omitted.
  • the control unit 21 calculates the parameters based on the acquired endoscopic image.
  • the parameters include color tone parameters, brightness parameters, spatial frequency parameters or noise amount parameters of the endoscopic image.
  • the color tone parameter may be, for example, a value obtained by averaging the pixel values of R, G, or B of each pixel constituting the endoscopic image within the entire screen or a predetermined range in the screen, or the pixel value in the image. It may be the frequency of occurrence of pixel values based on a histogram showing the overall distribution of. R is red, G is green, and B is the pixel value of the blue sub-pixel.
  • the luminance parameter may be, for example, the luminance in each pixel, that is, ((R + G + B) / 3), or the number of pixels corresponding to each luminance value based on the luminance histogram showing the luminance distribution of the pixels in the image. And the degree of bias of the distribution may be used.
  • the spatial frequency parameter may be, for example, the frequency distribution of the image data obtained by the Fourier transform.
  • Spatial frequency represents the number of repetitions of the pattern included in the unit length.
  • the spatial frequency represents the number of repetitions of sinusoidal shading changes per unit length for a two-dimensional image. In this case, the spatial frequency becomes high at a place where the light and shade changes rapidly, and becomes low at a place where the light and shade changes slowly.
  • the noise amount parameter is the amount of image noise and is represented by the standard deviation (SD: Standard Deviation), which is the square root of the variance.
  • SD Standard Deviation
  • the image noise is a high-frequency component having a high spatial frequency among the non-uniform brightness generated in the captured image.
  • the standard deviation is represented by a value that indicates how the data is scattered.
  • the control unit 21 acquires the model information of the endoscope 1.
  • the model information includes the series and model number of the endoscope, the number of pixels of the image sensor, the target site information (for example, the upper gastrointestinal tract), and the like.
  • the control unit 21 acquires model information from the endoscope 1 (scope).
  • the control unit 21 acquires the model number from the endoscope 1.
  • the control unit 21 may acquire model information corresponding to the acquired model number from the storage unit 22.
  • the control unit 21 uses a site identification model 271 that outputs an identification result that identifies the site of the subject when the parameters calculated based on the endoscopic image and the acquired model information are input. Identify the site.
  • the site identification process will be described later.
  • the control unit 21 uses a system information output model 272 that outputs the second system information when the parameters calculated based on the endoscopic image, the acquired model information, and the identified part of the subject are input. Acquires (outputs) the second system information. Since the items included in the second system information are the same as the items included in the first system information, the description thereof will be omitted. The second system information acquisition process will be described later.
  • the control unit 21 acquires the first system information from the system information DB 273 of the large-capacity storage unit 27.
  • the control unit 21 compares the acquired first system information with the second system information to determine the difference. Specifically, the control unit 21 obtains the first system information with respect to the red intensity, the blue intensity, the brightness setting information, the emphasis mode setting information, the lamp aperture setting information, and the voltage or current to the lamp of the image. Each item is compared with each item of the corresponding second system information.
  • control unit 21 determines that the two match, it determines that no abnormality has been detected. When it is determined that the two do not match, the control unit 21 determines whether or not the system settings can be changed. Specifically, the control unit 21 acquires the threshold value of system information from the threshold value DB 274 of the large-capacity storage unit 27. The control unit 21 determines whether or not each item of the second system information is within the threshold value based on the acquired threshold value.
  • the control unit 21 When it is determined that each item or a part of the items of the second system information is out of the threshold range, the control unit 21 outputs a message including the fact that the difference has been detected to the display device 3.
  • the control unit 21 determines that each item of the second system information is within the threshold range, the control unit 21 changes the system setting by using the first system information. For example, as an example in which the red strength of the first system information is set to "2”, when the control unit 21 determines that the red strength of the second system information is "3", the red strength of the system is set. To "2".
  • control unit 21 determines that the value of the lamp diaphragm of the second system information is lower than the value of the lamp diaphragm of the first system information, an abnormality in the light source system causes a loss in the amount of light from the normal state. there is a possibility.
  • the control unit 21 changes the value of the lamp diaphragm of the system to the value of the lamp diaphragm of the first system information.
  • control unit 21 determines that the system setting change is successful, the control unit 21 outputs a message including the fact that the change is successful to the display device 3. Before changing the system settings, a confirmation message for changing the settings may be output to the user (doctor). In this case, the system setting change is executed with the consent of the user. When the control unit 21 determines that the system setting change has failed, the control unit 21 outputs a message including the change failure to the display device 3.
  • FIG. 7 is an explanatory diagram illustrating the site identification model 271.
  • the site identification model 271 is used as a program module that is a part of artificial intelligence software.
  • the site identification model 271 is a classifier for which a neural network has been constructed (generated) in which parameters calculated based on an endoscopic image and model information are input and the result of predicting the site of a subject is output. ..
  • the neural network is, for example, a CNN (Convolutional Neural Network), which is an input layer that accepts inputs of parameters and model information calculated based on an endoscopic image, and an output layer that outputs a result of predicting a part of a subject. And an intermediate layer that has been trained by backpropagation.
  • Each layer has one or more neurons (nodes), and each neuron has a value. Then, neurons between one layer and the next layer are connected by edges, and each edge has variables (or parameters) such as weights and biases.
  • the value of the neuron in each layer is obtained by executing a predetermined operation based on the value of the neuron in the previous layer and the weight of the edge. Then, when the input data is input to the neurons of the input layer, the values of the neurons of the next layer are obtained by a predetermined calculation, and further, the values of the neurons of the next layer are obtained by using the data obtained by the calculation as an input. It is obtained by a predetermined calculation of the layer. Then, the value of the neuron in the output layer, which is the final layer, becomes the output data with respect to the input data.
  • the site identification model 271 will be described as a CNN, but the site identification model 271 is not limited to the CNN, and is a neural network other than the CNN, R-CNN (Regions with Convolutional Neural Networks), SVM. It may be a trained model constructed by any learning algorithm such as (Support Vector Machine), Bayesian network, or regression tree.
  • the control unit 21 compares the identification result output from the output layer with the labeled information of the part with respect to the teacher data, that is, the correct answer value, and the intermediate layer so that the output value from the output layer approaches the correct answer value.
  • the teacher data is data generated by associating the name of the part (for example, the large intestine) with the parameters calculated based on the endoscope image and the model information of the endoscope 1.
  • the variable is, for example, a weight between neurons (coupling coefficient), a coefficient of an activation function used in each neuron, and the like.
  • the method of optimizing the variables is not particularly limited, but for example, the control unit 21 optimizes various variables by using the backpropagation method.
  • the control unit 21 performs the above processing on the parameters and model information included in the teacher data, and generates the part identification model 271.
  • the generation process of the site identification model 271 is not limited to the above-mentioned process.
  • the control unit 21 may generate a site identification model for each model of endoscope.
  • a large intestine identification model for identifying the large intestine may be generated.
  • the site identification model 271 is generated by the processor 2
  • the present invention is not limited to this.
  • the site identification model 271 may be generated by an external device (for example, a server or the like).
  • control unit 21 of the processor 2 may download and install the part identification model 271 generated by the external device by the communication unit 29. Further, the control unit 21 may read and install the site identification model 271 generated by the external device from the portable storage medium 2a or the semiconductor memory 2b via the reading unit 26. The update process of the site identification model 271 may be performed by the processor 2 or by an external device.
  • the control unit 21 acquires an endoscope image from the endoscope 1, the control unit 21 identifies the site of the subject using the site identification model 271.
  • the input layer of the site identification model 271 accepts inputs of the parameters "x1 to xn" calculated based on the endoscopic image and the model information "x (n + 1)".
  • x1 to xn indicate the color tone parameter, the luminance parameter, the spatial frequency parameter, or the noise amount parameter of the endoscopic image described above.
  • x (n + 1) indicates model information including the endoscope series, model number, number of pixels of the image sensor, or target site information.
  • the average value of each pixel value of R, G or B of each pixel constituting the endoscopic image, the degree of bias of the distribution based on the histogram, or the spatial frequency is input to the part identification model 271. If this is the case, it will have a great effect on the identification result of the site of the subject.
  • the intermediate layer extracts the features of the input information by changing the number of dimensions of the input information input from the input layer.
  • the intermediate layer is then predicted to be at each site of the subject according to the extracted features by the fully connected layer whose parameters have been learned by backpropagation.
  • the prediction result is output to the output layer having a plurality of neurons. As shown, the probability value in the mouth is 0.02, the probability value in the esophagus is 0.03, the probability value in the stomach is 0.02, the probability value in the small intestine is 0.03, and The prediction result that the probability value of the large intestine is 0.9 is output.
  • the part identification process is not limited to the process of identifying the part by the above-mentioned machine learning.
  • the control unit 21 of the processor 2 uses a local feature extraction method such as A-KAZE (Accelerated KAZE) or SIFT (Scale Invariant Feature Transform) based on the change in color or fold of each part from the endoscopic image.
  • the site may be identified.
  • the control unit 21 of the processor 2 may accept the identification result of the doctor identifying the site of the subject based on the medical expertise by the operation input unit 23.
  • FIG. 8 is an explanatory diagram illustrating the system information output model 272.
  • the system information output model 272 is used as a program module that is a part of artificial intelligence software.
  • the system information output model 272 inputs the parameters calculated based on the endoscopic image, the model information and the part of the subject (part identification result) output from the part identification model 271, and the second system information.
  • This is an output device for which a neural network has been constructed (generated) that outputs the result of predicting.
  • the neural network is CNN
  • system information output model 272 is described as being a CNN, but the system information output model 272 is not limited to the CNN, and is a neural network other than the CNN, R-CNN, SVM, Bayesian network, or. It may be a trained model constructed by an arbitrary learning algorithm such as a regression tree.
  • the control unit 21 compares the prediction result output from the output layer with the labeled information of each item of the system information, that is, the correct answer value with respect to the teacher data, so that the output value from the output layer approaches the correct answer value.
  • the variables used for arithmetic processing in the intermediate layer are optimized.
  • the teacher data is data generated by associating each item of system information with the parameters calculated based on the endoscope image, the model information of the endoscope 1, and the part of the subject.
  • the control unit 21 performs the above processing on the parameters and various information included in the teacher data, and generates the system information output model 272.
  • the system information output model 272 generation process is not limited to the above-mentioned process.
  • the control unit 21 may generate a system information output model for each model of the endoscope, or may generate a system information output model for each part of the subject.
  • a system information output model may be generated for each item of system information. For example, a color intensity determination model for determining the intensity of red or blue of an image, a luminance determination model for determining brightness, or the like may be generated.
  • system information output model 272 is generated by the processor 2
  • the present invention is not limited to this.
  • the system information output model 272 may be generated by an external device.
  • the control unit 21 acquires the site of the subject using the site identification model 271
  • the control unit 21 acquires the second system information using the system information output model 272.
  • the input layer of the system information output model 272 was output from the parameters "x1 to xn" calculated based on the endoscopic image, the model information "x (n + 1)", and the site identification model 271. Accepts the input of the part "x (n + 2)" of the subject. Since x1 to xn and x (n + 1) are the same as the above-mentioned input information, the description thereof will be omitted.
  • x (n + 2) indicates the site of the subject (for example, the large intestine).
  • the output result of the second system information is Has a great impact.
  • the intermediate layer extracts the features of the input information by changing the number of dimensions of the input information input from the input layer. After that, the intermediate layer predicts the probability of each item of the second system information according to the extracted features by the fully connected layer whose parameters are learned by backpropagation. The prediction result is output to the output layer having a plurality of neurons. As shown in the figure, the prediction result corresponding to the highest probability in each item of the second system information is output. Not limited to the output result described above, all probability values of each item of system information may be output.
  • the probability value of the combination of each item of the system information may be output.
  • a probability value of "red intensity: 3, blue intensity: 2, brightness: Level 2, lamp diaphragm: 128, voltage to lamp: 100v" may be output.
  • the combination corresponding to the highest probability among the probability values of the combinations may be output as the prediction result.
  • the control unit 21 of the processor 2 acquires an endoscope image taken by using the first system information from the endoscope 1 (step S201).
  • the control unit 21 calculates the parameters based on the acquired endoscopic image (step S202).
  • the control unit 21 acquires model information including a series of endoscopes, a model number, the number of pixels of an image sensor, target site information, and the like stored in the endoscope 1 in advance (step S203).
  • the control unit 21 uses a site identification model 271 that outputs an identification result that identifies the site of the subject when the parameters calculated based on the endoscopic image and the acquired model information are input. Identify the site (step S204).
  • the control unit 21 is a system information output model that outputs the second system information when the parameters calculated based on the endoscopic image, the acquired model information, and the part of the subject identified by the part identification model 271 are input.
  • the second system information is acquired by using 272 (step S205).
  • the control unit 21 transmits the red intensity, the blue intensity, the brightness setting information, the emphasis mode setting information, the lamp aperture setting information, and the voltage or current to the lamp from the system information DB 273 of the large-capacity storage unit 27. Acquire the first system information including the first system information (step S206). The control unit 21 compares each item of the acquired first system information with each item of the corresponding second system information to determine the difference (step S207).
  • control unit 21 determines that there is no difference between the first system information and the second system information (NO in step S207).
  • the control unit 21 returns to step S201.
  • the control unit 21 determines that there is a difference between the first system information and the second system information (YES in step S207)
  • the control unit 21 acquires the threshold value of the system information from the threshold value DB 274 of the large capacity storage unit 27 (step S208). ..
  • the control unit 21 determines whether or not each item of the second system information is within the threshold value based on the acquired threshold value (step S209).
  • the control unit 21 When it is determined that each item or a part of the items of the second system information is out of the threshold range (NO in step S209), the control unit 21 outputs an abnormal message including the fact that the difference has been detected to the display device 3. (Step S210).
  • the display device 3 displays an abnormal message output from the processor 2 (step S301).
  • the control unit 21 of the processor 2 determines that each item of the second system information is within the threshold range (YES in step S209), the control unit 21 changes the system setting using the first system information (step S211). ..
  • the control unit 21 determines whether or not the system setting change has been successful (step S212). When the control unit 21 determines that the system setting change has not been successful (NO in step S212), the control unit 21 outputs a message including the change failure to the display device 3 (step S213). The display device 3 displays a message output from the processor 2 to the effect that the change has failed (step S302).
  • control unit 21 of the processor 2 determines that the system setting change is successful (YES in step S212)
  • the control unit 21 outputs a message including the fact that the change is successful to the display device 3 (step S214).
  • the control unit 21 returns to step S201.
  • the display device 3 displays a message output from the processor 2 to the effect that the change is successful (step S303).
  • FIG. 11A and 11B are schematic views of an endoscopic image displaying a message on the display device 3.
  • Reference numeral 3a is an observation screen (area) of the endoscope.
  • Reference numeral 3b is an area for displaying a message.
  • the display device 3 displays the endoscopic image output from the processor 2 on 3a, and displays the message output from the processor 2 on 3b.
  • the notification display screen is not limited to the layout described above.
  • the message may be displayed on the observation screen 3a of the endoscope.
  • FIG. 11A shows an example in which, when the system setting change is successful, a message including the fact that the change is successful is displayed.
  • FIG. 11B shows an example in which, when the system setting change fails, a message including the change failure is displayed.
  • the learning model by monitoring the system abnormality using the learning model, it is possible to infer the factors that affect the image quality, so that the system settings can be changed according to the inferred factors. ..
  • the second embodiment relates to a form for identifying a site of a subject using an image feature amount.
  • the description of the contents overlapping with the first embodiment will be omitted.
  • the image feature amount is a numerical value of the image geometry of each region on the image, the image value, and the feature parameter calculated from these, and is obtained by applying an image processing method.
  • the image feature amount may be the average value of the pixel values of R, G, or B of each pixel constituting the image, the average value of the brightness, the degree of bias of the brightness, and the like.
  • the identification result for identifying the site of the subject is output, but the present invention is not limited to this.
  • a process of outputting an identification result that identifies a part of a subject when an endoscopic image taken by using the first system information is directly input to the part identification model 271 will be described.
  • FIG. 12 is an explanatory diagram illustrating the site identification model 271 of the second embodiment.
  • the site identification model 271 is a classifier for which a neural network has been constructed that inputs an endoscopic image taken by using the first system information and outputs a result of predicting a site of a subject.
  • the neural network is, for example, a CNN, and has an input layer that accepts input of an endoscopic image, an output layer that outputs a result of predicting a part of a subject, and an intermediate layer that has been learned by backpropagation.
  • the input layer has a plurality of neurons that accept the input of the pixel value of each pixel included in the endoscopic image, and passes the input pixel value to the intermediate layer.
  • the intermediate layer has a plurality of neurons for extracting the image features of the endoscopic image, and the extracted image features are passed to the output layer.
  • the intermediate layer has an endoscopic image in which a convolution layer that convolves the pixel values of each pixel input from the input layer and a pooling layer that maps the pixel values convoluted by the convolution layer are alternately connected. Finally, the feature amount of the image is extracted while compressing the pixel information of.
  • the intermediate layer then predicts the probability that the endoscopic image is at each site of the subject by the fully connected layer whose parameters have been learned by backpropagation. The prediction result is output to the output layer having a plurality of neurons.
  • the endoscopic image may be input to the input layer after the feature amount is extracted by passing through the convolution layer and the pooling layer which are alternately connected.
  • the image input to the site identification model 271 is not limited to the endoscopic image.
  • the control unit 21 shows a histogram image showing the overall distribution of pixel values in the image, a brightness histogram image showing the distribution of the brightness of the pixels in the image, or a spatial frequency based on the endoscopic image. Generate a graph image or the like.
  • the control unit 21 inputs the generated graph image into the site identification model 271 that has been learned by deep learning using the graph image included in the teacher data, and outputs the identification result of identifying the site of the subject.
  • the input layer of the system information output model 272 receives the input of the pixel value of each pixel included in the endoscopic image and the part of the subject output from the part identification model 271 and receives the input to the intermediate layer. hand off.
  • the intermediate layer extracts the image feature amount of the endoscopic image from the pixel value of each passed pixel.
  • the intermediate layer predicts the probability of each item of system information based on the part of the subject and the extracted image feature amount, and outputs the prediction result to the output layer.
  • the third embodiment relates to a mode in which the information processing device 4 monitors a system abnormality by using artificial intelligence.
  • the description of the contents overlapping with the first and second embodiments will be omitted.
  • the part identification process and the second system information output process are performed by the processor 2 using the learning model, but in the present embodiment, the above-described process is performed by the information processing device 4. ..
  • the information processing device 4 is an information processing device that constructs a learning model, determines a system abnormality using the learning model, processes various information, stores, and transmits / receives.
  • the information processing device 4 is, for example, a server device, a personal computer, a general-purpose tablet PC (personal computer), or the like.
  • FIG. 13 is a schematic view showing a configuration example of the endoscope system of the third embodiment.
  • the contents overlapping with FIGS. 1 and 3 are designated by the same reference numerals and the description thereof will be omitted.
  • the system shown in FIG. 13 includes an endoscope 1, a processor 2, a display device 3, and an information processing device 4. Each device transmits and receives electric signals, video signals, etc. via a connector.
  • the processor 2 is captured by using the first system information set by the endoscope system in use, the model information of the endoscope stored in the endoscope 1 in advance, and the first system information. Acquire an endoscopic image.
  • the processor 2 outputs the acquired first system information, model information, and endoscopic image to the information processing device 4.
  • the control unit 21 of the information processing device 4 calculates the parameters based on the endoscopic image output from the processor 2.
  • the control unit 21 identifies the part of the subject by using the part identification model 271 that outputs the identification result of identifying the part of the subject when the parameters calculated based on the endoscopic image and the model information are input. To do. Since the site identification process is the same as that of the first or second embodiment, the description thereof will be omitted.
  • the control unit 21 outputs a system information output model 272 that outputs the second system information when the parameters calculated based on the endoscopic image and the model information and the part of the subject identified by the part identification model 271 are input. It is used to acquire the second system information. Since the second system information acquisition process is the same as that of the first or second embodiment, the description thereof will be omitted.
  • the control unit 21 compares each item of the first system information with each item of the corresponding second system information to determine the difference. When the control unit 21 determines that there is a difference between the first system information and the second system information, the control unit 21 acquires the threshold value of the system information from the threshold value DB 274 of the large-capacity storage unit 27. The control unit 21 determines whether or not each item of the second system information is within the threshold value based on the acquired threshold value.
  • the control unit 21 When it is determined that each item or a part of the items of the second system information is out of the threshold range, the control unit 21 outputs an abnormal message including the fact that the difference has been detected to the display device 3. When the control unit 21 determines that each item of the second system information is within the threshold range, the control unit 21 outputs a notification of the system setting change to the processor 2. The processor 2 changes the system settings by using the first system information in response to the notification of the system setting change output from the information processing device 4.
  • the processor 2 outputs the result (for example, success or failure) of the system setting change to the information processing device 4.
  • the control unit 21 of the information processing device 4 outputs an appropriate message to the display device 3 according to the result of the setting change output from the processor 2.
  • the 14 and 15 are flowcharts showing a processing procedure when monitoring a system abnormality with the information processing device 4.
  • the processor 2 acquires the first system information stored in the endoscope system during normal use (step S221).
  • the control unit 21 acquires the model information of the endoscope stored in the endoscope 1 in advance (step S222).
  • the processor 2 acquires an endoscope image taken by using the first system information from the endoscope 1 (step S223).
  • the processor 2 outputs the acquired first system information, model information, and endoscopic image to the information processing device 4 (step S224).
  • the processor 2 returns to step S221.
  • the control unit 21 of the information processing device 4 stores the first system information output from the processor 2 in the system information DB 273 of the large-capacity storage unit 27 (step S421).
  • the processor 2 outputs the first system information and the model information to the information processing device 4, but the present invention is not limited to this.
  • the first system information and the model information may be stored in advance in the storage unit 22 or the large-capacity storage unit 27 of the information processing device 4.
  • the control unit 21 of the information processing device 4 calculates the parameters based on the endoscopic image output from the processor 2 (step S422).
  • the control unit 21 uses a site identification model 271 that outputs an identification result that identifies the site of the subject when the parameters calculated based on the endoscopic image and the model information output from the processor 2 are input. , Identify the site of the subject (step S423).
  • the control unit 21 outputs a system information output model 272 that outputs the second system information when the parameters calculated based on the endoscopic image and the model information and the part of the subject identified by the part identification model 271 are input.
  • the second system information is acquired by using (step S424).
  • the control unit 21 compares each item of the first system information with each item of the corresponding second system information and determines the difference (step S425). When the control unit 21 determines that there is no difference between the first system information and the second system information (NO in step S425), the control unit 21 returns to step S421. When the control unit 21 determines that there is a difference between the first system information and the second system information (YES in step S425), the control unit 21 acquires the threshold value of the system information from the threshold value DB 274 of the large-capacity storage unit 27 (step S426). ..
  • the control unit 21 determines whether or not each item of the second system information is within the threshold value based on the acquired threshold value (step S427). When it is determined that each item or a part of the items of the second system information is out of the threshold range (NO in step S427), the control unit 21 outputs an abnormal message including the fact that the difference has been detected to the display device 3. (Step S428).
  • control unit 21 determines that each item of the second system information is within the threshold range (YES in step S427), the control unit 21 outputs a notification of the system setting change to the processor 2 (step S429).
  • the processor 2 changes the system setting by using the first system information in response to the notification of the system setting change output from the information processing device 4 (step S225).
  • the processor 2 determines whether or not the system setting change has been successful (step S226). When the processor 2 determines that the system setting change has not been successful (NO in step S226), the processor 2 outputs a notification of the change failure to the information processing device 4 (step S227).
  • the control unit 21 of the information processing device 4 outputs a message including the change failure to the display device 3 in response to the change failure notification output from the processor 2 (step S430).
  • the processor 2 determines that the system setting change is successful (YES in step S226)
  • the processor 2 outputs a notification of the change success to the information processing device 4 (step S228).
  • the control unit 21 of the information processing device 4 outputs a message including the change success to the display device 3 in response to the change success notification output from the processor 2 (step S431).
  • FIG. 16 is a functional block diagram showing the operation of the processors 2 of the first and second embodiments.
  • the control unit 21 executes the control program 2P
  • the processor 2 operates as follows.
  • the functional block diagram showing the operation is similarly applied to the information processing device 4 of the third embodiment.
  • the image acquisition unit 20a acquires an endoscopic image taken using the first system information.
  • the calculation unit 20b calculates the parameters based on the endoscopic image acquired by the image acquisition unit 20a.
  • the first learning model 20c outputs an identification result that identifies the site of the subject when the parameter calculated by the calculation unit 20b is input.
  • the second learning model 20d outputs the second system information when the parameters calculated by the calculation unit 20b and the identification result output by the first learning model 20c are input.
  • the determination unit 20e determines the difference between the second system information output by the second learning model 20d and the first system information.
  • the change unit 20f changes the system settings based on the determination result determined by the determination unit 20e.
  • the model information acquisition unit 20g acquires model information of the endoscope.
  • the fourth embodiment is as described above, and the other parts are the same as those of the first to third embodiments. Therefore, the corresponding parts are designated by the same reference numerals and detailed description thereof will be omitted.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biomedical Technology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Physics & Mathematics (AREA)
  • Pathology (AREA)
  • Optics & Photonics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Signal Processing (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Endoscopes (AREA)
  • Instruments For Viewing The Inside Of Hollow Bodies (AREA)
  • Image Processing (AREA)
PCT/JP2019/032134 2019-08-16 2019-08-16 内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法 Ceased WO2021033216A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
ES19941927T ES3024141T3 (en) 2019-08-16 2019-08-16 Processor for endoscope, endoscope system, information processing device, program, and information processing method
EP19941927.6A EP4014831B1 (en) 2019-08-16 2019-08-16 Processor for endoscope, endoscope system, information processing device, program, and information processing method
US17/634,389 US12390089B2 (en) 2019-08-16 2019-08-16 Processor for endoscope, endoscope system, information processing apparatus, non- transitory computer-readable storage medium, and information processing method using learning models
PCT/JP2019/032134 WO2021033216A1 (ja) 2019-08-16 2019-08-16 内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法
JP2021541340A JP7116849B2 (ja) 2019-08-16 2019-08-16 内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/032134 WO2021033216A1 (ja) 2019-08-16 2019-08-16 内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法

Publications (1)

Publication Number Publication Date
WO2021033216A1 true WO2021033216A1 (ja) 2021-02-25

Family

ID=74660591

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/032134 Ceased WO2021033216A1 (ja) 2019-08-16 2019-08-16 内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法

Country Status (5)

Country Link
US (1) US12390089B2 (https=)
EP (1) EP4014831B1 (https=)
JP (1) JP7116849B2 (https=)
ES (1) ES3024141T3 (https=)
WO (1) WO2021033216A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023106327A (ja) * 2022-01-20 2023-08-01 株式会社Ａｉメディカルサービス 検査支援装置、検査支援方法および検査支援プログラム
WO2025238709A1 (ja) * 2024-05-14 2025-11-20 オリンパスメディカルシステムズ株式会社 画像処理装置、画像処理装置の作動方法、および、画像処理装置のプログラム

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118247273B (zh) * 2024-05-28 2024-11-22 深圳市安立信电子有限公司 一种内窥镜图像检测方法、系统、设备及介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013056001A (ja) * 2011-09-07 2013-03-28 Olympus Corp 蛍光観察装置
JP2014036738A (ja) * 2012-08-15 2014-02-27 Hoya Corp 内視鏡システム
JP2016158682A (ja) 2015-02-27 2016-09-05 Hoya株式会社 画像処理装置
WO2016208016A1 (ja) * 2015-06-24 2016-12-29 オリンパス株式会社 画像処理装置、画像処理方法、及び画像処理プログラム
WO2017126425A1 (ja) * 2016-01-18 2017-07-27 オリンパス株式会社 医療用サーバシステム
WO2018105063A1 (ja) * 2016-12-07 2018-06-14 オリンパス株式会社 画像処理装置
WO2018159083A1 (ja) * 2017-03-03 2018-09-07 富士フイルム株式会社 内視鏡システム、プロセッサ装置、及び、内視鏡システムの作動方法

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7627386B2 (en) * 2004-10-07 2009-12-01 Zonaire Medical Systems, Inc. Ultrasound imaging system parameter optimization via fuzzy logic
JP6656357B2 (ja) * 2016-04-04 2020-03-04 オリンパス株式会社 学習方法、画像認識装置およびプログラム
US10127659B2 (en) * 2016-11-23 2018-11-13 General Electric Company Deep learning medical systems and methods for image acquisition
JP6785942B2 (ja) * 2017-03-03 2020-11-18 富士フイルム株式会社 内視鏡システム、プロセッサ装置、及び、内視鏡システムの作動方法
US10755428B2 (en) * 2017-04-17 2020-08-25 The United States Of America, As Represented By The Secretary Of The Navy Apparatuses and methods for machine vision system including creation of a point cloud model and/or three dimensional model
EP3745969A4 (en) * 2018-02-03 2021-10-13 Caze Technologies SURGICAL SYSTEMS WITH AUTOMATIC DETECTION AND LEARNING CAPABILITIES AND RELATED PROCESSES
JP2019175211A (ja) * 2018-03-29 2019-10-10 株式会社リコー ヘルプ情報提供システム、ヘルプ情報提供方法およびプログラム
US10705323B2 (en) * 2018-04-24 2020-07-07 Synaptive Medical (Barbados) Inc. Surgical microscope system with automatic zoom control
WO2020008651A1 (ja) * 2018-07-06 2020-01-09 オリンパス株式会社 内視鏡用画像処理装置、及び、内視鏡用画像処理方法、並びに、内視鏡用画像処理プログラム
EP3598161A1 (de) * 2018-07-18 2020-01-22 Siemens Healthcare GmbH Erzeugen von einstellungsparametern für eine magnetresonanztomographie-sequenz mittels eines bedingbar erzeugenden künstlichen neuronalen netzwerks
US10796181B2 (en) * 2018-09-18 2020-10-06 GE Precision Healthcare LLC Machine learning based method and system for analyzing image artifacts and imaging system failure
CN109447973B (zh) 2018-10-31 2021-11-26 腾讯医疗健康(深圳)有限公司 一种结肠息肉图像的处理方法和装置及系统
CN113498323B (zh) * 2019-02-26 2024-08-13 富士胶片株式会社 医用图像处理装置、处理器装置、内窥镜系统、医用图像处理方法、及记录介质
WO2020183791A1 (ja) * 2019-03-11 2020-09-17 キヤノン株式会社 画像処理装置および画像処理方法
US12350104B2 (en) * 2019-05-06 2025-07-08 Koninklijke Philips N.V. Systems and methods for controlling volume rate
CN110246137B (zh) * 2019-06-19 2021-12-03 东软医疗系统股份有限公司 一种成像方法、装置及存储介质
EP3761231A1 (en) * 2019-07-01 2021-01-06 Koninklijke Philips N.V. Fmri task settings with machine learning
EP3760125A1 (en) * 2019-07-05 2021-01-06 Koninklijke Philips N.V. Self-learning distributed system with automated ground-truth generation

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013056001A (ja) * 2011-09-07 2013-03-28 Olympus Corp 蛍光観察装置
JP2014036738A (ja) * 2012-08-15 2014-02-27 Hoya Corp 内視鏡システム
JP2016158682A (ja) 2015-02-27 2016-09-05 Hoya株式会社 画像処理装置
WO2016208016A1 (ja) * 2015-06-24 2016-12-29 オリンパス株式会社 画像処理装置、画像処理方法、及び画像処理プログラム
WO2017126425A1 (ja) * 2016-01-18 2017-07-27 オリンパス株式会社 医療用サーバシステム
WO2018105063A1 (ja) * 2016-12-07 2018-06-14 オリンパス株式会社 画像処理装置
WO2018159083A1 (ja) * 2017-03-03 2018-09-07 富士フイルム株式会社 内視鏡システム、プロセッサ装置、及び、内視鏡システムの作動方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023106327A (ja) * 2022-01-20 2023-08-01 株式会社Ａｉメディカルサービス 検査支援装置、検査支援方法および検査支援プログラム
WO2025238709A1 (ja) * 2024-05-14 2025-11-20 オリンパスメディカルシステムズ株式会社 画像処理装置、画像処理装置の作動方法、および、画像処理装置のプログラム

Also Published As

Publication number Publication date
EP4014831A4 (en) 2023-04-26
US12390089B2 (en) 2025-08-19
ES3024141T3 (en) 2025-06-03
EP4014831A1 (en) 2022-06-22
US20220322915A1 (en) 2022-10-13
EP4014831C0 (en) 2025-04-23
JPWO2021033216A1 (https=) 2021-02-25
JP7116849B2 (ja) 2022-08-10
EP4014831B1 (en) 2025-04-23

Similar Documents

Publication Publication Date Title
US11869183B2 (en) Endoscope processor, information processing device, and endoscope system
JP2021100555A (ja) 医療画像処理装置、内視鏡システム、診断支援方法及びプログラム
US12169963B2 (en) Processing system, image processing method, learning method, and processing device
US20220000337A1 (en) Endoscope processor, information processing device, and endoscope system
JPWO2018159363A1 (ja) 内視鏡システム及びその作動方法
JP7116849B2 (ja) 内視鏡用プロセッサ、内視鏡システム、情報処理装置、プログラム及び情報処理方法
WO2018230130A1 (ja) 内視鏡装置及び表示画像生成方法
CN114901119A (zh) 图像处理系统、内窥镜系统以及图像处理方法
EP4111938A1 (en) Endoscope system, medical image processing device, and operation method therefor
US12169948B2 (en) Processor for endoscope, endoscope system, information processing apparatus, non-transitory computer-readable storage medium, and information processing method
JP7154274B2 (ja) 内視鏡用プロセッサ、情報処理装置、内視鏡システム、プログラム及び情報処理方法
EP3860424B1 (en) Dynamic illumination to identify tissue type
US20250194910A1 (en) Endoscope control device, method of changing wavelength characteristics of illumination light, and information storage medium
US20230036858A1 (en) Systems and methods for machine readable identification of surgical tools in-situ
JP7817465B2 (ja) 内視鏡プロセッサ、内視鏡プロセッサの作動方法、内視鏡装置のプログラム、および、内視鏡装置
JP2025087307A (ja) 内視鏡装置
US12048510B2 (en) Method, system, and software program product for controlling an excitation light source for endoscopic fluorescence imaging
WO2024004013A1 (ja) プログラム、情報処理方法、及び情報処理装置
KR20250152725A (ko) 인공지능 기반 내시경 병변 통합 탐지 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19941927

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021541340

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019941927

Country of ref document: EP

Effective date: 20220316

WWG Wipo information: grant in national office

Ref document number: 2019941927

Country of ref document: EP

WWG Wipo information: grant in national office

Ref document number: 17634389

Country of ref document: US