JPWO2022024423A5 - - Google Patents

Description

It is preferable that the image control processor has a learning model that outputs one or more pieces of procedure assistance information based on the input of the discrimination result, and that the learning model has already been trained. The image control processor stores discrimination results, treatment tool inventory information, treatment pattern information that is information about associations between lesions and treatment tools according to guidelines, anatomical regions such as the pharynx, esophagus, stomach, small intestine, and large intestine. Surrounding environment information related to blood vessel running status, patient history, examination/treatment history, patient risk assessment information such as currently used medicines, procedure information including operator information, one or more combined procedure information It is preferable to have a learning model for improving the appropriateness of the treatment instrument and outputting the procedure support information by inputting , and the learning model is already learned. The image control processor preferably has a learning model for outputting an estimated treatment time based on the input of the surgical assistance information related to the treatment tool calculated by the surgical assistance information calculation unit, and the learning model is preferably already learned.

An image control processor is provided, and the image control processor obtains an inspection image, stores it in a template image storage memory, calculates the feature amount of the diagnosed template image associated with the treatment instrument, and calculates the feature amount of the inspection image. The feature amount of the inspection image is compared with the feature amount of the template image that has already been diagnosed to calculate the overall degree of matching. The procedure information including the inventory information is referred to, the appropriateness of the treatment tool associated with the selected template image is calculated, and the procedure support information including the treatment tool whose appropriateness satisfies the second condition is output. , the examination image and the treatment tools included in the procedure support information are preferably displayed on the display. It is preferable that an inspection image is obtained as a template image, and the inspection image, the discrimination result related to the inspection image, and the treatment instrument are associated with each other and stored in the template image storage memory.

A light source processor is provided, and the light source processor controls emission of first illumination light and second illumination light having different emission spectra and emits the first illumination light when acquiring an inspection image. and a second illumination period in which the second illumination light is emitted, the first illumination light is emitted in the first emission pattern, the second illumination light is emitted in the second emission pattern, and the first illumination light is emitted in the second illumination period. A first illumination light image based on the light, a second illumination light image based on the second illumination light, and a result of discrimination processing for the second illumination light image are superimposed and displayed on a display image based on the first illumination light image. and are preferably acquired as inspection images. The first light emission pattern includes a 1A pattern in which the number of frames in the first illumination period is the same in each first illumination period, and a pattern 1A in which the number of frames in the first illumination period is different in each first illumination period. Any one of the 1B pattern is preferable. The second light emission pattern is a 2A pattern in which the number of frames in the second illumination period is the same in each second illumination period, and the emission spectrum of the second illumination light is the same in each second illumination period. , the number of frames in the second illumination period is the same in each second illumination period, and the emission spectrum of the second illumination light is different in each second illumination period, the second B pattern, the second illumination period The number of frames is different in each second illumination period, and the emission spectrum of the second illumination light is the same in each second illumination period, and the number of frames in the second illumination period. is different in each second illumination period, and the emission spectrum of the second illumination light is any one of the second D patterns in which the emission spectrum is different in each second illumination period.

It is an explanatory view explaining composition of an endoscope system. 1 is a block diagram showing functions of an endoscope system; FIG. 4 is a graph showing spectra of violet light V, blue light B, green light G, and red light R; 4 is a graph showing an emission spectrum of first special light; 7 is a graph showing the emission spectrum of the second special light; It is explanatory drawing which shows the 1st A pattern in superimposition mode, or the 2nd A pattern. FIG. 11 is an explanatory diagram showing a 1B pattern in superimposition mode; FIG. 11 is an explanatory diagram showing a second B pattern in the superimposition mode; FIG. 11 is an explanatory diagram showing a second C pattern in superimposition mode; FIG. 11 is an explanatory diagram showing a 2D pattern in superimposition mode; 4 is a graph showing the spectral transmittance of each color filter of the imaging sensor; FIG. 4 is an explanatory diagram showing a first imaging period and a second imaging period; FIG. 4 is an explanatory diagram showing the configuration of a superimposed image generation unit; It is explanatory drawing which shows the illumination control in superimposition mode, a discrimination process, and an image display in chronological order. FIG. 3 is an explanatory diagram showing a control relationship among a discrimination processing section, a technique support information presentation mode switching section, a first technique support information calculation section, a technique information storage memory, and a display control section of the processor device according to the first embodiment; FIG. 4 is an explanatory diagram showing information input to a first surgical assistance information calculation model of the processor device according to the first embodiment; FIG. 10 is an explanatory diagram showing that when a discrimination result P is input to a technique assistance information calculation model, technique assistance information P is output; FIG. 10 is an explanatory diagram showing that when a discrimination result Q is input to a technique assistance information calculation model, technique assistance information Q is output; FIG. 10 is an explanatory diagram showing that when a discrimination result P and treatment instrument inventory information S are input to a procedure assistance information calculation model, procedure assistance information A is output; FIG. 10 is an explanatory diagram showing that when a discrimination result P and treatment instrument inventory information T are input to a procedure assistance information calculation model, procedure assistance information B is output. FIG. 9 is an explanatory diagram showing that when a discrimination result P and treatment pattern information S are input to a technique support information calculation model, technique support information C is output; FIG. 10 is an explanatory diagram showing that when a discrimination result P and treatment pattern information T are input to a technique support information calculation model, technique support information D is output; FIG. 9 is an explanatory diagram showing that when a discrimination result P and surrounding environment information S are input to a technique support information calculation model, technique support information E is output; FIG. 10 is an explanatory diagram showing that when a discrimination result P and surrounding environment information T are input to a technique support information calculation model, technique support information F is output; FIG. 10 is an explanatory diagram showing that when a discrimination result P and patient risk determination information S are input to a technique support information calculation model, technique support information G is output; FIG. 9 is an explanatory diagram showing that when a discrimination result P and patient risk determination information T are input to a technique support information calculation model, technique support information H is output. FIG. 10 is an explanatory diagram showing that when a discrimination result P and operator information S are input to a technique assistance information calculation model, technique assistance information I is output. FIG. 10 is an explanatory diagram showing that when a discrimination result P and operator information T are input to a technique assistance information calculation model, technique assistance information J is output. FIG. 11 is an explanatory diagram showing a procedure support information presentation screen when the suitability of a treatment instrument is first. FIG. 11 is an explanatory diagram showing a procedure support information presentation screen when the suitability of a treatment instrument is second. FIG. 11 is an explanatory diagram showing a procedure support information presentation screen when displaying a warning; FIG. 10 is an explanatory diagram showing a procedure support information presentation screen on which a treatment start button is displayed; FIG. 10 is an explanatory diagram showing that an expected treatment time P is output when the procedure support information P is input to the treatment time calculation model; FIG. 10 is an explanatory diagram showing that an expected treatment time Q is output when the procedure support information Q is input to the treatment time calculation model; FIG. 11 is an explanatory diagram showing a procedure support information presentation screen on which an estimated treatment time is calculated; FIG. 10 is an explanatory diagram showing a series of flow of switching of the technique support information presentation mode and the technique support information presentation mode in the first embodiment; FIG. 10 is an explanatory diagram showing the relationship between information received and information transmitted by a second technical assistance information calculation unit of the processor device according to the second embodiment; FIG. 11 is an explanatory diagram showing the relationship among an image processing unit, a technique support information presentation mode switching unit, a second technique support information calculation unit, a template image storage memory, a technique information storage memory, and a display control unit of the processor device according to the second embodiment. Processor device, image processing unit, technique support information presentation mode switching unit, inside of second technique support information calculation unit, template image storage memory, technique information storage memory, display control unit, still image acquisition unit, and diagnosis in the second embodiment It is explanatory drawing which shows the relationship of a result input part. FIG. 10 is an explanatory diagram showing the internal relationship of a second surgical assistance information calculation unit of the processor device according to the second embodiment; FIG. 12 is an explanatory diagram showing a series of flow of switching of the technique support information presentation mode and the technique support information presentation mode in the second embodiment;

Further, in the second illumination observation mode, the light source processor 21 sets the light amount ratio of the violet light V, the blue light B, the green light G, and the red light R as short-wavelength narrow band lights to Vs:Bs:Gs: Each of the LEDs 20a to 20d is controlled so as to emit special light of Rs. The light amount ratio Vs:Bs:Gs:Rs is different from the light amount ratio Vc:Bc:Gc:Rc used in the first illumination observation mode, and is appropriately determined according to the observation purpose. For example, when emphasizing superficial blood vessels, it is preferable to make Vs larger than other Bs, Gs, and Rs, and when emphasizing middle-layer blood vessels, Gs is preferably made larger than other Vs, Gs, and Rs. is also preferably increased.

Further, when the light source processor 21 emits light by automatically switching between the first illumination light and the second illumination light in the superimposition mode, the light source processor 21 emits the first illumination light in the first light emission pattern and emits the second illumination light. emit light in the second emission pattern. Specifically, as shown in FIG. 6, the first light emission pattern is a 1A pattern in which the number of frames in the first illumination period is the same in each first illumination period, and as shown in FIG. It is preferable that the number of frames in the first illumination period is any one of a 1B pattern in which the number of frames is different in each first illumination period. In the figure, time represents the direction of time passage.

The imaging sensor 43 is a primary color sensor, and includes B pixels (blue pixels) having blue color filters, G pixels (green pixels) having green color filters, and R pixels (red pixels) having red color filters. and three types of pixels. As shown in FIG. 11, the blue color filter BF mainly transmits light in the blue band, specifically light in the wavelength band of 380 to 560 nm. The transmittance of the blue color filter BF peaks around wavelengths of 460 to 470 nm. The green color filter GF mainly transmits light in the green band, specifically light in the wavelength band of 460 to 620 nm. The red color filter RF mainly transmits light in the red band, specifically light in the wavelength band of 580 to 760 nm.

In the superimposition mode, as shown in FIG. 12 , the imaging processor 44 reads the signal from the imaging sensor 43 while exposing the imaging sensor 43 to the first illumination light in the first illumination period. Output the image signal. A period during which the first image signal is output is referred to as a first imaging period. The first image signal includes the B1 image signal output from the B pixel, the G1 image signal output from the G pixel, and the R1 image signal output from the R pixel. Further, the imaging processor 44 causes the imaging sensor 43 to output a second image signal by performing signal reading while the imaging sensor 43 is exposed to the second illumination light in the second illumination period. A period during which the second image signal is output is referred to as a second imaging period. The second image signal includes a B2 image signal output from the B pixel, a G2 image signal output from the G pixel, and an R2 image signal output from the R pixel.

As shown in FIG. 2, a CDS/AGC (Correlated Double Sampling/Automatic Gain Control) circuit 45 applies correlated double sampling (CDS) and automatic gain control (AGC) to the analog image signal obtained from the imaging sensor 43. conduct. The image signal that has passed through the CDS/AGC circuit 45 is converted into a digital image signal by an A/D (Analog/Digital) converter 46 . A digital image signal after A/D conversion is input to the processor unit 14 .

The image processing switching unit 54 selects the transmission destination of the image signal from the noise reduction unit 53 according to the set mode between the first illumination light image generation unit 55a and the second illumination light image generation unit 55a in the image processing unit 55. Either the unit 55b or the superimposed image generation unit 55c is selected. Specifically, when the inspection image display first illumination observation mode is set, the image signal from the noise reduction section 53 is input to the first illumination light image generation section 55a. When the inspection image display second illumination observation mode is set, the image signal from the noise reduction section 53 is input to the second illumination light image generation section 55b. When the superimposition mode is set, the image signal from the noise reduction section 53 is input to the superimposition image generation section 55c.

In the case of the inspection image display superimposition mode, a superimposed image is generated in the superimposed image generator 55c shown in FIG. The superimposed image generator 55c includes a display image generator 55d, a discrimination processor 55e, and a superimposed processor 55f. The display image generation unit 55d performs the same first illumination light image image processing as described above on the input R1 image signal, G1 image signal, and B1 image signal for one frame. The R1 image signal, G1 image signal, and B1 image signal subjected to the image processing for the first illumination light image are used as images for display. The discrimination processing unit 55e performs discrimination processing on the input R2 image signal, G2 image signal, and B2 image signal for the specific frame. The superimposition processing unit 55f generates a superimposed image by performing superimposed image processing such as superimposing a discrimination result, which is a result of the discrimination processing, on a display image.

The superimposed image display control processing will be described below. As shown in FIG. 14, when the first emission pattern is the 1A pattern and the second pattern is the 2B pattern (the number of frames in the second illumination period: the same, the emission spectrum of the second illumination light: different) , white light W as the first illumination light for two frames, and purple light V, green light G, and red light R as the second illumination light for only one frame during the emission of the white light W. In the case of illumination, a display image is obtained by performing image processing for a first illumination light image on a first image signal obtained by illumination with white light.

On the other hand, discrimination processing is performed on the second image signals (R2 image signal, G2 image signal, B2 image signal) obtained by illumination with the violet light V, and a discrimination result V is obtained. Similarly, discrimination processing is performed on the second image signals (R2 image signal, G2 image signal, B2 image signal) obtained by illumination with the green light G, and a discrimination result G is obtained. Further, discrimination processing is performed on the second image signals (R2 image signal, G2 image signal, B2 image signal) obtained by illumination with the red light R, and a discrimination result R is obtained. These discrimination results V, G, and R are displayed in the display image as a grouped discrimination result T after the discrimination processing for the red light R is completed. The discrimination results V, G, and R may be displayed individually in the display image, and the discrimination results obtained by combining at least two of the discrimination results V, G, and R may be used as a superimposed image. good.

For example, as shown in FIGS. 27 and 28, the discrimination result output by the discrimination processing unit 55e and inputted to the first technique support information calculation unit 60 is the common discrimination result P55ep, and When the input operator information is operator information S85a (FIG. 27) or operator information T85b (FIG. 28) different from each other, the procedure support information calculation model 60M The procedure support information I60i (FIG. 27) is output, and if the operator information T85b is input, the procedure support information J60j (FIG. 28) is output.

10 Endoscope system 12 Endoscope 12a Insertion section 12b Operation section 12c Bending section 12d Tip section 12f Observation mode switching switch 12g Technique support information presentation switch 12h Still image acquisition instruction switch 12i Zoom operation section 13 Light source device 14 Processor device 15 Display 16 UIs
20 light source unit 20a V-LED
20b B-LED
20c G-LED
20d R-LED
21 light source processor 22 optical path coupling unit 23 light guide 30a illumination optical system 30b imaging optical system 31 illumination lens 41 objective lens 42 zoom lens 43 imaging sensor 44 imaging processor 45 CDS/AGC circuit 46 A/D converter 50 image acquisition unit 52 DSP
53 noise reduction unit 54 image processing switching unit 55 image processing unit 55a first illumination light image generation unit 55b second illumination light image generation unit 55c superimposed image generation unit 55d display image generation unit 55e discrimination processing unit 55eM discrimination model 55ep discrimination result P.
55eq Differentiation result Q
55f Superimposition processing unit 56 Technique support information presentation mode switching unit 60 First technique support information calculation unit 60M Technique support information calculation model 60a Technique support information A
60b Technique support information B
60c Technique support information C
60d Technique support information D
60e Technique support information E
60f Technique support information F
60g Technique support information G
60h Technique support information H
60i Procedure support information I
60j Technique support information J
60p Technique support information P
60q Procedure support information Q
61 display control unit 62 treatment time calculation unit 62M treatment time calculation model 62p expected treatment time P
62q Anticipated treatment time Q
63 Still image acquisition unit 64 Diagnosis result input unit 70 Central control unit 80 Technique information storage memory 81 Treatment tool inventory information 81a Treatment tool inventory information S
81b Inventory information T of treatment tools
82 Treatment pattern information 82a Treatment pattern information S
82b Treatment pattern information T
83 Surrounding environment information 83a Surrounding environment information S
83b Surrounding environment information T
84 Patient risk determination information 84a Patient risk determination information S
84b Patient risk determination information T
85 Operator information 85a Operator information S
85b Operator information T
120 Technique support information presentation screen 121 Examination image 123 Technique support information 125 Treatment instrument image 126a Downward order switching icon 126b Upward order switching icon 130 Warning display 140 Estimated treatment time 150 Approve button 151 Deny button 152 Start of treatment Button 153 Save test image button 154 Screenshot button 200 Second technical support information calculator 202 Template image storage memory 203 Second illumination light image feature quantity calculator 204 Template image feature quantity calculator 205 Comprehensive matching degree calculator 207 Template Image selection unit 209 Appropriateness calculation unit 211 Technique support information output unit

Claims (2)

Equipped with an image control processor,
The image control processor,
Acquire inspection images,
calculating a feature amount of a diagnosed template image stored in a template image storage memory and associated with a treatment instrument;
calculating a feature amount of the inspection image;
comparing the feature amount of the inspection image and the feature amount of the diagnosed template image to calculate an overall degree of matching;
selecting a template image that satisfies the first condition for the overall degree of matching;
referring to procedure information including inventory information of treatment tools, calculating the appropriateness for the treatment tools associated with the selected template image;
outputting procedure support information including a treatment instrument whose appropriateness satisfies a second condition;
An endoscope system that displays the examination image and the treatment tool included in the surgical assistance information on a display. The first light emission pattern includes a 1A pattern in which the number of frames in the first illumination period is the same in each of the first illumination periods, and a pattern in which the number of frames in the first illumination period is the same in each of the first illumination periods. 18. The endoscope system according to claim 17, which is any one of a 1B pattern that is different in period.

Images