WO2021039718A1 - 内視鏡システム及びその作動方法 - Google Patents

内視鏡システム及びその作動方法 Download PDF

Info

Publication number
WO2021039718A1
WO2021039718A1 PCT/JP2020/031845 JP2020031845W WO2021039718A1 WO 2021039718 A1 WO2021039718 A1 WO 2021039718A1 JP 2020031845 W JP2020031845 W JP 2020031845W WO 2021039718 A1 WO2021039718 A1 WO 2021039718A1
Authority
WO
WIPO (PCT)
Prior art keywords
measurement
image
light
subject
beam irradiation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/031845
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
大介 宇都宮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to EP20858181.9A priority Critical patent/EP4024113B1/en
Priority to CN202080060690.3A priority patent/CN114286961B/zh
Priority to JP2021542889A priority patent/JP7167352B2/ja
Publication of WO2021039718A1 publication Critical patent/WO2021039718A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0655—Control therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00064—Constructional details of the endoscope body
    • A61B1/00071—Insertion part of the endoscope body
    • A61B1/0008—Insertion part of the endoscope body characterised by distal tip features
    • A61B1/00096—Optical elements
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0605—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements for spatially modulated illumination
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0623—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements for off-axis illumination
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0638—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements providing two or more wavelengths
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/26—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes using light guides

Definitions

  • the present invention relates to an endoscope system that displays a measurement marker for measuring the size of a subject and a method of operating the same.
  • an endoscope system having a light source device, an endoscope, and a processor device, the distance to the subject or the size of the subject is acquired.
  • a beam irradiation region such as a spot appears on the subject by irradiating the beam light.
  • a measurement marker for measuring the size of the subject is displayed on the subject image in correspondence with the position of the spot.
  • An object of the present invention is to provide an endoscope system capable of stably detecting an irradiation region of a measurement beam light and a method of operating the same.
  • the endoscope system of the present invention is provided at a position different from the beam light emitting portion at the beam light emitting portion and the tip portion that irradiate the subject with the measurement beam light, and receives the light from the subject. It has an optical system, and the measurement beam light is provided with an endoscope that emits light obliquely with respect to the optical axis of the imaging optical system and a processor device. The processor device is obtained based on the light received by the imaging optical system.
  • the subject image including the beam irradiation area that appears on the subject by the irradiation of the measurement beam light is acquired, the beam irradiation area is recognized from the subject image, and the size of the subject is measured based on the beam irradiation area.
  • It has a processor that displays a measurement image in which a measurement marker for measurement is displayed on the subject image on a display, and the beam irradiation area in the subject image covers the white central area and the periphery of the central area for measurement. It has a pattern of a specific shape including a peripheral region having a feature amount based on the beam light, and the processor recognizes a beam irradiation region having the pattern of the specific shape.
  • the processor recognizes the beam irradiation area by the learning model that outputs the beam irradiation area in response to the input of the subject image. It is preferable that the processor recognizes the beam irradiation region by performing pattern matching processing on the subject image using the template image of the beam irradiation region determined in advance.
  • the processor can recognize the beam irradiation region of the pattern in which the specific shape is deformed.
  • the feature amount is preferably at least one of the color of the measurement beam light, a color other than the color of the measurement beam light, and the brightness, brightness, saturation, and hue of the measurement beam light.
  • the specific shape is preferably circular.
  • the specific shape is preferably line-shaped.
  • the processor receives a subject image including a plurality of color images, sets the light intensity of the measurement beam light to a specific light intensity, and sets the pixel value of each color image to the maximum pixel value, thereby making the central region of the beam irradiation region white. It is preferable to do so.
  • the processor displays a measurement marker corresponding to the position of the beam irradiation region on the measurement image.
  • the processor preferably displays a measurement marker corresponding to the pattern of the beam irradiation region on the measurement image.
  • the present invention has a beam light emitting portion that irradiates a subject with measurement beam light, and an imaging optical system that is provided at a tip portion at a position different from the beam light emitting portion and receives light from the subject.
  • an endoscope system including an endoscope that emits light for measurement at an angle to the optical axis of the imaging optical system and a processor device having a processor
  • the processor receives light in the imaging optical system.
  • the beam irradiation region formed on the subject by the measurement beam light is recognized as a pattern having a specific shape including the white central region and the peripheral region, thereby stabilizing the irradiation region of the measurement beam light.
  • the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18, and a user interface 19.
  • the endoscope 12 has an insertion portion 12a to be inserted into the subject, an operation portion 12b provided at the base end portion of the insertion portion 12a, and a universal cable 12c.
  • the universal cable 12c captures an image of a light guide 28 (see FIG. 3) that guides the illumination light emitted by the light source device 14, a control line for transmitting a control signal used for controlling the endoscope 12, and an observation target.
  • This is a cable in which a signal line for transmitting the obtained image signal, a power line for supplying power to each part of the endoscope 12, and the like are integrated.
  • a connector 29 for connecting to the light source device 14 is provided at the tip of the universal cable 12c.
  • the light source device 14 generates illumination light by, for example, a semiconductor light source such as an LED (Light Emitting Diode) or an LD (Laser Diode) or a halogen lamp such as a xenon lamp.
  • a semiconductor light source such as an LED (Light Emitting Diode) or an LD (Laser Diode) or a halogen lamp such as a xenon lamp.
  • the light source device 14 is electrically connected to the processor device 16, and the connector 29 of the endoscope 12 is connected to the processor device 16 via the light source device 14. Transmission and reception of control signals, image signals, etc. between the light source device 14 and the connector 29 is wireless communication. Therefore, the light source device 14 wirelessly transmits a control signal or the like transmitted / received to / from the connector 29 to the processor device 16. Further, the light source device 14 supplies the connector 29 with electric power for driving the endoscope 12, and this electric power is also supplied wirelessly.
  • the processor device 16 controls the amount of illumination light emitted by the light source device 14, the light emission timing, and each part of the endoscope 12, and uses an image signal obtained by imaging an observation target irradiated with the illumination light to obtain an endoscope image. To generate. Further, the processor device 16 is electrically connected to the display 18 and the user interface 19. The display 18 displays an endoscopic image generated by the processor device 16, information about the endoscopic image, and the like. The user interface 19 has a function of accepting input operations such as function settings.
  • the endoscope 12 has a normal observation mode, a special observation mode, and a length measurement mode, and these three modes are switched by a mode changeover switch 13a provided on the operation unit 12b of the endoscope 12.
  • the normal observation mode is a mode in which the observation target is illuminated by the illumination light.
  • the special observation mode is a mode in which the observation target is illuminated with special light different from the illumination light.
  • the illumination light and the beam light for measurement are illuminated on the observation target, and the measurement marker used for measuring the size of the observation target and the like is displayed on the subject image obtained by imaging the observation target. ..
  • the illumination light is light used to give brightness to the entire observation target and observe the entire observation target.
  • Special light is light used to emphasize a specific area of an observation target.
  • the measurement beam light is light used for displaying a measurement marker.
  • the operation unit 12b of the endoscope 12 is provided with a freeze switch 13b for operating a still image acquisition instruction for instructing acquisition of a still image of a subject image.
  • a freeze switch 13b for operating a still image acquisition instruction for instructing acquisition of a still image of a subject image.
  • the screen of the display 18 freezes, and at the same time, an alert sound (for example, "pee") indicating that a still image is acquired is emitted.
  • the still image of the subject image obtained before and after the operation timing of the freeze switch 13b is stored in the still image storage unit 42 (see FIG. 3) in the processor device 16.
  • the still image storage unit 42 is a storage unit such as a hard disk or a USB (Universal Serial Bus) memory.
  • the processor device 16 can be connected to the network, the still image of the subject image is stored in the still image storage server (not shown) connected to the network in place of or in addition to the still image storage unit 42. You may.
  • a still image acquisition instruction may be given using an operating device other than the freeze switch 13b.
  • a foot pedal may be connected to the processor device 16 to give a still image acquisition instruction when the user operates the foot pedal (not shown) with his / her foot. You may use the foot pedal for mode switching.
  • a gesture recognition unit (not shown) that recognizes the user's gesture is connected to the processor device 16, and when the gesture recognition unit recognizes a specific gesture performed by the user, a still image acquisition instruction is given. You may do it.
  • the mode switching may also be performed using the gesture recognition unit.
  • a line-of-sight input unit (not shown) provided near the display 18 is connected to the processor device 16, and the line-of-sight input unit recognizes that the user's line of sight is within a predetermined area of the display 18 for a certain period of time or longer. If this happens, a still image acquisition instruction may be given.
  • a voice recognition unit (not shown) may be connected to the processor device 16 so that when the voice recognition unit recognizes a specific voice emitted by the user, a still image acquisition instruction may be given. The mode switching may also be performed using the voice recognition unit.
  • an operation panel such as a touch panel may be connected to the processor device 16 to give a still image acquisition instruction when the user performs a specific operation on the operation panel. The mode switching may also be performed using the operation panel.
  • the tip portion 12d of the endoscope 12 has a substantially circular shape, and an imaging optical system 21 that receives light from the subject and an illumination optical system that irradiates the subject with illumination light. 22, a beam light emitting unit 23 that radiates measurement beam light to the subject, an opening 24 for projecting the treatment tool toward the subject, and an air supply water supply nozzle 25 for performing air supply / water supply. And are provided.
  • the optical axis Ax of the imaging optical system 21 extends in a direction perpendicular to the paper surface.
  • the vertical first direction D1 is orthogonal to the optical axis Ax
  • the horizontal second direction D2 is orthogonal to the optical axis Ax and the first direction D1.
  • the imaging optical system 21 and the beam light emitting portion 23 are provided at different positions of the tip portion 12d, respectively, and are arranged along the first direction D1.
  • the light source device 14 includes a light source unit 26 and a light source control unit 27.
  • the light source unit 26 generates illumination light or special light for illuminating the subject.
  • the illumination light or special light emitted from the light source unit 26 is incident on the light guide 28 and is applied to the subject through the illumination lens 22a.
  • the light source unit 26 includes, as a light source of illumination light, a white light source that emits white light, or a plurality of light sources including a white light source and a light source that emits light of other colors (for example, a blue light source that emits blue light). Is used.
  • the light source unit 26 as a light source of special light, a light source that emits wideband light including blue narrow band light for emphasizing surface layer information such as surface blood vessels is used.
  • the light source control unit 27 is connected to the system control unit 41 of the processor device 16.
  • the illumination light may be a white mixed color light in which blue light, green light, and red light are combined. In this case, it is preferable to design the illumination optical system 22 so that the irradiation range of green light is larger than the irradiation range of red light.
  • the light source control unit 27 controls the light source unit 26 based on an instruction from the system control unit 41.
  • the system control unit 41 gives an instruction regarding the light source control to the light source control unit 27, and also controls the light source 23a (see FIG. 5) of the beam light emitting unit 23.
  • the system control unit 41 controls to turn on the illumination light and turn off the measurement beam light.
  • the special light is turned on and the measurement beam light is turned off.
  • the system control unit 41 controls to turn on the illumination light and turn on the measurement beam light.
  • the illumination optical system 22 has an illumination lens 22a, and the light from the light guide 28 is irradiated to the observation target through the illumination lens 22a.
  • the image pickup optical system 21 includes an objective lens 21a, a zoom lens 21b, and an image pickup element 32.
  • the reflected light from the observation target enters the image sensor 32 via the objective lens 21a and the zoom lens 21b. As a result, a reflected image to be observed is formed on the image sensor 32.
  • the zoom lens 21b has an optical zoom function for enlarging or reducing the subject as a zoom function by moving between the telephoto end and the wide end.
  • the optical zoom function can be switched on and off by the zoom operation unit 13c (see FIG. 1) provided in the operation unit 12b of the endoscope, and when the optical zoom function is ON, the zoom operation unit is further turned on. By manipulating 13c, the subject is enlarged or reduced at a specific magnification.
  • the image sensor 32 is a color image sensor, which captures a reflected image of a subject and outputs an image signal.
  • the image sensor 32 is preferably a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal-Oxide Semiconductor) image sensor, or the like.
  • the image pickup device 32 used in the present invention is a color image pickup sensor for obtaining a red image, a green image, and a red image of three colors of R (red), G (green), and B (blue).
  • the red image is an image output from a red pixel provided with a red color filter in the image pickup element 32.
  • the green image is an image output from a green pixel provided with a green color filter in the image sensor 32.
  • the blue image is an image output from a blue pixel provided with a blue color filter in the image sensor 32.
  • the image sensor 32 is controlled by the image pickup control unit 33.
  • the red color filter RF has a high transmittance in the red band of 600 to 700 nm
  • the red pixel has a high sensitivity in the red band.
  • the green color filter GF has a high transmittance in the green band of 500 to 600 nm
  • the green pixel has a high sensitivity in the green band.
  • the blue color filter BF has a high transmittance in the blue band of 400 to 500 nm
  • the blue pixel has a high sensitivity in the blue band.
  • the red pixel also has sensitivity in the green band or the blue band.
  • the green pixel also has sensitivity in the red band or the blue band.
  • the blue pixel also has sensitivity in the red band or the green band.
  • the image signal output from the image sensor 32 is transmitted to the CDS / AGC circuit 34.
  • the CDS / AGC circuit 34 performs correlated double sampling (CDS (Correlated Double Sampling)) and automatic gain control (AGC (Auto Gain Control)) on an image signal which is an analog signal.
  • CDS Correlated Double Sampling
  • AGC Automatic gain control
  • the image signal that has passed through the CDS / AGC circuit 34 is converted into a digital image signal by the A / D converter (A / D (Analog / Digital) converter) 35.
  • the A / D converted digital image signal is input to the communication I / F (Interface) 37 of the light source device 14 via the communication I / F (Interface) 36.
  • the processor device 16 programs related to various processes are incorporated in a program memory (not shown).
  • the program in the program memory is operated by the system control unit 41 configured by the processor, the processor device 16 realizes the functions of the reception unit 38, the signal processing unit 39, and the display control unit 40.
  • the receiving unit 38 is connected to the communication I / F (Interface) 37 of the light source device 14.
  • the communication I / F receives the image signal transmitted from the communication I / F 37 and transmits it to the signal processing unit 39.
  • the signal processing unit 39 has a built-in memory that temporarily stores an image signal received from the receiving unit 38, and processes an image signal group that is a set of image signals stored in the memory to generate a subject image.
  • the receiving unit 38 may directly send the control signal related to the light source control unit 27 to the system control unit 41.
  • the processing unit or storage unit for example, the irradiation area recognition unit 58 or the marker table 62
  • the length measuring processor and the processor device 16 are kept in a state where they can communicate with each other so that images or various information can be transmitted and received.
  • the blue image of the subject image is on the B channel of the display 18, the green image of the subject image is on the G channel of the display 18, and the red image of the subject image is on the G channel.
  • a color subject image is displayed on the display 18 by performing signal allocation processing assigned to each of the R channels of the display 18. Also in the length measurement mode, the same signal allocation processing as in the normal observation mode is performed.
  • the special observation mode when the special observation mode is set, the red image of the subject image is not used for the display of the display 18, and the blue image of the subject image is used for the B channel and the G channel of the display 18.
  • the signal processing unit 39 By assigning the green image of the subject image to the R channel of the display 18, the pseudo-color subject image is displayed on the display 18.
  • the signal processing unit 39 When the signal processing unit 39 is set to the length measurement mode, the signal processing unit 39 performs a structure enhancement process for emphasizing the structure of blood vessels and the like on the subject image, and a normal part and a lesion part of the observation target. A color difference enhancement process that extends the color difference may be performed.
  • the display control unit 40 displays the subject image generated by the signal processing unit 39 on the display 18.
  • the system control unit 41 controls the image pickup device 32 via the image pickup control section 33 provided in the endoscope 12.
  • the image pickup control unit 33 also controls the CDS / AGC34 and the A / D35 in accordance with the control of the image pickup element 32.
  • the beam light emitting unit 23 emits the measurement beam light Lm obliquely with respect to the optical axis Ax (see FIG. 7) of the imaging optical system 21.
  • the beam light emitting unit 23 includes a light source 23a, a diffractive optical element DOE23b (Diffractive Optical Element), a prism 23c, and an emitting unit 23d.
  • the light source 23a emits light of a color (specifically, visible light) that can be detected by the pixels of the image pickup element 32, and is a light emitting element such as a laser light source LD (LaserDiode) or an LED (LightEmittingDiode).
  • the light source 23a is provided on a scope electric substrate (not shown).
  • the scope electric board is provided at the tip end portion 12d of the endoscope, and receives electric power from the light source device 14 or the processor device 16 to supply electric power to the light source 23a.
  • the wavelength of the light emitted by the light source 23a is, for example, 600 nm or more and 650 nm or less red (beam light color) laser light, but light in other wavelength bands, for example, 495 nm or more and 570 nm or less. Green light or blue light may be used.
  • the light source 23a is controlled by the system control unit 41, and emits light based on an instruction from the system control unit 41.
  • the DOE23b converts the light emitted from the light source into a measurement beam light for obtaining measurement information.
  • the amount of light for the measurement beam light is adjusted from the viewpoint of protecting the human body, eyes, and internal organs, and is adjusted to such an amount that the observation range of the endoscope 12 is sufficiently overexposed (pixel saturation). Is preferable.
  • the prism 23c is an optical member for changing the traveling direction of the measurement beam light after conversion by DOE23b.
  • the prism 23c changes the traveling direction of the measurement beam light so as to intersect the field of view of the imaging optical system 21 including the objective lens 21a. The details of the traveling direction of the measurement beam light will also be described later.
  • the measurement beam light Lm emitted from the prism 23c is irradiated to the subject through the emitting portion 23d formed of the optical member.
  • a spot SP as a beam irradiation region is formed in the subject.
  • the position of the spot SP is recognized by the irradiation area recognition unit 58 (see FIG. 8), and a measurement marker indicating the size of the subject is set according to the position of the spot SP.
  • the display control unit 40 displays the measurement image on which the set measurement marker is displayed on the subject image on the display 18.
  • the position of the spot SP and the observation distance are related, and when the position of the spot SP is located below, the observation distance is small and the spot SP The higher the position of, the larger the observation distance.
  • the set measurement marker Ma is displayed on the subject image.
  • the radius from the spot SP on the measurement marker Ma represents the actual size of the subject (for example, 5 mm).
  • the size (radius) of the measurement marker Ma changes according to the position of the spot SP, that is, the observation distance. For example, when the position of the spot SP is lower, the size of the measurement marker Ma is large, and as the position of the spot SP is higher, the size of the measurement marker Ma is smaller.
  • the measurement markers include a plurality of types such as a first measurement marker and a second measurement marker, and which type of measurement marker is to be displayed on the subject image. Can be selected according to the user's instructions. As the user's instruction, for example, the user interface 19 is used.
  • the exit portion 23d may be a measurement assist slit formed in the tip portion 12d of the endoscope.
  • the emitting portion 23d is composed of an optical member, it is preferable to apply an antireflection coating (AR (Anti-Reflection) coating) (antireflection portion).
  • AR Anti-Reflection
  • the antireflection coating is provided is that when the measurement beam light is reflected without passing through the emission unit 23d and the ratio of the measurement beam light irradiated to the subject decreases, the irradiation area recognition unit 58 described later However, it becomes difficult to recognize the position of the spot SP formed on the subject by the measurement beam light.
  • the beam light emitting unit 23 may be any as long as it can emit the measurement beam light toward the field of view of the imaging optical system 21.
  • the light source 23a may be provided in the light source device, and the light emitted from the light source 23a may be guided to the DOE 23b by an optical fiber.
  • the measurement beam light Lm crosses the field of view of the imaging optical system 21. May be configured to emit light.
  • the measurement beam light is emitted in a state where the optical axis Lm of the measurement beam light intersects the optical axis Ax of the imaging optical system 21.
  • the measurement beam in the imaging range (indicated by arrows Qx, Qy, Qz) at each point at the near end Px, the center vicinity Py, and the far end Pz of the range Rx. It can be seen that the positions of the spots SP formed on the subject (points where the arrows Qx, Qy, and Qz intersect with the optical axis Ax) are different depending on the light Lm.
  • the shooting angle of view of the imaging optical system 21 is represented in the region sandwiched between the two solid lines 101, and the measurement is performed in the central region (the region sandwiched between the two dotted lines 102) having less aberration in the shooting angle of view. I have to.
  • the size of the subject can be determined from the movement of the spot position with respect to the change in the observation distance. Can be measured. Then, by imaging the subject illuminated by the measurement beam light with the image sensor 32, a subject image including the spot SP can be obtained.
  • the position of the spot SP differs depending on the relationship between the optical axis Ax of the imaging optical system 21 and the optical axis Lm of the measurement beam light Lm and the observation distance, but if the observation distance is short, the same actual size is used.
  • the number of pixels indicating the size increases, and the number of pixels decreases as the observation distance increases.
  • the signal processing unit 39 of the processor device 16 includes an irradiation area recognition unit 58, a measurement marker setting unit 61, and a marker table 62.
  • the signal processing unit 39 is input with a subject image of the subject illuminated by the illumination light.
  • the special observation mode is set, the subject image of the subject illuminated by the special light is input.
  • the length measurement mode is set, the subject image of the subject illuminated by the illumination light and the measurement beam light is input.
  • the irradiation area recognition unit 58 recognizes a beam irradiation area having a pattern of a specific shape from the subject image.
  • the pattern having a specific shape includes a white central region CR1 and a peripheral region SR1 that covers the periphery of the central region and has a feature amount based on the measurement beam light. If the beam irradiation region is the spot SP described above, the pattern having a specific shape has a circular shape as shown in FIG. In this case, the white central region CR1 is circular and the peripheral region SR1 is ring-shaped.
  • FIG. 10 shows the distribution of pixel values of each color image in a subject image including a red image RP, a green image GP, and a blue image BP as a plurality of color images. Since the pixel values of the red image RP, the green image GP, and the blue image BP in the central region CR1 have reached the maximum pixel value (for example, 255), the central region CR1 is white. In this case, when the measurement beam light is incident on the image sensor 32, as shown in FIG. 11, in the wavelength range WMB of the measurement beam light, not only the red color filter RF of the image sensor 32 but also the green color is used. The color filter GF and the blue color filter BF all transmit the measurement beam light with a transmittance of 100%.
  • the peripheral region SR1 the pixel value of the red image RP is larger than the pixel value of the green image GP or the blue image BP. Therefore, the peripheral region SR1 is reddish.
  • the pixel values of the red image RP, the green image GP, and the blue image BP in the central region CR1 are set to the maximum pixel values by emitting the light amount of the measurement beam light with a specific light amount.
  • the irradiation area recognition unit 58 can recognize the spot SP having the above-mentioned specific shape and feature amount. Specifically, as shown in FIG. 12, the irradiation area recognition unit 58 has a learning model 59 that recognizes the spot SP by outputting the spot SP, which is the beam irradiation area, in response to the input of the subject image. Is preferable.
  • the learning model 59 is machine-learned by a large number of teacher data associated with the subject image and the already recognized beam irradiation region. As machine learning, it is preferable to use CNN (Convolutional Neural Network).
  • the spot SP By recognizing the spot SP using the learning model 59, not only the circular spot SP (see FIG. 9) composed of the circular central region CR1 and the ring-shaped peripheral region SR1 but also a specific shape It is also possible to recognize the spot SP of a pattern deformed from a certain circular shape. For example, as shown in FIG. 13A, the spot SP deformed in the vertical direction can also be recognized. Further, as shown in FIG. 13B, a spot SP in which a part of the circular shape is missing and deformed can also be recognized.
  • the features of the peripheral region SR1 that can be recognized by the learning model 59 include blue and green in addition to red, which is the color of the measurement beam light.
  • the features of the peripheral region SR1 that can be recognized by the learning model 59 include the brightness, brightness, saturation, and hue of the measurement beam light.
  • the brightness, brightness, saturation, and hue of the measurement beam are acquired by performing brightness conversion processing or brightness, saturation, and hue conversion processing on the peripheral area of the spot SP included in the subject image. Is preferable.
  • the irradiation area recognition unit 58 uses a template image of the spot SP as a template image of a predetermined beam irradiation area, and performs pattern matching processing on the subject image. It may have a pattern matching processing unit 60 that recognizes the spot SP.
  • the template image of the spot SP is stored in the template image storage unit 60a.
  • the template image storage unit 60a not only the template image of the circular spot SP composed of the circular central region CR1 and the ring-shaped peripheral region SR1, but also the spot of the pattern deformed from the circular shape having a specific shape.
  • the template image of SP is also stored.
  • the pattern matching processing unit 60 the pattern matching processing can be performed depending on the pattern of the feature amount of the peripheral portion SR.
  • the feature quantities that can be pattern-matched include the distribution of red, which is the color of the beam light for measurement, and the distribution of blue and green.
  • the features that can be pattern-matched include the brightness, brightness, saturation, and hue distribution of the measurement beam light.
  • the measurement marker setting unit 61 uses a first measurement marker that represents the actual size of the subject (the same size as the actual size) as a measurement marker for measuring the size of the subject based on the position of the spot SP.
  • the measurement marker setting unit 61 corresponds to the position of the spot SP by referring to the marker table 62 that stores the measurement marker image whose display mode differs depending on the irradiation position of the spot SP and the position of the spot SP in association with each other.
  • the marker table 62 displays a measurement marker image whose display mode differs depending on the marker display position for displaying the first measurement marker on the display 18, the position of the spot SP and the marker display position. It may be stored in association with.
  • the measurement marker image for example, has a different size or shape depending on the irradiation position of the spot SP.
  • the display of the measurement marker image will be described later.
  • the stored contents of the marker table 62 are maintained even when the power of the processor device 16 is turned off.
  • the marker table 62 stores the measurement marker image and the irradiation position in association with each other, and stores the distance to the subject corresponding to the irradiation position (distance between the tip portion 12d of the endoscope 12 and the subject) and the measurement. It may be stored in association with the marker image.
  • the display control unit 40 displays the measurement image on which the measurement marker is displayed on the subject image on the display 18 based on the position of the spot SP which is the beam irradiation region. Specifically, the display control unit 40 displays the measurement image on which the first measurement marker is superimposed, centering on the spot SP, on the display 18.
  • the first measurement marker for example, a circular measurement marker is used. In this case, as shown in FIG. 15, when the observation distance is close to the near-end Px, the actual size is 5 mm (horizontal direction and vertical of the subject image) in accordance with the center of the spot SP1 formed on the tumor tm1 of the subject. The marker M1 indicating the direction) is displayed. When the measurement marker is displayed on the display 18, the observation distance may also be displayed on the display 18.
  • the actual size is 5 mm (horizontal and vertical directions of the subject image) in accordance with the center of the spot SP2 formed on the tumor tm2 of the subject.
  • the indicator M2 is displayed. Since the marker display position of the marker M2 is located at the center of the subject image that is not easily affected by distortion by the imaging optical system 21, the marker M2 is circular without being affected by distortion or the like. ..
  • a marker M3 indicating an actual size of 5 mm (horizontal direction and vertical direction of the subject image) is displayed so as to be aligned with the center of the spot SP3 formed on the tumor tm3 of the subject.
  • the size of the first measurement marker corresponding to the same actual size of 5 mm becomes smaller as the observation distance becomes longer.
  • the shape of the first measurement marker differs depending on the marker display position according to the influence of the distortion caused by the imaging optical system 21.
  • the center of the spot SP and the center of the marker are displayed so as to coincide with each other.
  • the first measurement marker is located at a position away from the spot SP. May be displayed.
  • the first measurement marker corresponding to the actual size of the subject of 5 mm is displayed, but the actual size of the subject is an arbitrary value (for example, 2 mm) according to the observation target and the observation purpose. , 3 mm, 10 mm, etc.) may be set.
  • the first measurement marker has a substantially circular shape, but as shown in FIG. 18, it may have a cross shape in which vertical lines and horizontal lines intersect. Further, a graduated cross shape in which a scale Mx is added to at least one of a cross-shaped vertical line and a horizontal line may be used.
  • the first measurement marker a distorted cross shape in which at least one of a vertical line and a horizontal line is tilted may be used.
  • the first measurement marker may be a circle in which a cross shape and a circle are combined and a cross shape.
  • the first measurement marker may be a measurement point cloud type in which a plurality of measurement point EPs corresponding to the actual size from the spot are combined.
  • the number of the first measurement markers may be one or a plurality, and the color of the first measurement markers may be changed according to the actual size.
  • the first measurement marker As the first measurement marker, as shown in FIG. 19, three concentric markers M4A, M4B, and M4C (the sizes are 2 mm, 5 mm, and 10 mm in diameter, respectively) having different sizes are placed on the tumor tm4.
  • the spot SP4 formed in the above may be displayed on the subject image. Since a plurality of these three concentric markers are displayed, the trouble of switching can be saved, and measurement is possible even when the subject has a non-linear shape.
  • a combination of multiple conditions can be prepared in advance and selected from the combinations. It may be.
  • the marker M5A is represented by a dotted line representing red
  • the marker M5B is represented by a solid line representing blue
  • the marker M5C is represented by a alternate long and short dash line representing white.
  • the first measurement marker in addition to a plurality of concentric markers, as shown in FIG. 21, a plurality of distorted concentric markers in which each concentric circle is distorted may be used.
  • the distorted concentric markers M6A, M6B, and M6C are displayed on the subject image centering on the spot SP5 formed on the tumor tm5.
  • the measurement beam light As the measurement beam light, light formed as a spot when the subject is irradiated is used, but other light may be used.
  • a planar measurement beam light formed as an intersection line 80 on the subject may be used.
  • an intersecting line 80 By irradiating the subject with the planar measurement beam light, an intersecting line 80, which is a line-shaped beam irradiation region, is formed on the subject.
  • a second measurement marker including the intersection line 80 and the scale 82 as an index of the size of the subject (for example, the polyp P) is generated on the intersection line 80.
  • the irradiation area recognition unit 58 recognizes the position of the intersection line 80 (irradiation position of the measurement beam light) as a beam irradiation area having a pattern of a specific shape.
  • the intersection line 80 includes a white central region CR2 and a peripheral region SR2 that covers the white central region R2 and has a feature amount based on the measurement beam light.
  • the central region CR2 has a line shape, and the peripheral region SR2 is composed of two lines separated by a specific interval.
  • the irradiation area recognition unit 58 can recognize the intersection line 80 having the above pattern and feature amount. It should be noted that the irradiation area recognition unit 58 can also recognize the intersection line 80 that is deformed from the line shape such that a part of the line is missing.
  • the intersection line 80 recognized by the irradiation area recognition unit 58 As for the intersection line 80 recognized by the irradiation area recognition unit 58, the lower the spot SP, the closer the observation distance, and the higher the intersection line 80, the farther the observation distance. Therefore, when displaying the second measurement marker on the measurement image, the distance between the scales 82 increases as the intersection line 80 is located lower, and the distance between the scales 82 increases as the intersection line 80 is located above. The interval becomes smaller.
  • the measurement marker corresponding to the position of the beam irradiation region such as the spot SP or the intersection line 80 is displayed on the measurement image, but instead of this, the measurement marker corresponding to the pattern of the beam irradiation region is displayed. May be displayed on the measurement image.
  • the beam irradiation region is the spot SP, as shown in FIG. 24, the diameter DM of the white central region CR1 of the spot SP is obtained, and the measurement marker corresponding to the diameter DM is displayed on the measurement image.
  • the diameter DM is large, and the larger the observation distance, the smaller the diameter.
  • a measurement marker image whose size, shape, etc. differ depending on the diameter DM is determined.
  • the mode changeover switch 13a When the mode changeover switch 13a is operated to switch to the length measurement mode, the subject is irradiated with the illumination light and the measurement beam light. In the subject, the spot SP is formed by the measurement beam light. By imaging a subject including the spot SP, a subject image including the spot SP can be obtained.
  • the receiving unit 38 receives the subject image output from the endoscope 12.
  • the irradiation area recognition unit 58 recognizes the position of the spot SP from the subject image.
  • the irradiation region recognition unit 58 recognizes the position of the spot SP including the white central region CR1 and the peripheral region SR1 that covers the periphery of the central region CR1 and has a feature amount based on the measurement beam light.
  • the display control unit 40 displays on the display 18 a measurement image in which a measurement marker for measuring the size of the subject is displayed on the subject image based on the position of the spot SP recognized by the irradiation area recognition unit 58. ..
  • the measurement marker corresponds to the position of the spot SP, and its size and shape change depending on the position of the spot SP.
  • the hardware structure of the processing unit that executes various processes is various processors as shown below.
  • the circuit configuration is changed after manufacturing the CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), etc., which are general-purpose processors that execute software (programs) and function as various processing units. It includes a programmable logic device (PLD), which is a possible processor, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for executing various processes, and the like.
  • PLD programmable logic device
  • One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). May be done. Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software. There is a form in which this processor functions as a plurality of processing units.
  • SoC System On Chip
  • a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used.
  • the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.
  • the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements.
  • the hardware structure of the storage unit is a storage device such as an HDD (hard disk drive) or SSD (solid state drive).
  • Another embodiment of the present invention is as follows. It has a beam light emitting unit that irradiates a subject with measurement beam light, and an imaging optical system that is provided at a tip portion at a position different from the beam light emitting unit and receives light from the subject.
  • the measurement beam light is an irradiation of the measurement beam light, which is a subject image obtained based on an endoscope that emits light obliquely with respect to the optical axis of the imaging optical system and light received by the imaging optical system.
  • the irradiation area recognition unit Based on the receiving unit that acquires a subject image including the beam irradiation region appearing on the subject, the irradiation area recognition unit that recognizes the position of the beam irradiation region from the subject image, and the position of the beam irradiation region, the subject.
  • an endoscopic system including a processor device having a display control unit that displays a measurement image in which a measurement marker for measuring the size of the light is displayed on the subject image on the display unit.
  • the processor device The irradiation region recognition unit recognizes the beam irradiation region having a pattern of a specific shape including a white central region and a peripheral region that covers the periphery of the central region and has a feature amount based on the measurement beam light. Endoscopic system.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Optics & Photonics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Signal Processing (AREA)
  • Endoscopes (AREA)
PCT/JP2020/031845 2019-08-28 2020-08-24 内視鏡システム及びその作動方法 Ceased WO2021039718A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP20858181.9A EP4024113B1 (en) 2019-08-28 2020-08-24 Endoscope system and method for operating same
CN202080060690.3A CN114286961B (zh) 2019-08-28 2020-08-24 内窥镜系统及其工作方法
JP2021542889A JP7167352B2 (ja) 2019-08-28 2020-08-24 内視鏡システム及びその作動方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-155553 2019-08-28
JP2019155553 2019-08-28

Publications (1)

Publication Number Publication Date
WO2021039718A1 true WO2021039718A1 (ja) 2021-03-04

Family

ID=74684192

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/031845 Ceased WO2021039718A1 (ja) 2019-08-28 2020-08-24 内視鏡システム及びその作動方法

Country Status (4)

Country Link
EP (1) EP4024113B1 (https=)
JP (1) JP7167352B2 (https=)
CN (1) CN114286961B (https=)
WO (1) WO2021039718A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4218563A1 (en) * 2022-02-01 2023-08-02 FUJIFILM Corporation Endoscope system and endoscope system operation method

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116048249A (zh) * 2022-12-20 2023-05-02 上海澳华内镜股份有限公司 内窥镜操作系统
CN119055163B (zh) * 2024-11-04 2025-02-25 湖南省华芯医疗器械有限公司 一种前端组件及内窥镜

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05203882A (ja) * 1992-01-27 1993-08-13 Olympus Optical Co Ltd 内視鏡の照明光学系
JP2010175579A (ja) * 2009-01-27 2010-08-12 Fujifilm Corp ライトガイド及び光源装置並びに内視鏡システム
WO2018051680A1 (ja) 2016-09-15 2018-03-22 富士フイルム株式会社 内視鏡システム
WO2018180249A1 (ja) * 2017-03-28 2018-10-04 富士フイルム株式会社 計測支援装置、内視鏡システム、及びプロセッサ

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060226231A1 (en) * 2005-03-29 2006-10-12 University Of Washington Methods and systems for creating sequential color images
JP2009240621A (ja) * 2008-03-31 2009-10-22 Hoya Corp 内視鏡装置
JP6144914B2 (ja) * 2010-02-12 2017-06-07 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． レーザ改良された３ｄ表面再構成
JP6964592B2 (ja) * 2016-09-15 2021-11-10 富士フイルム株式会社 計測支援装置、内視鏡システム、内視鏡システムのプロセッサ、及び計測支援方法
WO2018055933A1 (ja) * 2016-09-20 2018-03-29 富士フイルム株式会社 計測支援装置、内視鏡システム、内視鏡システムのプロセッサ、及び計測支援方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05203882A (ja) * 1992-01-27 1993-08-13 Olympus Optical Co Ltd 内視鏡の照明光学系
JP2010175579A (ja) * 2009-01-27 2010-08-12 Fujifilm Corp ライトガイド及び光源装置並びに内視鏡システム
WO2018051680A1 (ja) 2016-09-15 2018-03-22 富士フイルム株式会社 内視鏡システム
WO2018180249A1 (ja) * 2017-03-28 2018-10-04 富士フイルム株式会社 計測支援装置、内視鏡システム、及びプロセッサ

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4024113A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4218563A1 (en) * 2022-02-01 2023-08-02 FUJIFILM Corporation Endoscope system and endoscope system operation method
US12133626B2 (en) 2022-02-01 2024-11-05 Fujifilm Corporation Endoscope system and endoscope system operation method

Also Published As

Publication number Publication date
JP7167352B2 (ja) 2022-11-08
EP4024113A4 (en) 2022-10-12
EP4024113B1 (en) 2025-06-25
JPWO2021039718A1 (https=) 2021-03-04
EP4024113A1 (en) 2022-07-06
CN114286961A (zh) 2022-04-05
CN114286961B (zh) 2025-02-25

Similar Documents

Publication Publication Date Title
JP7115897B2 (ja) 内視鏡装置
JP7167352B2 (ja) 内視鏡システム及びその作動方法
JP7099971B2 (ja) 内視鏡装置、キャリブレーション装置、及びキャリブレーション方法
EP3586719B1 (en) Endoscope apparatus
EP4016162B1 (en) Endoscope system and method for operating same
US12029386B2 (en) Endoscopy service support device, endoscopy service support system, and method of operating endoscopy service support device
CN114286641B (zh) 内窥镜装置及其工作方法以及计算机可读介质
EP3698714B1 (en) Endoscope apparatus
EP3915457B1 (en) Endoscope device
JP2020014808A (ja) 内視鏡装置及びその作動方法並び内視鏡装置用プログラム
JP7029359B2 (ja) 内視鏡装置及びその作動方法並び内視鏡装置用プログラム
JP7447249B2 (ja) テストチャート、検査システム、及び検査方法
JP7214840B2 (ja) 内視鏡、内視鏡装置
JP2020005745A (ja) 内視鏡装置
WO2022049806A1 (ja) キャリブレーション装置及び方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20858181

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021542889

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020858181

Country of ref document: EP

Effective date: 20220328

WWG Wipo information: grant in national office

Ref document number: 202080060690.3

Country of ref document: CN

WWG Wipo information: grant in national office

Ref document number: 2020858181

Country of ref document: EP