JP2022132940A - Endoscope and endoscope system - Google Patents

Abstract

To provide an endoscope and an endoscope system which can reduce an oversight risk of a disease, suppress increase in advance notice and inspection time to a spot where a lesion part is invisible and reduce a damage risk to a tissue surface in an organism.SOLUTION: An endoscope comprises: an insertion unit which has a tip part on a tip side and is inserted into a subject; an imaging unit which is arranged in the tip part and can acquire an optical image in the subject; and a LiDAR unit which is arranged in the insertion unit and can acquire three-dimensional information in the subject. The LIDAR unit acquires specific shape information about a specific shape existing in a visible region and an invisible region in an imaging region that can be imaged by the imaging unit as the three-dimensional information in the subject.SELECTED DRAWING: Figure 2

Description

The present invention relates to endoscopes and endoscope systems.

In general, many endoscopes have an imaging unit at the distal end, and when inserted into a subject, the imaging unit observes the front of the distal end in the longitudinal direction. In recent years, endoscopes have been used to acquire the distance to an observation target, and Patent Document 1 discloses an endoscope provided with a distance sensor in addition to an imaging unit.

JP 2014-117446 A Japanese Patent No. 5993370

By the way, the gastrointestinal tract including the large intestine, which is an example of tissue surfaces in vivo, has unique folds. Therefore, when the imaging unit is used to observe the front of the distal end in the longitudinal direction, and when the distance sensor is used to measure the distance, the folds may hide the lesion from the endoscope. In other words, there is a risk that lesions present in blind spots such as the back side of folds may be overlooked.

On the other hand, Patent Literature 2 discloses a method of physically stretching a fold and enlarging an observable region by using an endoscope having a balloon or multiple protrusions at the tip. However, physical stretching of the folds raises concerns about an increase in examination time and a risk of damage to the digestive tract, which may result in an increased burden on the patient.

The present invention has been made in view of such circumstances, and is intended to reduce the risk of overlooking a lesion, to give advance notice of a lesion that is not visible, to suppress an increase in examination time, and to prevent an increase in examination time. An object of the present invention is to provide an endoscope and an endoscope system that can reduce the risk of damage.

The endoscope of the first form includes an insertion section that has a distal end on the distal end side and is inserted into the subject, an imaging unit that is arranged at the distal end and is capable of acquiring an optical image of the interior of the subject, and an insertion section. and a LiDAR unit capable of acquiring three-dimensional information within the subject, wherein the LiDAR unit provides three-dimensional information within the subject in an imaging area that can be imaged by the imaging unit, Acquiring singular shape information about singular shapes existing in the visible area and the invisible area.

In the second form of endoscope, the peculiar shape is a shape related to the lesion.

In the endoscope of the third embodiment, the LiDAR unit acquires body cavity information regarding the body cavity of the subject into which the insertion section is inserted, as three-dimensional information inside the subject.

The endoscope system of the fourth form includes the above-described endoscope, and a processor device including a processor and a memory that transmits and receives signals between the imaging unit and the LiDAR unit. A singular shape is detected based on the received singular shape information.

In the endoscope system of the fifth embodiment, the memory stores information about the shape of the lesion, and the processor selects the shape of the lesion stored in the memory from the three-dimensional information acquired by the LiDAR unit. A singular shape is detected by obtaining a degree of similarity with information related to

In the endoscope system of the sixth form, the processor executes notification processing for notifying that the peculiar shape has been detected.

In the endoscope system of the seventh embodiment, the processor stores the detection of the peculiar shape in the memory, and executes notification processing of notifying the detection of the peculiar shape at an arbitrary timing.

In the endoscope system of the eighth form, the processor executes notification processing by sound when detecting the peculiar shape.

The endoscope system of the ninth form includes a display device connected to the processor device, the processor displays an optical image on the display device, and executes notification processing by displaying on the display device when a peculiar shape is detected. do.

In the endoscope system of the tenth form, a display device connected to the processor device is provided, and the LiDAR unit acquires body cavity information regarding the body cavity of the subject into which the insertion portion is inserted as three-dimensional information inside the subject. , the processor creates a three-dimensional model of the body cavity of the subject based on the body cavity information acquired by the LiDAR unit.

In the endoscope system of the eleventh form, the processor stores the position of the peculiar shape in the three-dimensional model in the memory as position information.

In the endoscope system of the twelfth form, the processor displays the three-dimensional model including the position of the peculiar shape on the display device based on the position information stored in the memory.

In the endoscope system of the thirteenth form, the processor displays the position of the tip in the three-dimensional model on the display device.

According to the present invention, it is possible to reduce the risk of overlooking a lesion, suppress advance notification of an invisible lesion, suppress an increase in examination time, and reduce the risk of damage to the tissue surface in the living body.

FIG. 1 is a schematic configuration diagram showing an example of the configuration of an endoscope system. FIG. 2 is an enlarged view of the distal end of the endoscope. FIG. 3 is a block diagram of the endoscope system. FIG. 4 is a block diagram showing processing functions realized by a CPU. FIG. 5 is a diagram for explaining observation of the inside of a subject using an endoscope. FIG. 6 is a diagram for explaining an observation image at position P1 in FIG. FIG. 7 is a diagram for explaining an observation image at position P2 in FIG. FIG. 8 is a diagram for explaining an observation image at position P3 in FIG. FIG. 9 is a diagram for explaining the observation state of the back side of a fold with an endoscope. FIG. 10 is a schematic diagram showing an observation image and a three-dimensional model displayed on a display device. FIG. 11 is an enlarged view of the distal end of the endoscope. FIG. 12 is a schematic configuration diagram showing an example of configuration of an ultrasonic endoscope system. FIG. 13 is an enlarged view of the distal end portion of the ultrasonic endoscope. FIG. 14 is a diagram for explaining observation of the inside of a subject using an ultrasonic endoscope. FIG. 15 is a schematic diagram of a case where an ultrasonic image and a three-dimensional model are displayed on a display device.

Preferred embodiments of an endoscope and an endoscope system according to embodiments will be described below with reference to the accompanying drawings.

<Configuration diagram of endoscope system>
FIG. 1 is a configuration diagram of an endoscope system 1 including an endoscope 10. As shown in FIG. The endoscope system 1 includes an endoscope 10 , a light source device 14 , a processor device 16 and a display device 18 .

The endoscope 10 includes an operation section 22 in which operation members are arranged, and an insertion section 24 provided on the distal end side of the operation section 22 and inserted into the subject. The insertion section 24 has a longitudinal direction Ax extending from the proximal end to the distal end, and includes a flexible section 26, a curved section 28, and a distal section 30 in order from the proximal end toward the distal end.

The operation part 22 is configured in a substantially cylindrical shape as a whole. The operation portion 22 has an operation portion main body 34 and a grip portion 36 connected to the operation portion main body 34 . A proximal end of the insertion portion 24 is provided on the distal end side of the grasping portion 36 via a bend prevention tube 38 .

The operation portion main body 34 is provided with a pair of angle knobs 40 and lock levers 42 , 42 for bending the bending portion 28 as one of the operation members. An air/water supply button 44 , a suction button 46 , and a plurality of operation buttons 48 are arranged in parallel along the long axis direction Ax on the operation unit main body 34 . The operation button 48 is a button for switching operations such as stillness and recording of the observed optical image and its function.

A treatment instrument introduction port 52 for introducing a treatment instrument is arranged in the grip portion 36 of the operation section 22 . A treatment instrument (not shown) introduced from the treatment instrument introduction port 52 is passed through the treatment instrument channel and led out from the distal end of the distal end portion 30 .

A universal cable 54 is provided in the operation unit main body 34 . A connector device 56 is provided on the distal end side of the universal cable 54 . The connector device 56 is detachably connected to the light source device 14 . The light source device 14 is electrically connected to the processor device 16 , and the connector device 56 of the endoscope 10 is connected to the processor device 16 via the light source device 14 . The light source device 14 and the connector device 56 can transmit and receive control signals, image signals and the like by optical communication. The light source device 14 transmits to the processor device 16 a control signal or the like transmitted and received by optical communication via the connector device 56 . The light source device 14 wirelessly supplies power for driving the endoscope 10 via the connector device 56 .

<Tip of endoscope>
FIG. 2 is an enlarged view of the distal end of the endoscope. A distal end portion 30 of the endoscope 10 has a substantially circular shape in plan view. An observation window 70 , two illumination windows 72 arranged on both sides of the observation window 70 , a forceps port 74 , and an air/water supply nozzle 75 are arranged on the distal end surface of the distal end portion 30 .

Air or water can be supplied to the air/water nozzle 75 by operating the air/water supply button 44 (see FIG. 1) of the operation portion main body 34 . The air/water supply button 44 is a button that can be operated in two stages, the first operation supplying air and the second operation supplying water. A treatment instrument introduced from the treatment instrument introduction port 52 protrudes from the forceps port 74 . When the suction button 46 is operated, body fluid such as blood is suctioned from the forceps port 74 .

A LiDAR (light detection and ranging) unit 76 is arranged in the endoscope 10 . The LiDAR unit 76 includes a laser light source 78, an optical sensor 80, and control circuitry to be described below. The LiDAR unit 76 irradiates an object with a laser beam from a laser light source 78 and detects the laser beam that bounces off the object with an optical sensor 80 to measure the time until the laser beam bounces off the object. This allows the distance and direction to the object to be measured. The LiDAR unit 76 can acquire three-dimensional information by acquiring information on the distance to the object as a point cloud.

<Block diagram of endoscope system>
FIG. 3 is a block diagram of the endoscope system 1. As shown in FIG. As shown in FIG. 3 , the imaging unit 100 and the LiDAR unit 76 are arranged at the distal end portion 30 of the endoscope 10 and configured to be capable of transmitting and receiving signals to and from the processor device 16 .

The image pickup unit 100 includes an image pickup lens 101 , an image pickup element 102 arranged on the base end side of the image pickup lens 101 , an AFE (Analog Front End) 103 , and a control circuit 104 .

The imaging unit 100 has an imaging lens 101 arranged at a position corresponding to the observation window 70 . The imaging device 102 is, for example, a CMOS (Complementary MOS) type color imaging device.

An observation image viewed directly ahead from the longitudinal direction Ax is captured by the imaging device 102 via the imaging lens 101 . The imaging device 102 converts the observed image into an electrical signal. An analog signal from the imaging element 102 is adjusted by the AFE 103 . A control circuit 104 controls the operation of the image sensor 102 . By inserting the insertion portion 24 (see FIG. 1) of the endoscope 10 into the subject, the imaging unit 100 can acquire an optical image of the interior of the subject. The configuration of the imaging unit 100 can be changed as appropriate as long as it can acquire an optical image, and is not limited to the configuration of FIG.

A LiDAR unit 76 is arranged on the distal end face side of the distal end portion 30 of the endoscope 10 . The LiDAR unit 76 comprises a laser light source 78 and a light sensor 80 and a control circuit 82 arranged proximal to the laser light source 78 and the light sensor 80 .

For the laser light source 78 of the LiDAR unit 76, for example, a Vertical Cavity Surface Emitting Laser (VCSEL) can be applied. The photosensor 80 can apply, for example, a Single Photon Avalanche Diode (SPAD). Further, by arranging a diffractive optical element (DOE) (not shown) on the irradiation surface of the laser light source 78, the laser light from the laser light source 78 is irradiated in a predetermined direction and in a predetermined pattern.

A control circuit 82 controls the entire LiDAR unit 76 . The control circuit 82 of the LiDAR unit 76 irradiates the object with a laser beam from the laser light source 78, and detects the laser beam that bounces off the object with the optical sensor 80, thereby measuring the time until the laser beam hits the object and bounces off. measure. This allows the distance and direction to the object to be measured. The LiDAR unit 76 acquires three-dimensional information up to the object as a point cloud. By inserting the endoscope 10 into the subject, the LiDAR unit 76 can acquire three-dimensional information inside the subject. As long as the LiDAR unit 76 can acquire three-dimensional information, its configuration can be changed as appropriate, and is not limited to the configuration shown in FIG. The three-dimensional information inside the subject acquired by the LiDAR unit 76 can include body cavity information about the body cavity of the subject and specific shape information about a specific shape represented by a shape about a lesion.

Two light guides 110 are arranged at the distal end portion 30 . The exit surfaces of the two light guides 110 are positioned to correspond to the two illumination windows 72, respectively.

The light source device 14 includes a light source 140 for illumination and a light source control circuit 141 . The light source device 14 causes the light from the light source 140 to enter the incident end of the light guide 110 . The illuminance of illumination light from the light source 140 is controlled by a light source control circuit 141 .

The processor device 16 includes an image input controller 161 and a CPU (Central Processing Unit) 162 . An optical image output from the imaging unit 100 is input to the CPU 162 via the image input controller 161 . CPU 162 performs image processing necessary for the optical image. The display control unit 163 displays the optical image processed by the CPU 162 on the display device 18 . The operator can visually recognize the optical image of the inside of the subject displayed on the display device 18 .

The CPU 162 performs image processing on the optical image, switching of the optical image to be displayed on the display device 18, superimposition processing, electronic zoom processing, display switching of the optical image according to the operation mode, and processing of specific components (for example, , luminance signal).

Three-dimensional information inside the subject acquired by the LiDAR unit 76 is input to the CPU 162 . The CPU 162 executes various processes on the three-dimensional information inside the subject. The CPU 162 can detect the peculiar shape based on the peculiar shape information included in the three-dimensional information within the subject. Further, the CPU 162 can create a three-dimensional model based on the body cavity information of the three-dimensional information inside the subject.

Existence information indicating that the CPU 162 has detected the peculiar shape and/or the created three-dimensional model is displayed on the display device 18 via the display control unit 163 . The CPU 162 executes a process of superimposing the presence information of the singular shape and the optical image, a process of separating the optical image from the singular shape and the three-dimensional model, and the like. A processing result by the CPU 162 is displayed on the display device 18 via the display control unit 163 .

The memory 165 stores information necessary for processing performed by the CPU 162, such as programs, optical images, three-dimensional information, information generated along with the processing of the CPU 162, arbitrary input information, and the like. Information not stored in the memory 165 can be acquired from outside the endoscope system 1 via a recording medium or network. An external memory such as a database on a network is used as part of the memory of the endoscope system 1 .

The sound processing unit 166 causes the speaker 167 to output sound based on the signal from the CPU 162 . The type of sound is not limited as long as it can be recognized by the operator, such as warning sound or voice. The sound processing unit 166 may generate sounds or use sounds stored in the memory 165 .

The processor unit 16 has an operation unit 168 . The operation unit 168 includes an operation mode setting switch (not shown). The illumination of the light source device 14 can be manipulated. An operator or the like can instruct the CPU 162 to perform each process via the operation unit 168 and the input interface (I/F) 169 .

The CPU 162 can control the entire endoscope system 1 including the control circuit 104 of the imaging unit 100, the control circuit 82 of the LiDAR unit 76, the light source control circuit 141 of the light source device 14, and the like.

FIG. 4 is a block diagram showing processing functions implemented by the CPU 162. As shown in FIG. The CPU 162 can include, for example, an image processing unit 170 , a detection processing unit 171 , a notification processing unit 172 , a three-dimensional model creation unit 173 and a position detection processing unit 174 . The image processing unit 170, the detection processing unit 171, the notification processing unit 172, the three-dimensional model creation unit 173, and the position detection processing unit 174 are part of the CPU 162, and the CPU 162, which is a processor, executes the processing of each unit.

<Observation with an endoscope>
FIG. 5 is a diagram showing a state in which the insertion portion of the endoscope is inserted into the subject. FIG. 5 shows that the endoscope 10 is inserted into, for example, the large intestine 200, which is one of the digestive tracts, and observations are made at positions P1, P2, and P3. The positions P1, P2 and P3 can be changed by moving the endoscope 10 along the large intestine 200 in the forward direction F indicated by the arrow or the backward direction B indicated by the arrow. In FIG. 5, the endoscopes 10 at positions P1, P2, and P3 are displayed in an overlapping manner. Although the inner wall of the gastrointestinal tract is exemplified as the in vivo tissue surface, the surface of in vivo tissue such as the respiratory tract (trachea, bronchi, etc.) can also be included.

The large intestine 200 has a plurality of folds 201 on its inner surface. A lesion 202 such as a polyp exists on the front side of the fold 201 (the side facing the backward direction B of the endoscope 10), or the lesion 202 exists on the back side of the fold 201 (the side facing the forward direction F of the endoscope 10). obtain. A lesion 202 existing on the front side of the fold 201 can be observed relatively easily by the imaging unit 100 of the endoscope 10 . On the other hand, the lesion 202 present on the back side of the fold 201 becomes a blind spot and is difficult to observe with the imaging unit 100 .

The endoscope 10 of the embodiment includes an imaging unit 100 and a LiDAR unit 76 at the distal end portion 30 to observe the inner surface of the large intestine 200 . The viewing range V1 is the viewing range of the imaging unit 100, and the viewing range V2 is the viewing range of the LiDAR unit 76. FIG. The three-dimensional information acquired by the LiDAR unit 76 includes information (hereinafter referred to as “peculiar shape information”) on a unique shape existing outside the imaging area that can be imaged by the imaging unit 100 . That is, the LiDAR unit 76 provides, as three-dimensional information within the subject, unique shape information about unique shapes existing in the visible region and the invisible region in the imaging region that can be imaged by the imaging unit 100 indicated by the visual field range V1. It is obtainable. The singular shape information in this example is distance information indicating the distance to the point group that specifies the singular shape, but other information may be used as long as the singular shape can be specified. The viewing range V2 of the LiDAR unit 76 includes visible and non-visible areas in the imaging area. Here, in FIG. 5, the three-dimensional information inside the subject is three-dimensional information on the inner surface of the large intestine 200, for example.

The invisible area in the imaging unit 100 is an area hidden by the back surface of the observation target in the imaging area, for example, the folds 201 in FIG. Examples of the invisible region include a region hidden by the folds 201, a blind region that bends at the end of the path, and the near side when the endoscope 10 is moved in the backward direction B.

Therefore, by providing the LiDAR unit 76 in addition to the imaging unit 100 , the endoscope 10 can detect the presence of the lesion 202 existing in the invisible region such as the back side of the fold 201 . The provision of the LiDAR unit 76 also makes it possible to detect the presence of lesions 202 present in the visibility area.

6 to 8 show observation images corresponding to positions P1, P2 and P3 in FIG. The observed image in FIG. 6 corresponds to position P1, the observed image in FIG. 7 corresponds to position P2, and the observed image in FIG. 8 corresponds to position P3. Although optical images DP1, DP2, and DP3 are shown in FIGS. 6, 7, and 8, they may simply be referred to as optical images DP when they are not associated with positions P1, P2, and P3.

As shown in FIG. 6, when the endoscope 10 is positioned at the position P1, an optical image DP1 acquired by the imaging unit 100 in the visual field range V1 is displayed as an observation image. The optical image DP<b>1 acquired by the imaging unit 100 is subjected to image processing and the like by the image processing unit 170 (see FIG. 4 ), and displayed on the display device 18 . The optical image DP1 includes folds 201 present on the inner surface of the large intestine 200 and lesions 202 present on the front side of the folds 201 .

The LiDAR unit 76 acquires three-dimensional information in the viewing range V2 while the imaging unit 100 acquires the optical image DP1. Three-dimensional information inside the subject is input to the CPU 162 (see FIG. 5), and the detection processing unit 171 executes detection processing. As will be described later, the detection processing unit 171 detects the peculiar shape by extracting peculiar shape information about the peculiar shape from the three-dimensional information acquired by the LiDAR unit 76 . Here, the three-dimensional information acquired by the LiDAR unit 76 does not include peculiar shape information about a peculiar shape typified by the lesion 202 existing behind the fold 201 which is the invisible area of the imaging unit 100 . On the other hand, it contains peculiar shape information about the lesion 202 present on the front side of the fold 201, which is the visibility area. In this case, the detection processing unit 171 does not detect the peculiar shape present on the back side of the fold 201 , but detects the peculiar shape present on the front side of the fold 201 . In FIG. 6, when the detection processing unit 171 detects a peculiar shape, the notification processing unit 172 instructs the image processing unit 170 to display the notification image WI on the display device 18, for example. The image processing unit 170 superimposes the notification image WI on the optical image DP<b>1 and causes the display device 18 to display it via the display control unit 163 . In the optical image DP1 of FIG. 6, an elliptical notification image WI is superimposed on the lesion 202 and displayed, and an arc-shaped notification image WI is displayed on the outer circumference of the optical image DP1. The notification image WI is not particularly limited as long as it can be recognized by the operator, and colors, characters, presence/absence of icons, and the like can be applied.

The notification image WI only needs to make the operator aware of the possibility that the lesion 202 exists, and as shown in FIG. Both locations may be recognized.

The three-dimensional information inside the subject acquired by the LiDAR unit 76 may be stored in the memory 165 in parallel with the detection processing by the detection processing unit 171 . Based on the stored three-dimensional information within the subject, the three-dimensional model creation unit 173 of the CPU 162 can create a three-dimensional model. Also, the created three-dimensional model can be processed by the image processing unit 170 and displayed on the display device 18 via the display control unit 163 .

A processing method of the detection processing unit 171 is not particularly limited. For example, the memory 165 stores information about the shape of the actual lesion (hereinafter referred to as “lesion shape information”). A plurality of types of lesion shape information may be stored for each site of the subject to be observed. The detection processing unit 171 refers to the information about the shape of the lesion stored in the memory 165, and stores in the memory 165 for each predetermined area (determination area) from the three-dimensional information acquired from the LiDAR unit 76. It is also possible to obtain a similarity with the lesion shape information obtained, extract information of a region higher than a preset similarity threshold as peculiar shape information, and detect a peculiar shape based on the extracted peculiar shape information.

Further, the detection processing unit 171 may specify a three-dimensional shape from the three-dimensional information acquired by the LiDAR unit 76 and detect a shape different from other portions as a singular shape.

When the detection processing unit 171 detects the peculiar shape, it can notify the detection of the peculiar shape in real time. Further, the detection may be stored in the memory 165 as detection information. Based on the detection information stored in the memory 165, the notification processing unit 172 can notify that the peculiar shape has been detected at any timing.

As shown in FIG. 7, an optical image DP2 acquired by the imaging unit 100 is displayed on the display device 18 as an observation image when the endoscope 10 is positioned at the position P2. Similar to FIG. 6, optical image DP2 includes folds 201 present on the inner surface of large intestine 200. FIG.

Since the endoscope 10 has moved in the forward direction F, the lesion 202 on the front side of the fold 201 is not displayed in DP2. Also, at the position of the optical image DP2, the lesion 202 on the back side of the fold 201 is not present in the visible region, so the lesion 202 on the back side of the fold 201 is not displayed.

When the endoscope 10 is positioned at the position P2, unlike the position P1, the three-dimensional information acquired by the LiDAR unit 76 includes the lesion 202 present behind the fold 201, which is an invisible area of the imaging unit 100. The detection processing unit 171 detects the peculiar shape existing on the back side of the fold 201 because the peculiar shape information about the peculiar shape to be formed is included. In this case, in FIG. 7, when the detection processing unit 171 detects the peculiar shape, the notification processing unit 172 instructs the image processing unit 170 to display the notification image WI on the display device 18, for example. The image processing unit 170 superimposes the notification image WI on the optical image DP<b>1 and causes the display device 18 to display it via the display control unit 163 . Although the lesion 202 existing on the back side is not displayed in the optical image DP2 of the display device 18, the notification image WI for notifying that the existence of the lesion 202 has been detected is superimposed on the position corresponding to the lesion 202. displayed. In the optical image DP2 of FIG. 7, an elliptical curve notification image WI with a dashed line and an arrow-shaped notification image WI are displayed.

As shown in FIG. 8, an optical image DP3 acquired by the imaging unit 100 is displayed on the display device 18 as an observation image when the endoscope 10 is positioned at the position P3. The optical image includes folds 201 present on the inner surface of colon 200 . On the other hand, the optical image does not include the lesions 202 present on the front and back sides of the fold 201 .

When the endoscope 10 is positioned at the position P3, the three-dimensional information acquired by the LiDAR unit 76 includes information about the lesion 202 in the blind area that bends ahead of the path, which is the invisible area of the imaging unit 100. Contains singular shape information. Three-dimensional information inside the subject is input to the CPU 162 (see FIG. 5), and the detection processing unit 171 executes detection processing. Since the three-dimensional information includes peculiar shape information, the detection processing unit 171 detects the existence of the peculiar shape.

In FIG. 8, when the detection processing unit 171 detects a peculiar shape, the notification processing unit 172 instructs the image processing unit 170 to display the notification image WI on the display device 18, for example. The image processing unit 170 superimposes the notification image WI on the optical image DP3 and causes the display device 18 to display it via the display control unit 163 . It corresponds to the position of the lesion 202 present on the front side of the fold 201 in the area that bends ahead of the path and becomes blind. By displaying the notification image WI together with the optical image DP3, the operator can recognize the possibility that the lesion 202 exists in the blind area that bends ahead of the route. The notification image WI provides advance notification of the lesion 202 that is not visible. In the display device 18 of FIG. 8, an arc-shaped notification image WI is displayed on the outer periphery of the optical image DP3, and a dashed elliptic curve notification image WI is superimposed and displayed on the outside of the optical image DP3. The arc-shaped notification image WI around the optical image DP3 preferably has a different color from the arc-shaped notification image WI around the optical image DP1 in FIG.

As shown in FIG. 9, at the timing when the notification image WI is displayed, the operator operates the angle knobs 40, 40 (see FIG. 1) while confirming the optical image displayed on the display device 18 to take an image. The unit 100 directly observes the back side of the fold 201 . The imaging unit 100 acquires an optical image of the lesion 202 . The operator can visually recognize the optical image of the lesion 202, thereby enabling diagnosis and the like.

In addition, in FIGS. 6 to 8, the case where the notification process is performed using the notification image WI has been described. However, when the detection processing unit 171 detects the peculiar shape, the notification processing unit 172 may instruct the sound processing unit 166 to output sound from the speaker 167 . Also, the notification process may be executed simultaneously using the notification image WI and the sound. The installation position of the speaker 167 is not particularly limited, and it may be connected to the display device 18 or may be built therein.

FIGS. 6 to 8 show a case where the notification image WI is displayed in real time on the optical image DP3 at arbitrary timing as the endoscope 10 moves in the forward direction F. FIG. However, the notification image WI may be displayed on the optical image DP3 at a timing different from real time.

For example, when the endoscope 10 is moved in the forward direction F, the detection processing unit 171 detects a peculiar shape at a specific position based on the peculiar shape information included in the three-dimensional information acquired by the LiDAR unit 76. , the detection information (information indicating the singular shape) is stored in the memory 165 . Furthermore, the endoscope 10 is moved in the forward direction F to continue observation. When the endoscope 10 is moved in the backward direction B after observation and reaches a specific position, the notification processing unit 172 executes processing based on the detection information stored in the memory 165 . A notification image WI is displayed in the optical image DP3. In this way, notification processing can be performed at arbitrary timing. The operator can visually recognize the optical image of the lesion 202 at an arbitrary timing, enabling diagnosis and the like.

As described above, according to the embodiment, the risk of overlooking the lesion 202 can be reduced. If the lesion 202 having a unique shape is not detected, observation with the endoscope 10 is performed in a normal procedure. As a result, examination time can be shortened, and the risk of damage to the gastrointestinal tract can be reduced.

In the above description, the case where the lesion 202 existing on the back side of the fold 201 is detected as a peculiar shape and the notification image WI is displayed on the optical image DP3 has been described. However, the present invention is not limited to this, and even when the lesion 202 existing on the front side of the fold 201 is detected as a peculiar shape, the notification image WI may be displayed superimposed on the optical image DP. The risk of overlooking the lesion 202 can be reduced.

Next, a case of creating a three-dimensional model from the three-dimensional information inside the subject acquired from the LiDAR unit 76 will be described.

As shown in FIG. 5, the imaging unit 100 of the endoscope 10 acquires an optical image, and the LiDAR unit 76 acquires three-dimensional information inside the subject. The 3D model creating unit 173 creates a 3D model based on the 3D information. The three-dimensional information includes body cavity information about body cavities in the subject, and the three-dimensional model creation unit 173 can create a three-dimensional model of the body cavity of the subject.

The endoscope 10 is inserted into the digestive tract and observed while moving in the forward direction F. The endoscope 10 sequentially acquires three-dimensional information inside the subject. Therefore, as the endoscope 10 advances in the forward direction F, the three-dimensional model creating unit 173 sequentially creates three-dimensional models of the body cavity of the subject. The created three-dimensional model may be displayed on the display device 18 in sequence.

When the detection processing unit 171 detects a singular shape based on the singular shape information included in the three-dimensional information acquired by the LiDAR unit 76, the three-dimensional model creation unit 173 converts the position of the singular shape in the three-dimensional model to the position information. is stored in the memory 165 as . The three-dimensional model of the body cavity and the positional information of the peculiar shape are subjected to image processing and the like by the image processing unit 170 and can be displayed on the display device 18 .

When the endoscope 10 moves in the forward direction F and reaches the forwardmost position, the three-dimensional model creation unit 173 can create the position information of the unique shape throughout the subject and the three-dimensional model of the body cavity. The memory 165 may store the three-dimensional model of the body cavity of the subject and the positional information of the specific shape.

A position detection processing unit 174 of the CPU 162 shown in FIG. 4 acquires position information of the distal end portion 30 of the endoscope 10 .

The position detection processing unit 174 acquires the position and direction of the distal end portion 30 of the endoscope 10 as position information. The position information of the distal end portion 30 acquired by the position detection processing portion 174 is subjected to image processing and the like by the image processing portion 170, and on the display device 18, the positional information of the distal end portion 30 is combined with the positional information of the unique shape and the position of the body cavity. It can be displayed superimposed on the 3D model.

The position detection processing unit 174 may acquire position information from a detection sensor (not shown) mounted on the distal end portion 30 of the endoscope 10, for example. An endoscope position detection system using a magnetic sensor as a detection sensor can be applied.

The position detection processing section 174 may acquire the position information of the distal end portion 30 of the endoscope 10 from the three-dimensional model sequentially created by the three-dimensional model creating section 173 . For example, one of the created three-dimensional models is used as a reference and compared with the next created three-dimensional model. Calculate how much the characterizing portion of the reference three-dimensional model is enlarged or reduced in the next created three-dimensional model, and obtain the amount of movement. Positional information of the tip 30 can be estimated.

The three-dimensional model creation unit 173 may create a three-dimensional model by associating the acquired three-dimensional information, the optical image, and the positional information of the distal end portion 30 and store it in the memory 165 . A three-dimensional model is available after treatment with the endoscope 10 . For example, by using a high-quality optical image and peculiar shape information contained in three-dimensional information, it can be used as high-quality teacher data for machine learning.

FIG. 10 is a schematic diagram of an observation image and a three-dimensional model displayed on a display device. As shown in FIG. 10, the optical image DP and the three-dimensional model MD are displayed side by side on one screen on the display device 18 . In FIG. 10, the endoscope 10 that has been inserted once is advanced to the vicinity of the cecum and then retracted, and an image of the large intestine model 300 is created up to the end of the large intestine.

The optical image DP is an optical image obtained from the imaging unit 100 of the endoscope 10 . A three-dimensional model MD is created by the three-dimensional model creation unit 173 . The three-dimensional model MD includes a large intestine model 300, a fold model 301, and a lesion model 302, which is a peculiar shape model. Further included is an endoscope model 310 and a tip model 330 . The tip model 330 indicates the relative position in the large intestine model 300 based on the position information of the position detection processing section 174 . The operator can confirm the actual position of the distal end portion 30 of the endoscope 10 from the three-dimensional model MD.

Optical image DP displays the inner surface of large intestine 200 . The optical image DP displays a fold 201 and a notification image WI at a position corresponding to a peculiar-shaped lesion 202 (not shown).

Since the operator can confirm the notification image WI of the optical image DP and the lesion model 303 of the three-dimensional model MD, the risk of overlooking the lesion 202 behind the fold 201 can be reduced.

The case where the LiDAR unit 76 is arranged on the distal end surface of the distal end portion 30 of the endoscope 10 in FIG. 2 is illustrated. However, it is not limited to this position. FIG. 11 is an enlarged view of the distal end portion 30 of the endoscope 10. FIG. As shown in FIG. 11 , the LiDAR unit 76 may be arranged on the outer peripheral surface (side surface) of the distal end portion 30 .

2 and 11 show the case where one LiDAR unit 76 is arranged at the distal end portion 30 , at least one LiDAR unit 76 may be arranged at the distal end portion 30 . For example, when a plurality of LiDAR units 76 are arranged, the LiDAR units 76 can be arranged on the outer peripheral surface (side surface) of the distal end portion 30 at regular intervals along the circumferential direction centered on the longitudinal direction Ax. Similarly, at least one imaging unit 100 needs only to be arranged at the distal end portion 30, and a plurality of imaging units 100 can be arranged.

Next, a case where the LiDAR unit is applied to an ultrasonic endoscope having a transducer will be described. FIG. 12 is a schematic configuration diagram showing an example of configuration of an ultrasonic endoscope system. In addition, the same code|symbol may be attached|subjected to the same structure as the endoscope system shown in FIG. 1, and description may be abbreviate|omitted.

<Configuration diagram of ultrasonic endoscope system>
The endoscope system 1 includes a convex ultrasonic endoscope 11 , a light source device 14 , a processor device 16 , an ultrasonic processor device 17 and a display device 18 .

The ultrasonic endoscope 11 includes an operation section 22 in which an operation member is arranged, and an insertion section 24 provided on the distal end side of the operation section 22 and inserted into the subject. The insertion section 24 includes a flexible section 26, a curved section 28, and a distal section 30 in order from the proximal end toward the distal end. A transducer 90 is positioned at the tip 30 .

Unlike the endoscope 10 shown in FIG. 1, the operation section main body 34 includes an erection lever 50 for erecting an erection table 35 (see FIG. 13). A light source connector 57 and an ultrasonic wave connector 59 are provided at the tip of the universal cable 54 . A processor connector 58 is branched from the light source connector 57 . The light source connector 57 is detachably connected to the light source device 14 . The processor connector 58 is detachably connected to the processor device 16 . An ultrasound connector 59 is detachably connected to the ultrasound processor device 17 .

The ultrasonic processor unit 17 causes the transducer 90 to generate ultrasonic waves. Further, the ultrasound processor device 17 generates an ultrasound image based on echo signals from the observation target received by the transducer 90 . The ultrasound image is displayed on the display device 18 .

<Tip of ultrasonic endoscope>
FIG. 13 is an enlarged view of the distal end portion of the ultrasonic endoscope. As shown in FIG. 13 , the tip 30 includes a tip body 31 and a transducer 90 on the tip side of the tip body 31 . The transducer 90 has a convex ultrasonic transducer 91 .

An observation window 70 , two illumination windows 72 , and an air/water nozzle 75 are arranged on the inclined surface 32 of the tip body 31 . The inclined surface 32 is inclined in a direction in which the observation window 70 approaches the base end side from the illumination window 72 with respect to the longitudinal direction Ax.

A treatment instrument lead-out port 33 is provided on the proximal end side of the tip body 31 . A recessed standing table accommodating portion 37 that communicates with the treatment instrument outlet 33 is provided in the distal end portion main body 31 . The standing table 35 is arranged in the standing table accommodating portion 37 so as to be rotatable to change the direction of withdrawal from the treatment instrument outlet 33 .

The stand 35 is connected to the stand-up lever 50 (see FIG. 12) of the operation section main body 34 via an operation wire (not shown). When the operation wire is pushed and pulled by operating the erecting lever 50, the erecting table 35 rotates through the shaft, and the erecting angle of the erecting table 35 is changed.

An imaging unit 100 (see FIG. 3) similar to the endoscope 10 is arranged at a position corresponding to the observation window 70 of the tip body 31 . Furthermore, the ultrasonic endoscope 11 has a LiDAR unit 76 similar to that of the endoscope 10 arranged on the inclined surface 32 of the distal end body 31 . LiDAR unit 76 comprises a laser light source 78 and an optical sensor 80 .

The ultrasonic endoscope 11 can acquire an ultrasonic image inside the subject with the transducer 90, an optical image inside the subject with the imaging unit 100, and three-dimensional information inside the subject with the LiDAR unit 76. . The processor device 16 shown in FIG. 12 includes a CPU 162 and the like for processing optical images and three-dimensional information, like the processor device 16 shown in FIG.

FIG. 14 shows a state of observing the gastrointestinal tract with the ultrasonic endoscope 11. As shown in FIG. FIG. 14 shows the stomach 220, duodenum 222, and pancreas 224 within the subject. In general, an ultrasound image of the pancreas 224 is observed while the transducer 90 of the ultrasound endoscope 11 is pressed against the inner wall of the stomach 220 and pulled as if rubbing, that is, while scanning the transducer 90 . While pressing the transducer 90, only the inner wall of the stomach 220 can be seen in the optical image, and it is necessary to grasp the position of the transducer 90 only with the ultrasound image. It is very difficult to grasp the position using an ultrasound image, which makes the procedure long and hinders the early training of doctors.

Since the ultrasonic endoscope 11 of the embodiment can acquire three-dimensional information inside the subject by means of the LiDAR unit 76, the CPU 162 of the processor device 16 and the like can create a three-dimensional model of the body cavity of the subject. Furthermore, the position of the distal end portion 30 of the ultrasonic endoscope 11 acquired by the position detection processing unit 174 can be superimposed and displayed on the three-dimensional model of the body cavity of the subject.

FIG. 15 is a schematic diagram of a case where an ultrasonic image and a three-dimensional model are displayed on a display device. As shown in FIG. 15, the display device 18 displays the ultrasonic image UI and the three-dimensional model MD side by side on one screen.

The ultrasound image UI is an ultrasound image obtained from the transducer 90 of the ultrasound endoscope 11 . The 3D model MD is created by the 3D model creating unit 173 (see FIG. 4). The 3D model MD includes a stomach model 320 and a duodenum model 322 . Further included is an endoscope model 311 and a tip model 330 . The tip model 330 indicates the relative positions of the stomach model 320 and the duodenum model 322 based on the position information of the position detection processing unit 174 (see FIG. 4). The operator can confirm the actual position of the distal end portion 30 of the ultrasonic endoscope 11 from the three-dimensional model MD.

Since the operator can confirm the position of the distal end portion 30 of the ultrasonic endoscope 11 in real time, the transducer 90 can be easily scanned.

Since the ultrasonic endoscope 11 of the embodiment does not need to acquire three-dimensional information from CT (Computed Tomography), it is particularly useful in small-scale hospitals without CT. If CT is not used, the exposure risk of the subject is reduced. Furthermore, it is expected to shorten procedure time, promote the training of doctors, and promote the spread of EUS.

Although the present invention has been described above, the present invention is not limited to the above examples, and various improvements and modifications may be made without departing from the scope of the present invention.

<Others>
In the above embodiment, the hardware structure of the processing unit that executes various processes is various processors as shown below. For various processors, the circuit configuration can be changed after manufacturing such as CPU (Central Processing Unit), which is a general-purpose processor that executes software (program) and functions as various processing units, FPGA (Field Programmable Gate Array), etc. Programmable Logic Device (PLD), which is a processor, ASIC (Application Specific Integrated Circuit), etc. be

One processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types (eg, multiple FPGAs, or combinations of CPUs and FPGAs). may Moreover, a plurality of processing units can be configured by one processor. As an example of configuring a plurality of processing units in a single processor, first, as represented by a computer such as a client or server, a single processor is configured by combining one or more CPUs and software. There is a form in which a processor functions as multiple processing units. Secondly, as typified by System On Chip (SoC), etc., there is a form of using a processor that realizes the function of the entire system including a plurality of processing units with a single IC (Integrated Circuit) chip. be. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

Further, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

1 endoscope system 10 endoscope 11 ultrasonic endoscope 14 light source device 16 processor device 17 ultrasonic processor device 18 display device 22 operation section 24 insertion section 26 flexible section 28 bending section 30 distal section 31 distal section main body 32 Inclined surface 33 Treatment instrument outlet 34 Operation unit main body 35 Standing table 36 Grasping part 37 Standing table housing part 38 Bent stop tube 40 Angle knob 42 Lock lever 44 Air supply/water supply button 46 Suction button 48 Operation button 50 Standing lever 52 Treatment instrument introduction Port 54 Universal cable 56 Connector device 57 Light source connector 58 Processor connector 59 Ultrasonic connector 70 Observation window 72 Illumination window 74 Forceps port 75 Air/water nozzle 76 LiDAR unit 78 Laser light source 80 Optical sensor 82 Control circuit 90 Transducer 91 Super Acoustic oscillator 100 Imaging unit 101 Imaging lens 102 Imaging device 103 AFE
104 control circuit 110 light guide 140 light source 141 light source control circuit 161 image input controller 162 CPU
163 display control unit 165 memory 166 sound processing unit 167 speaker 168 operation unit 169 input interface 170 image processing unit 171 detection processing unit 172 notification processing unit 173 three-dimensional model creation unit 174 position detection processing unit 200 large intestine 201 fold 202 lesion 220 stomach 222 duodenum 224 pancreas 300 large intestine model 301 fold model 302 lesion model 310 endoscope model 311 endoscope model 320 stomach model 322 duodenum model 330 tip model Ax longitudinal direction DP optical image DP1 optical image DP2 optical image DP3 optical image MD Three-dimensional model P1 Position P2 Position P3 Position UI Ultrasonic image V1 Visual field range V2 Visual field range WI Notification image

Claims (13)

an insertion section having a distal end on the distal end side and inserted into the subject;
an imaging unit arranged at the distal end portion and capable of acquiring an optical image of the inside of the subject;
and a LiDAR unit arranged in the insertion section and capable of acquiring three-dimensional information within the subject,
The endoscope, wherein the LiDAR unit acquires, as three-dimensional information within the subject, singular shape information about a singular shape existing in a visible region and a non-visually visible region in an imaging region that can be imaged by the imaging unit. The endoscope according to claim 1, wherein said specific shape is a shape related to a lesion. 3. The endoscope according to claim 1, wherein said LiDAR unit acquires body cavity information regarding a body cavity of said subject into which said insertion portion is inserted, as three-dimensional information within said subject. an endoscope according to any one of claims 1 to 3;
a processor device comprising a processor and a memory for transmitting and receiving signals between the imaging unit and the LiDAR unit;
An endoscope system comprising:
The endoscope system, wherein the processor detects the peculiar shape based on the peculiar shape information obtained by the LiDAR unit. the memory stores information about the shape of the lesion;
5. The processor, from the three-dimensional information acquired by the LiDAR unit, detects the peculiar shape by obtaining a degree of similarity with the information on the shape of the lesion stored in the memory. An endoscopic system as described. The endoscope system according to claim 4 or 5, wherein said processor executes notification processing for notifying that said peculiar shape has been detected. 7. The endoscope system according to claim 6, wherein said processor stores detection of said peculiar shape in said memory, and performs notification processing for notifying said detection of said peculiar shape at arbitrary timing. . The endoscope system according to claim 6 or 7, wherein said processor executes notification processing by sound when detecting said singular shape. a display device connected to the processor device;
The processor according to any one of claims 6 to 8, wherein the optical image is displayed on the display device, and when the singular shape is detected, notification processing is performed by displaying on the display device. optic system. a display device connected to the processor device;
The LiDAR unit acquires body cavity information regarding a body cavity of the subject into which the insertion portion is inserted, as three-dimensional information within the subject,
The endoscope system according to any one of claims 4 to 9, wherein the processor creates a three-dimensional model of the body cavity of the subject based on the body cavity information acquired by the LiDAR unit. The endoscope system according to claim 10, wherein said processor stores the position of said peculiar shape in said three-dimensional model in said memory as position information. 12. The endoscope system according to claim 11, wherein the processor displays the three-dimensional model including the position of the unique shape on the display device based on the positional information stored in the memory. 13. The endoscope system of claim 12, wherein the processor displays the position of the tip in the three-dimensional model on the display device.

Images