JP2021003453A - Endoscope treatment system - Google Patents

Abstract

To provide an endoscope treatment system capable of preventing perforation without being affected by skills of an operator.SOLUTION: An endoscope treatment system 1 is for giving treatment of a subject by a treatment apparatus while imaging an inside of a subject and includes: an endoscope part 2 imaging the inside of the subject and acquiring imaging data; a determination part 3 making determination on the basis of the imaging data acquired by the endoscope part; a monitor part 4 displaying video based on the imaging data; and an electric knife device 5 capable of incision of the subject. In the determination part, it is determined by situations of the electric knife device reflected in the imaging data and, when it is determined to be dangerous, a danger signal is transmitted to at least either the monitor part or the electric knife device.SELECTED DRAWING: Figure 1

Description

The present invention relates to an endoscopic treatment system for treating a subject with a treatment device while imaging the inside of the subject.

Endoscopic Submucosal Dissection (ESD) has enabled endoscopic resection of all early-stage cancers, regardless of colon tumor size. Patent Document 1 discloses a medical system in which an endoscope and an electric knife are combined.

The medical system disclosed in Patent Document 1 prevents malfunction of the touch panel and communication failure between the endoscopic video processor and peripheral devices due to the influence of noise generated when the electrosurgical knife device outputs a high-frequency current. be able to.

JP-A-2018-167021

However, ESD is a method of peeling the submucosal layer after incising the mucosal layer and the muscularis mucosae, and while the submucosal layer is being peeled off, the tip of the electric scalpel is visually recognized by the endoscopic screen. Can not do it. That is, because the tip of the electric knife cannot be visually recognized, even if the submucosal layer is intended to be peeled off, the muscle layer may be damaged or cut to cause perforation depending on the skill of the operator. Perforation causes peritonitis and must be prevented.

Therefore, an object of the present invention is to provide an endoscopic treatment system capable of preventing the occurrence of perforation without being affected by the skill of the operator.

In order to achieve the above object, the endoscopic treatment system of the present invention is an endoscopic treatment system for treating a subject with a treatment device while photographing the inside of the subject, and is inside the subject. An endoscope unit that captures an image and acquires imaging data, a determination unit that makes a determination based on the imaging data acquired by the endoscope unit, and a monitor unit that displays an image based on the imaging data. A treatment device capable of incising a subject is provided, and the determination unit determines whether or not it is dangerous from the condition of the treatment device reflected in the imaging data, and if it is determined to be dangerous, the determination unit It is characterized in that a danger signal is transmitted to at least one of a monitor unit and the treatment device.

Here, the determination unit is the result of machine learning using the tagged learning data to which the determination result of whether or not it is dangerous is attached to the imaging data showing the situation of both the inside of the subject and the treatment device. The determination can be made according to the above.

Further, the treatment device is an electrosurgical knife device, and the danger signal can be a signal for stopping the output of the electrosurgical knife device. Further, the monitor unit may be configured to be notified by the danger signal that the situation is dangerous. Then, the imaging data is a moving image, and the positional relationship between the subject and the treatment device is sequentially changed.

The endoscopic treatment system of the present invention configured in this way includes a determination unit that makes a determination based on imaging data acquired by the endoscopic unit that images the inside of the subject, and a treatment that enables incision of the subject. It is equipped with a device. Then, the determination unit determines whether or not it is dangerous from the condition of the treatment device reflected in the imaging data, and if it is determined to be dangerous, transmits a danger signal to at least one of the monitor unit and the treatment device.

If the judgment unit determines a dangerous situation such as perforation, a warning signal is issued to the monitor unit, or the operation of the treatment device is restricted, the skill of the operator is sufficient. It is possible to prevent the occurrence of perforation without being affected by.

Then, if the determination by the determination unit is performed based on the result of machine learning using the tagged learning data in advance, the determination is performed with high accuracy even for the imaging data in a situation where the determination is difficult. You will be able to do it.

Further, if the treatment device is an electrosurgical knife device and the danger signal is a signal for stopping the output of the electrosurgical knife device, it is possible to prevent perforation from occurring even if the operator mistakenly operates the device. Further, even if the monitor unit is configured to be notified by a danger signal that the situation is dangerous, the operator can be prompted to perform an appropriate operation.

If the imaging data to be determined by the determination unit is a moving image, the dangerous situation can be accurately determined from the positional relationship between the subject and the treatment device, which changes rapidly during treatment, and the occurrence of perforation can be prevented. Can be done.

It is a block diagram explaining the structure of the endoscopic treatment system of this embodiment. It is explanatory drawing which showed typically the structure of the endoscopic treatment system of this embodiment. It is explanatory drawing which showed typically the cross section of the digestive tract wall. It is a flowchart explaining the flow of processing at the time of applying AI technique to a determination part. This is an example of moving image data showing the situation inside the subject and the handle of the electric knife.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating the configuration of the endoscopic treatment system 1 of the present embodiment, and FIG. 2 is an explanatory diagram schematically showing the configuration of the endoscopic treatment system 1.

The endoscopic treatment system 1 of the present embodiment includes an endoscopic unit 2 that images a human esophagus, a digestive organ such as the stomach and large intestine, a biliary tract, a bladder, and a bladder. It is equipped with a treatment device such as an electric scalpel device 5 for excising a lesion such as a tumor or a polyp in a sample.

The endoscope unit 2 is a device for observing the inside of the subject from the outside of the subject. The endoscope unit 2 includes an imaging unit 21 inserted into the subject and an image processing unit 22 that processes the imaging data captured by the imaging unit 21.

The image pickup unit 21 has an image pickup device such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). When an optical image of a part in a subject is imaged by this image sensor, an electric signal corresponding to the optical image is output.

The endoscope unit 2 includes a flexible endoscope in which a portion to be inserted into a subject is bent and a rigid endoscope in which the portion to be inserted into the subject is not bent. When a flexible endoscope is used, the imaging data captured by the CCD at the tip is electrically guided to the monitor unit 4 for display. When a rigid endoscope is used, imaging data is acquired by connecting a lens system from the tip to the eyepiece outside the body and connecting a camera head or the like to the eyepiece.

The image processing unit 22 performs signal processing of the imaging data obtained by imaging the inside of the subject. Here, the signal processing refers to a process of converting an electrical signal, which is imaging data obtained by imaging the inside of a subject, input by the imaging unit 21 into a video signal and outputting it.

The video signal converted from the captured data is displayed on the monitor unit 4 as a moving image. Further, the video signal converted from the captured data is recorded in the storage unit 6. The storage unit 6 can be configured by a large-capacity storage medium such as a hard disk drive or a solid state drive (SSD), a non-volatile memory such as a flash memory, or the like.

The endoscope unit 2 is provided with an operation unit, a control unit, and the like (not shown), and an operator or the like can perform operations for starting and ending imaging and adjusting conversion by the image processing unit 22. .. In addition, an image based on the captured data can be displayed on the monitor unit 4, and recording and stopping operations can be performed on the storage unit 6.

The electrosurgical knife device 5 is a treatment device that performs treatment such as incision or hemostasis on a target site (lesion) in a subject. The electric knife device 5 includes an electric knife 51 and an operation unit 52 for controlling the electric knife 51.

The operation unit 52 is a mechanism for the operator to give an instruction such as outputting a high-frequency current from the electric knife 51, and is composed of a foot pedal, a button, and the like. For example, when the operator steps on the foot pedal (52), the output switch 53 is turned on, and a high-frequency current is output from the tip portion 512 at the tip of the handle portion 511 of the electric knife 51 (see FIG. 2).

For example, a high-frequency current is output from the tip 512 of the electric knife 51 only while the operator is stepping on the foot pedal, and treatments such as incision and hemostasis of a target portion in the subject are performed. Here, the electrosurgical knife device 5 after startup has a high output state immediately before, during and immediately after the output of the high frequency current, and a standby state in which the output of the high frequency current is stopped.

The endoscopic treatment system 1 of the present embodiment includes a determination unit 3 that makes a determination based on the imaging data acquired by the endoscope unit 2. The determination unit 3 determines whether or not there is a danger in the treatment with the electric knife 51 or the like based on the imaging data in the subject obtained from the endoscope unit 2.

FIG. 3 schematically shows a cross section of the gastrointestinal wall as an example of the site of the subject. The gastrointestinal wall is composed of five layers from the space TI side to the intra-abdominal TO. Specifically, it has a laminated structure of mucosal layer W1, muscularis mucosae W2, submucosal layer W3, muscular layer W4, and serosa W5 from the lumen TI side.

When performing endoscopic resection of early cancer by endoscopic submucosal dissection (ESD), the submucosal layer W3 is peeled off after the mucosal layer W1 and the muscularis mucosae W2 are incised. However, if the fourth layer, the muscle layer W4, is damaged or torn, the wall structure will collapse, leading to perforation.

In short, perforation refers to a state in which the lumen TI of the digestive tract and the TO in the abdominal cavity communicate with each other. When the perforation cavity cannot be closed (clipping) endoscopically during the ESD operation, closure by emergency surgery is required. Even if the perforation cavity can be closed endoscopically, intestinal juice may leak from the wound and peritonitis may develop, in which case emergency surgery is indicated.

Therefore, the determination unit 3 determines whether or not a dangerous situation such as perforation is caused by the treatment with the electric knife 51 or the like. In short, it is determined whether or not it is dangerous from the situation between the electric knife 51 and the inside of the subject reflected in the imaging data acquired by the endoscope unit 2.

For example, when the imaging unit 21 of the endoscope unit 2 is inserted into the lumen TI of the digestive tract, imaging data showing the mucosal layer W1 can be obtained. Further, during the treatment around the lesion by the electric knife 51, in addition to the mucosal layer W1 and the submucosal layer W3 exposed by the incision, the handle portion 511 and the tip portion 512 of the electric knife 51 are treated. It is projected according to the progress (see Fig. 5).

Before inserting the electric knife 51 into the gastrointestinal wall, both the handle portion 511 and the tip portion 512 are shown, but when the tip portion 512 is inserted into the gastrointestinal tract wall for incision or hemostasis, imaging data is obtained. The image data shows only the handle portion 511 and the mucous membrane layer W1.

In short, the positional relationship between the situation in the subject such as the mucosal layer W1 and the electric knife 51 is reflected in the imaging data as a moving image while being sequentially changed. From these changes in the positional relationship, the state of treatment by the electric knife 51 is grasped, and it is determined whether or not the situation is dangerous.

The determination processing unit of the determination unit 3 is constructed by machine learning such as deep learning by artificial intelligence (AI) technology. FIG. 4 shows a flowchart for explaining the flow of processing when applying the AI technique to the determination unit 3.

For example, ESD is performed in the order of (1) marking, (2) local injection, (3) incision, (4) exfoliation of the submucosal layer, (5) excision of the lesion, (6) hemostasis, and (7) pathological examination. To. Then, in these procedures, the electric knife 51 is used in the treatments of (1) marking, (3) incision, (4) exfoliation of the submucosal layer, and (5) excision of the lesion, and a high-frequency current is output. It will be.

Here, "(1) marking" means to burn the surface of the mucosal layer W1 to make a mark, and "(3) incision" means to enclose the marking made around the lesion. A procedure for cutting the mucosal layer W1 and the muscularis mucosae W2 at the tip 512 of the knife 51. Further, "(4) peeling of the submucosal layer" refers to a procedure of peeling off the lesion together with the submucosal layer W3 at the tip portion 512 of the electric knife 51. Further, "(5) excision of the lesion" refers to a procedure of peeling the lesion with the tip 512 or the snare of the electric knife 51 until the lesion is cut off.

In the procedure using these electric scalpels 51, the tip portion 512 of the electric scalpel 51 cannot be visually recognized by the endoscopic screen while the submucosal layer W3 is being peeled off. That is, since the tip portion 512 of the electric knife 51 cannot be visually recognized, a phenomenon occurs in which the muscle layer W4 is damaged or cut depending on the skill of the operator even if the submucosal layer W3 is intended to be peeled off.

In short, depending on the skill of the operator, there are cases where perforation does not occur and cases where perforation occurs even when the tip portion 512 of the electric knife 51 cannot be visually recognized. In other words, even if the tip portion 512 is not displayed, the monitor unit 4 is viewing a moving image (imaging data) in which the situation in the subject and the positional relationship between the handle portion 511 of the electric knife 51 are projected. It can be seen that a highly skilled surgeon can finish the procedure without causing perforation.

Therefore, in step S1, moving image data at the time of endoscopic treatment is collected. For example, during the operation of 1000 to 2000 large intestine ESD (endoscopic submucosal dissection of large intestine malignant tumor), a moving image based on the imaging data taken by the endoscopic unit 2 and recorded in the storage unit 6 Use video data (video data). The moving image data recorded for each surgical example is divided for each of the above-mentioned ESD procedures, for example. Then, only the moving image data in which the status of the treatment in which perforation may occur is recorded is extracted. For example, moving image data recording the status of (3) incision, (4) exfoliation of the submucosal layer, and (5) excision of the lesion is extracted.

Subsequently, in step S2, a tag is added to the extracted moving image data. In short, the moving image data in which the electric knife 51 is peeling off the submucosal layer W3 is divided into two groups, a dangerous situation in which the muscle layer W4 is cut and a safe situation in which the muscle layer W4 is not cut. Tag each as "danger" or "safe". The tagged video data created in this way becomes the tagged learning data.

In step S3, machine learning is performed by the AI learning unit 7 (see FIG. 1). The AI learning unit 7 is provided in, for example, a supercomputer (NVIDIA DGX-station). In the AI learning unit 7, deep learning is performed using the tagged learning data created in step S2 as teacher data.

FIG. 5 shows an example of moving image data showing the situation of both the lumen TI inside the subject and the handle portion 511 of the electric knife 51. At the moment of this example, the mucosal layer W1, the submucosal layer W3 exposed by the incision, and the stalk 511 are shown, but the positional relationship between them changes from moment to moment. Further, at this stage, the tip portion 512 of the electric knife 51 is not shown.

The moving image data during the treatment of such tagged learning data is given as input data to the learning model of the AI learning unit 7, and the tag "danger" or "safety" given to the tagged learning data is used as a teacher. Learning is done.

Then, when the moving image data of the tagged learning data is input, the deep learning by the AI learning unit 7 is continued until the probability that the same determination result as the tag given in advance is obtained reaches a predetermined level. The trained model obtained in this way is the result of machine learning.

Therefore, in step S4, the AI determination processing unit (learned model), which is the learning product, is mounted on the determination unit 3 (see FIG. 1). The determination unit 3 can be configured by a general personal computer or a device provided with a CPU (Central Processing Unit) equivalent to the general personal computer.

The AI determination processing unit can continue to use the one installed first as long as high determination accuracy can be obtained for any treatment situation. In addition, the AI determination processing unit can be updated by causing the AI learning unit 7 to perform machine learning as appropriate based on the moving image data of the surgical examples that are accumulated every day even after mounting.

When the AI determination processing unit is mounted on the determination unit 3, treatment by the electric knife device 5 accompanied by the AI determination can be performed (step S5). Therefore, the treatment of large intestine ESD performed by the endoscopic treatment system 1 including the endoscopic unit 2, the electrosurgical knife device 5, and the determination unit 3 will be described with reference to FIG.

First, when the endoscope unit 2 is inserted into the lumen TI of the large intestine, the imaging data imaged by the imaging unit 21 at the tip is continuously sent to the image processing unit 22, converted into an image signal, and monitored. It is projected in Part 4.

The surgeon advances the tip of the endoscope unit 2 to the lesioned site while observing the condition of the lumen TI reflected on the monitor unit 4. Then, when the lesion is projected on the monitor unit 4, colonic ESD is started.

First, the area to be cut out is marked around the lesion with an electric knife 51 protruding from the tip of the endoscope portion 2, and a drug is injected (locally injected) into the submucosal layer W3 to make the lesion float.

Subsequently, the mucosal layer W1 and the muscularis mucosae W2 around the lesion are incised with an electric knife 51 so as to surround the marking. The incision by the electric knife 51 is performed by outputting a high frequency current from the tip portion 512.

Then, this high-frequency current is output only while the operator is stepping on the foot pedal (52). In short, by stepping on the foot pedal (52), the output switch 53 of the electrosurgical knife device 5 is turned on, and a high-frequency current is output from the tip portion 512.

Then, the situation (positional relationship) of both the lumen TI and the electric knife 51 during the incision is projected on the monitor unit 4 and can be observed by the operator, and is automatically detected by the AI determination processing unit of the determination unit 3. AI determination is made.

In short, the video signal converted by the video processing unit 22 is not only transmitted to the monitor unit 4, but also to the determination unit 3 at the same time. Then, when the intraoperative video signal is input to the AI determination processing unit of the determination unit 3, "danger" or "safety" is sequentially determined for the video data based on the input imaging data.

Then, while the determination result of "safety" continues to be obtained, the high frequency current is output from the tip portion 512 as if the foot pedal (52) was depressed. On the other hand, when the determination result of "danger" is obtained, the determination unit 3 generates a danger signal and transmits it to the monitor unit 4 and the output switch 53.

The danger signal transmitted to the monitor unit 4 via the image processing unit 22 outputs an alarm screen and sound to the monitor unit 4 to call attention to the operator. On the other hand, the danger signal transmitted to the output switch 53 forcibly turns off the output switch 53. That is, while the danger signal is being transmitted, even if the operator depresses the foot pedal (52), the high frequency current is not output from the tip portion 512.

It is possible to prevent the phenomenon of perforation from occurring due to the alarm displayed on the monitor unit 4 and the forced off of the output switch 53 in this way. The alarm and the forced off may be configured so that both are activated or one of them is activated.

Such monitoring of automatic recognition by the determination unit 3 is continued until the lesion is stripped from the submucosal layer W3, the lesion is resected, and the hemostasis treatment is completed. The resected lesion is collected and sent for pathological examination.

Next, the operation of the endoscopic treatment system 1 of the present embodiment will be described.
The endoscopic treatment system 1 of the present embodiment configured in this way includes a determination unit 3 that makes a determination based on the imaging data acquired by the endoscope unit 2 that images the inside of the subject, and a determination unit 3 of the subject. It is equipped with an electrosurgical knife device 5 capable of making an incision. Then, the determination unit 3 determines whether or not it is dangerous from the situation of the electric knife 51 reflected in the imaging data, and if it is determined to be dangerous, a danger signal is sent to at least one of the monitor unit 4 and the electric knife device 5. Send.

If the determination unit 3 determines a dangerous situation such as perforation, a warning signal is issued to the monitor unit 4, or the operation of the electrosurgical knife device 5 is restricted by the danger signal. It is possible to prevent the occurrence of perforation regardless of the skill of the operator.

Then, if such a determination by the determination unit 3 is performed based on the result of machine learning using the tagged learning data in advance, the determination can be made with high accuracy even for the imaging data in a situation where the determination is difficult. You will be able to do it.

Further, if the treatment device is the electrosurgical knife device 5 and the danger signal is a signal for stopping the output of the electrosurgical knife device 5, even if the operator mistakenly operates the foot pedal (52), perforation occurs. Can be prevented.

Further, even if the monitor unit 4 is configured to notify that the situation is dangerous by a danger signal, it is appropriate to alert the operator and confirm whether or not the treatment can be continued as it is. It is possible to encourage various operations.

Then, if the imaging data to be determined by the determination unit 3 is a moving image, a dangerous situation is accurately determined from the positional relationship between the subject and the electric knife 51, which changes rapidly during the treatment, and the occurrence of perforation is generated. Can be prevented.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes to the extent that the gist of the present invention is not deviated are described in the present invention. Included in the invention.

For example, in the above-described embodiment, the electric knife device 5 has been described as an example of the treatment device, but the present invention is not limited to this, and the present invention is applied to a treatment device such as a high-frequency snare or hot biopsi forceps. be able to.

Further, in the above-described embodiment, the determination unit 3 equipped with the AI determination processing unit has been described, but the present invention is not limited to this, and the determination unit may be used to determine a dangerous situation by image recognition.

Further, in the above-described embodiment, the treatment by endoscopic submucosal dissection (ESD) has been described as an example, but the present invention is not limited to this, and endoscopic mucosal resection (EMR) is not limited thereto. ) And all surgical procedures using a laparoscopic endoscope can also be applied to the present invention. For example, in the above embodiment, it has been explained that the object that should not be cut by the electric knife 51 is the muscular layer W4 in the gastrointestinal tract wall, but this object is applied to nerves, blood vessels, bronchi, the outer wall of the gastrointestinal tract, etc. It can be replaced to determine whether it is dangerous or not.

1: Endoscopic treatment system 2: Endoscopic unit 21: Imaging unit 3: Judgment unit 4: Monitor unit 5: Electric knife device (treatment device)
51: Electric knife 7: AI learning department

Claims (5)

It is an endoscopic treatment system for treating a subject with a treatment device while imaging the inside of the subject.
An endoscope unit that captures the inside of a subject and acquires imaging data,
A determination unit that makes a determination based on the imaging data acquired by the endoscope unit, and a determination unit.
A monitor unit that displays an image based on the captured data, and
Equipped with a treatment device capable of incising the subject,
The determination unit determines whether or not it is dangerous from the condition of the treatment device reflected in the imaging data, and if it is determined to be dangerous, transmits a danger signal to at least one of the monitor unit and the treatment device. An endoscopic treatment system characterized by The determination unit makes a determination based on the result of machine learning using tagged learning data with a determination result of whether or not it is dangerous for the imaging data showing the situation of both the subject and the treatment device. The endoscopic treatment system according to claim 1, wherein the endoscopic treatment system is performed. The endoscopic treatment system according to claim 1 or 2, wherein the treatment device is an electrosurgical knife device, and the danger signal is a signal for stopping the output of the electrosurgical knife device. The endoscopic treatment system according to any one of claims 1 to 3, wherein the monitor unit is notified by the danger signal that the situation is dangerous. The endoscopic treatment system according to any one of claims 1 to 4, wherein the imaging data is a moving image, and the positional relationship between the subject and the treatment device is sequentially changed. ..