JP6749020B2 - Endoscope navigation device - Google Patents

Description

TECHNICAL FIELD The present invention relates to an endoscopic device that performs diagnosis and treatment in the medical field.

Endoscope devices have been widely used in the medical field and industrial field from the past. The endoscope apparatus acquires an image of a target object at the tip of the insertion section by using the insertion section, an imaging element provided at the tip of the insertion section, and a lighting device, and displays the image by an imaging apparatus and an image display apparatus. However, the operator can perform the inspection. Further, a bending mechanism is provided at the tip of the endoscope, and for example, it is possible to perform a bending operation with two degrees of freedom in up and down and left and right. Due to this bending operation, for example, in the medical field, steering when inserting an endoscope into the digestive tract, controlling the visual field when observing or diagnosing tissue, and further when excising a tumor etc. The position and direction of the treatment tool can be controlled.

Conventionally, when inserting an endoscope into the digestive tract, the operator grasps the bending state of the endoscope and the state of the insertion portion while performing a bending operation according to each region of the lumen through which the endoscope passes. It is necessary to follow the procedure for moving the insert part back and forth and rotating it. It took a great deal of experience to acquire these procedures.

For example, Japanese Patent No. 2710384 detects the insertion direction of the endoscope by extracting a dark region of the endoscope image, and the endoscope operator sees the detected insertion direction of the endoscope and performs bending operation and insertion. It has been devised to perform an operation to insert the endoscope, or to insert the endoscope by automatically pointing the distal end portion in the detected insertion direction.

Patent No. 2710384

According to Japanese Patent No. 2710384, the insertion direction of the endoscope is detected by extracting the dark area of the endoscope image, and the tip portion is automatically oriented. However, what direction is the tip portion in which direction? As to whether to direct the input operation, there was no description other than to direct the tip toward the dark area. In the actual insertion of the endoscope, the dark part may be brought to the center of the endoscopic image or may be brought to the lower part of the endoscopic image. It was necessary to have a means for performing the operation easily based on the recognition.

Further, the dark part is only one index in the operation of the endoscope, and the position of the bubble appearing in the endoscopic image, the curved state of the endoscope, and the state of the insertion part are also used as indices in addition to the index. It is necessary to go through the procedure of the bending operation and the operation of advancing/retreating/rotating the insertion portion for each part of the lumen through which is passed. It took a great deal of experience to acquire these procedures.

The computer navigates the procedure of the bending operation and the operation of advancing/retreating/rotating the insertion portion according to the region of the lumen through which the endoscope passes. Furthermore, the information on the insertion of the endoscope by navigation is stored in a database, and the machine learning is performed based on the database to update the navigation program and improve the accuracy. Specifically, the following configuration is used. Characterize.

The endoscope navigation device of the first invention is
In the endoscopic navigation device,
An imaging device for endoscopic images,
An image processing device that analyzes and detects image information data relating to a feature amount of a target object that is an index of endoscopic operation displayed in the endoscopic image in the endoscopic image of the endoscopic image ,
A function to detect insertion state data consisting of one or more of the bending state of the endoscope, rotation of the insertion portion, and insertion length,
Timing and operation amount of the bending operation of the endoscope and the operation of rotating and advancing and retracting the insertion portion in the operation of inserting the endoscope into the gastrointestinal lumen using the image information data and the insertion state data regarding the characteristic amount as parameters. A navigation program in which the procedures regarding
The endoscope rotation and retreat of the bending operation and the insertion portion of the said endoscope derived by referring to the navigation program the parameters of the image information data and the insertion status data relating to the feature amount based upon insertion operations The present invention is characterized by having a navigation function of presenting at least two or more procedures relating to operation timing and operation amount simultaneously or continuously with images or sounds.

The endoscope navigation device of the second invention is
In the endoscopic navigation device,
A function of recording the insertion procedure as navigation data,
A function of recording the contents of the operation of the endoscope as operation data based on the navigation data,
A function to calculate effect evaluation result data that measures and evaluates the result of endoscope operation,
A function of associating the image information data, the insertion state data, the navigation data, the operation data, and the effect determination result data with each other in a database,
A function for machine learning the relationship between the data accumulated in the database,
And a function of changing the navigation program by using a result of the machine learning.

The endoscope navigation device of the third invention is
In the endoscopic navigation device,
A step of analyzing and detecting image information data relating to the feature amount of the target object which is an index of the endoscopic operation displayed on the endoscopic image in the endoscopic image of the endoscopic image;
Bending state of the endoscope, rotation of the insertion portion, a step of detecting insertion state data consisting of any one or more of the insertion length ,
Timing and operation amount of the bending operation of the endoscope and the operation of rotating and advancing and retracting the insertion portion in the operation of inserting the endoscope into the gastrointestinal tract, using the image information data and the insertion state data regarding the characteristic amount as parameters. A step of converting the procedure related to data into a navigation program in advance,
In the operation of inserting the endoscope, the endoscope in the operation of inserting the endoscope into the gastrointestinal lumen is referred to by referring to the navigation program based on the parameters of the image information data and the insertion state data regarding the feature amount. A step for guiding the procedure regarding the operation amount of the mirror,
A step of presenting at least two or more procedures relating to the operation amount of the endoscope in the operation of inserting the endoscope into the lumen of the digestive tract simultaneously or continuously with an image or a sound. ..

The operation of the endoscope is facilitated, and the accuracy of diagnosis and treatment by the endoscope is improved and the time is shortened without requiring the operator to have high skill or skill.

It is a figure which shows the structure of the system of an endoscope navigation apparatus. It is a figure which shows the structure of an endoscope insertion position sensor. It is a figure which shows the operation interface which touches the dark part in an endoscopic image, and performs a bending operation. FIG. 5 is a diagram showing a navigation menu in the insertion operation of the colonoscope of the first embodiment. FIG. 6 is a diagram showing a procedure in an insertion operation of the colonoscope of the first embodiment. FIG. 10 is a diagram showing a flowchart performed in machine learning of navigation in the insertion operation of the colonoscope of the second embodiment.

(Example 1)
This embodiment relates to navigation in inserting an endoscope. The configuration of this embodiment is shown in FIG. The endoscope device 1 includes an insertion portion 10, an image sensor 2 provided at the tip of the insertion portion 10, an illumination device 3, and an illumination fiber 4 that guides illumination light to the tip of the insertion portion 10. The image signal of the object at the end of the unit 10 is acquired, the image is formed by the imaging device 5, and the image is displayed by the display monitor 17. As a result, the operator can perform endoscopic diagnosis/treatment by viewing the endoscopic image. Further, a bending mechanism 7 is provided at the tip of the endoscope, and for example, it is possible to perform a bending operation with two degrees of freedom of up, down, left and right. Due to this bending operation, for example, in the medical field, steering when inserting an endoscope into the digestive tract, controlling the visual field when observing or diagnosing tissue, and further when excising a tumor etc. The position and direction of the treatment tool can be controlled.

For the bending operation, the bending mechanism 7 including the connected bending tube 8 provided at the distal end of the endoscope, the pulley 12 provided on the operating portion 11 at hand, and the both along the insertion portion 10. It is adapted to be coupled by a deployed wire 9 and to be transmitted by drive. The pulley 12 is connected to a large gear 13, the large gear 13 meshes with a small gear 14, and the small gear 14 is connected to a motor 15. Further, the motor 15 is provided with an encoder 16 that detects a rotational position.

A display monitor 17 for endoscopic images is provided near the operation unit 11, and a touch panel 18 is integrated with the display monitor 17. A monitor for displaying an endoscopic image may be provided separately from the display monitor 17.
The encoder 16 and the motor 15 are electrically connected to the motor control circuit 19. The motor control circuit 19 is connected to the CPU 20. The display monitor 17 is connected to the image processing device 21, and the touch panel 18 is connected to the CPU 20. The signals of the imaging device are connected to the image processing device 21. The image processing device 21 is connected to the CPU 20.

Further, the insertion portion position sensor 30 can detect the advance/retreat position and the rotation position of the insertion portion 10. The insertion portion position sensor 30 is connected to the CPU 20.

FIG. 2 shows the structure of the insertion portion position sensor 30 which detects the advance/retreat position and the rotation position of the insertion portion 10. The illumination 31 and the image sensor 32 are provided on the inner surface of the cylindrical portion into which the insertion portion 10 of the endoscope can be loosely inserted, and the graduation line 33 and the insertion length marked on the outer periphery of the insertion portion 10 of the endoscope are set. The numeral 34 shown is read by the image sensor 32, and the insertion position of the insertion portion 10 is detected from the scale line 33 and the longitudinal position X of the numeral 34, and the rotational position of the insertion portion 10 is detected from the circumferential position Y of the numeral 34. be able to. FIG. 2 exemplifies that the position Y in the rotational direction of the center of the numeral “30” and the position X in the longitudinal direction are obtained by the image sensor 32, and the insertion length and the rotational position are calculated by the formulas in the figure. There is.

The insertion portion position sensor 30 is installed so that the positional relationship with the entrance of the lumen into which the endoscope is inserted does not shift. For example, in the case of an upper endoscope, it is built into the mouthpiece that is added to the mouth, or is fixed to the mouthpiece. Fixed through. When the insertion portion position sensor 30 is installed away from the mouth or anus, the offset distance of the insertion length displayed on the display monitor 17 described later can be corrected by inputting the distance to the CPU 20. ..

Further, here, an example is shown in which the advance/retreat position and rotation of the insertion portion 10 are detected using the scale line 33 and the numeral 34 indicating the insertion length printed on the insertion portion 10 of the conventional endoscope. The contents described in the insertion portion 10 of the endoscope are not limited to this. In order to detect the advance/retreat position and the rotation position 44 of the insertion portion 10, for example, a symbol that can identify the insertion length instead of a number or a line that indicates the rotation direction may be used. .. Further, instead of printing on the surface of the insertion section 10, a Hall element may be embedded, and instead of the image sensor 32, a magnetic sensor may detect the advance/retreat position and the rotation position of the insertion section 10. Actually, since the numbers printed on the insertion section 10 of the conventional endoscope are printed every 10 cm, in order to always recognize the numbers as images, the length of the insertion section position sensor 30 is at least 10 cm. Because of the above, printing the numbers at a short pitch and providing a symbol for recognizing the length for each scale line shortens the length of the insertion portion position sensor 30, and the mouthpiece for the upper endoscope. Contributes to not extending the length of, or limiting the working space near the anus with a lower endoscope.

A transparent sleeve-shaped sterilization cover 35 is detachably provided in the insertion hole of the insertion portion position sensor 30 for inserting the endoscope, so that the insertion portion position sensor 30 may contaminate the insertion portion 10 of the endoscope. Is being prevented.

The CPU 20 calculates a movement vector (called an input movement vector) when the touch panel 18 is touched with a finger and dragged. A table of the correspondence between this input movement vector and the movement vectors of the vertical and horizontal motors 15 is stored in the internal memory of the CPU 20, and the CPU 20 refers to this table to indicate the position and/or speed of the motor control circuit 19. Send a command. As a result, the motor 15 rotates, the bending mechanism 7 operates via the drive transmission mechanism, and the endoscopic image moves.

On the other hand, the image processing device 21 and the CPU 20 extract the feature amount of the endoscopic image including the touched portion of the touch panel 18 and its surroundings, and obtain a movement vector (called an output movement vector) on the image from the feature amount. .. Further, the CPU 20 obtains a difference between the input movement vector and the output movement vector, and performs feedback control by sending a position and/or speed command to the motor control circuit 19 so that the difference approaches zero.

In the following, as described above, “the image processing device 21 and the CPU 20 extract the feature amount of the endoscopic image including the touched portion of the touch panel 18 and its periphery, and move the object having the feature amount on the image. An example of a means for "obtaining a vector (output movement vector)" will be described.

For example, the block matching method, which is also used for electronic image stabilization of video, is used. The CPU 20 stores the images of the part where the touch panel 18 is touched with a finger and the part including the surroundings, and responds to the changed image including the movement, rotation and enlargement/reduction by the bending operation while shifting the overlapping position. The sum of absolute differences (SAD: Sum of Absolute Difference) of the pixels to be calculated is calculated, and the position where SAD is minimized is detected as the matching position. The movement vector to this matching position is obtained as the output movement vector described above.

At this time, pixels with the feature amount extracted from the endoscopic image are sampled as representative points, and matching is performed by shifting the overlapping positions of the images of the representative points, that is, using the representative point matching method, the speed is increased Can be reduced.

Alternatively, it is possible to use an optical flow mainly used for detection of a moving object or analysis of its motion. This represents the distribution of the movement vector between frames of the representative points on the image. Using this, the movement direction and speed of the portion touching the touch panel 18 with a finger and the surrounding area, and further the depth are related. You can get information. That is, with the optical flow, information about the depth of the object can also be used as the bending operation of the endoscope.

Subsequently, referring to FIG. 3, an operation interface for performing a bending operation by touching the dark portion 53 of the endoscopic image 6 with the touch panel 18 and dragging the touch panel 18 in a desired direction to assist the insertion of the endoscope. Will be described.

For example, in order to insert a colonoscope through the anus and go through the rectum, sigmoid colon, descending colon, transverse colon, and ascending colon so that the tip reaches the large intestine cecum, as opposed to stool migration. One guideline for operating the endoscope is the position of the dark portion 53, which is distant from the lumen. The illumination light from the distal end of the endoscope is less likely to reach the distant portion of the lumen, and as a result, the darker the image is, the farther away. Utilizing this fact, insertion of an endoscope aiming at the dark portion 53 has been conventionally performed in actual clinical practice. In this embodiment, as shown in FIGS. 3A and 3B, the image of the dark portion 53 is touched on the touch panel 18, and the touch panel 18 is dragged in a desired direction to perform a bending operation, and the internal view is performed. Assist in inserting the mirror.

FIG. 4 shows an example of a navigation menu 56 displayed on the display monitor 17 when the colonoscope is inserted. Large intestine model image 40 in which the large intestine is seen from the right side of the body, Large intestine model image 41 in which the large intestine is seen in front of the body, and Endoscope model image 42 of the endoscope superposed on the large intestine image, and The curved state 43, the rotation position 44 and the upward marker 45 of the endoscopic image, the curved arrow 46 for instructing the bending operation, the forward arrow 47 and the backward arrow 48 indicating the forward/backward movement of the insertion portion 10, and the insertion portion. A rotation arrow 49 for instructing a rotation operation of 10, insertion length 50 indicating the length of insertion of the insertion portion 10, and insertion method selection 51 are displayed together with the endoscopic image 6. These are the rotational position 44/insertion length 50 of the insertion part 10 detected by the insertion part position center 30 and the vertical and horizontal positions of the bending part detected by the encoder 16 via the motor 15 and the drive transmission mechanism, that is, It is calculated and displayed by the CPU 20 from the curved state 43 and the position of the dark portion 53 and/or the bubble 54 designated as the target of the endoscopic image 6.

FIG. 5 shows a part of the contents displayed as the navigation menu 56 of FIG. 4 in the step of inserting the colonoscope. First, select the insertion method of the colonoscope. As a method of inserting a colonoscope, there are a loop insertion method, a shaft holding shortening insertion method, and the like. A navigation program corresponding to these insertion methods is prepared in advance, and one of the insertion methods of the colonoscope is first selected. The following steps are examples of navigation when the axis-holding shortening insertion method is selected.

For colonoscopies, the patient is placed in the left lateral decubitus position (left side down) at the beginning of insertion. First, the tip of the endoscope is passed through the hole of the insertion portion position sensor 30, and insertion is started from the anus. At this time, an arrow is displayed on the display monitor so that the rotation position 44 faces the direction of 9 o'clock. For example, as shown in FIG. 5A, when the rotational position 44 of the insertion portion 10 detected by the insertion portion position sensor 30 is 12:00, the rotation arrow 49 is displayed so as to point to 9 o'clock. When the operator grips the insertion portion 10 and rotates it in the direction indicated by the rotation arrow 49, “STOP” is displayed as shown in FIG. 5B at the time when the rotation position 44 is 9:00. At this time, as shown in FIG. 5C, the dark portion 53 is displayed at 12:00 and the bubble 54 is displayed at 3 o'clock in the endoscopic image 6.

The positions of the dark portion 53 and the bubble 54 are measured by the image processing device 21 and the CPU 20, and when it is confirmed that they are in the 12 o'clock and 3 o'clock directions respectively, the insertion portion 10 is advanced as shown in FIG. 5(4). A forward arrow 47 is displayed. According to this, the operator grips the insertion portion 10 and starts moving forward. While the bubble 54 keeps the direction at 3 o'clock, as the dark portion 53 moves downward from the upper portion of the endoscopic image 6 as shown in FIG. When the distance becomes 13 cm, "STOP" is displayed at the tip of the forward arrow 47 as shown in Fig. 5(6). At this time, in the endoscopic image 6, as shown in FIG. 5(7), strong lumen bending and the dark portion 53 appear in the 7 o'clock direction.

Here, it is confirmed by the image processing of the image processing device 21 that the dark portion 53 is at the bottom and the bubble 54 is at the 3 o'clock direction, and further, the length inserted from the anus of the insertion portion 10 of the endoscope, that is, the insertion length. When it is confirmed that 50 is 13 cm, an arrow is displayed on the display monitor so that the rotation position 44 turns counterclockwise to the direction of 3 o'clock, as shown in FIG. 5(8). When the operator holds the insertion portion 10 and rotates the insertion portion 10 in the direction indicated by the arrow, "STOP" is displayed at the time when the rotational position 44 is 3 o'clock, as shown in FIG. The endoscopic image 6 at that time is displayed in the direction of 12 o'clock in the dark portion 53 and 9 o'clock in the bubble 54, as shown in FIG. 5(10).

The positions of the dark portion 53 and the bubble 54 are measured by the image processing device 21 and the CPU 20, and when it is confirmed that they are in the directions of 12 o'clock and 9 o'clock respectively, as shown in FIG. Is presented in the upward direction. Looking at this, the operator touches the dark portion 53 above the endoscopic image 6 on the touch panel 18 and drags it downward. The operation of touching and dragging downward is repeated, and when the upper curve becomes 120°, the upward curve arrow 46 disappears and “STOP” is displayed as shown in FIG. 5(12). As a result, as shown in the large intestine perspective model image 40 when viewed from the right side of the body in FIG. 5(13), the curved portion 55 is in a state of being caught by the bend at the start portion of the sigmoid colon.

Then, as shown in FIG. 5(14), a backward arrow 48 for instructing the backward movement of the insertion portion 10 is displayed. According to this, when the operator grasps the insertion part 10 and slowly retracts it, the large intestine sigmoid colon gradually moves to the anus side as shown in the large intestine perspective model image 40 viewed from the right side of the body in FIG. 5(15). As a result, the dark portion 53 appears on the right side or the left side above the endoscopic image 6.

The position of the dark portion 53 is measured by the image processing device 21 and the CPU 20, and a left rotation arrow 49 is displayed when the position of the dark portion 53 is the upper left side, and a right rotation arrow 49 is displayed when the dark portion 53 is the upper right side. A rotation arrow 49 is displayed. FIG. 5 (16) shows an example in which the right rotation arrow 49 is displayed when the dark portion 53 appears on the upper right side. The operator applies a right twist according to the direction of the rotation arrow 49 of the insertion portion 10 as shown in FIG. As a result, the sigmoid colon of the large intestine is attracted, and the lumen of the descending portion of the sigmoid colon is opened, as shown in the colon perspective model image 40 viewed from the right side of the body in FIG.

Finally, as shown in FIG. 5(19), in the endoscopic image 6, the dark portion 53 appears wide open and moves toward the center. When the image processing device 21 and the CPU 20 measure this state, that is, that the dark portion 53 is at the center and the bubble 54 is in the 9 o'clock direction, as shown in FIG. 49 and the backward arrow 48 disappear and "STOP" is displayed.

The above steps show an example of the navigation of the present invention in the case of inserting the sigmoid colon, which is considered to be the most difficult to insert a colonoscope. Proceed with the insertion further, and perform appropriate navigation in the same manner as described above until it finally reaches the cecum. Thereby, conventionally, the operator judges the bending operation, the rotation of the insertion part 10 and the operation of advancing and retreating based on the experience of the position of the dark part 53, the position of the bubble 54, and the rotation and length of the insertion part 10. However, the operation of inserting the endoscope can be supported by navigating the procedure of the determination and the operation as shown in FIG.

Since the insertion length 50 and the rotation position 44 displayed on the screen can be recorded simultaneously with the endoscopic image 6 of the lesion, for example, when performing colonoscopy again at a later date for follow-up observation, the lesion can be recorded. It is useful because the location of can be confirmed accurately and efficiently.

The colon perspective model image 40 viewed from the right side of the body displayed in FIG. 4, the colon perspective model image 41 viewed from the front of the body, and the endoscope insertion part model image 42 superimposed on the colon perspective image are the colon It is an image using a computer graphics considering the anatomy of. Anatomical information such as extension and deformation of the large intestine using the current insertion length 50 of the insertion section 10, the rotational position 44, the curved state 43, and the data of these processes from the start of insertion to the current position. Is an image created by referring to.

In the present embodiment, an example in which the bending operation is electrically driven by the motor 15 is shown, but a manual bending operation may be performed. Further, as a means for detecting the bending state, a sensor may be provided directly on the bending portion.

(Example 2)
The present embodiment relates to machine learning for navigation in the insertion operation of the colonoscope in the first embodiment. As shown in FIG. 6, the navigation program 60 is a program that incorporates the rules of thumb accumulated by a doctor who is experienced in colonoscopic insertion. Specifically, the image information of dark areas, bubbles, and lumen characteristics appearing in an image In addition, the endoscope is inserted using a predetermined parameter 66 from the state data 61 such as the insertion portion and the curved shape of the endoscope, and further, patient information, insertion method, body position, operator information, pretreatment state, and the like. Assistance of insertion by deriving navigation data 62, which is operation output information such as forward/backward movement, rotation, bending operation, air supply/water supply/suction operation, abdominal pressing, body position change, etc. Is to do.

In the navigation program 60, the contents of the actual endoscope operation according to the navigation data 62 are recorded as operation data 67, and as a result of the operation, patient pain, insertion time, patient monitor, etc. are measured in the insertion process, It has a function of analyzing these and calculating the effect determination result data 63 for evaluating the performance of the insertion method. Further, the above-mentioned state data 61, navigation data 62, operation data 67, and effect determination result data 63 can be accumulated in the database 64 at any time while being associated with each other. The database 64 is installed on the cloud, is connected by a network, and shares the database 64 with a plurality of endoscope devices.

Machine learning 65 using an AI algorithm 68 such as a Convolutional Neural Network (CNN) 69 is performed on this database, and the relationship between the effect determination result data 63 and predetermined parameters 66, navigation data 62, and operation data 67. Derive. The predetermined parameter 66 is changed to improve the performance of the navigation program 60 so that the effect determination result data 63 becomes higher. The means for machine learning 65 is not limited to the CNN described above.

The above embodiment may be configured as follows.
1. The endoscopic navigation device according to the second invention, wherein the endoscopic image data is a bubble and/or a dark part.
2. The endoscope navigation device according to the first invention or the second invention, characterized in that the insertion portion position sensor is provided with a sterilization sleeve for preventing contamination of the insertion portion of the endoscope.
3. The endoscopic navigation device according to the first invention or the second invention, wherein the position of the lesion can be recorded by displaying and recording the insertion length and the rotational position together with the endoscopic image.
4. Characterized by including a colon fluoroscopic model image created by computer graphics using the insertion state data and operation procedure data from the start of insertion to the present, and an endoscope insertion part model image superposed on the model image The endoscope navigation device according to the first invention and the second invention.
5. The endoscope navigation system according to the first and second inventions, characterized in that the insertion operation data includes insertion section advancing/retreating/rotating operation, bending operation, air supply/water supply/suction operation, abdominal pressure, and postural change Stationation device.
6. The endoscopic navigation device according to the second aspect of the invention, wherein the effect determination result data includes patient pain, partial and total insertion time, and patient monitor.

DESCRIPTION OF SYMBOLS 1 Endoscope device 2 Image pick-up element 3 Illumination device 4 Illumination fiber 5 Imaging device 6 Endoscopic image 7 Bending mechanism 8 Bending tube 9 Wire 10 Insertion part 11 Operation part 12 Pulley 13 Large gear 14 Small gear 15 Motor 16 Encoder 17 Display monitor 18 Touch panel 19 Motor control circuit 20 CPU
21 Image Processing Device 30 Insertion Position Sensor 31 Illumination 32 Image Sensor 33 Scale Line 34 Number 35 Sterilization Cover 40 Colon Fluoroscopy Model Image Seen from Right Side of Body 41 Colon Fluoroscopy Model Image 42 Seen From Front of Body 42 Insertion Model Image 43 Curved state 44 Rotation position 45 Upward marker 46 Curved arrow 47 Forward arrow 48 Backward arrow 49 Rotation arrow 50 Insertion length 51 Insertion method selection 53 Dark section 54 Bubble 55 Curved section 56 Navigation menu 60 Navigation program 61 State data 62 Navigation data 63 Effect Judgment result data 64 Database 65 Machine learning 66 Predetermined parameters 67 Operation data 68 AI algorithm 69 Convolutional Neural Network

Claims (3)

In the endoscopic navigation device,
An imaging device for endoscopic images,
An image processing device that analyzes and detects image information data relating to a feature amount of a target object that is an index of endoscopic operation displayed in the endoscopic image in the endoscopic image of the endoscopic image ,
A function to detect insertion state data consisting of one or more of the bending state of the endoscope, rotation of the insertion portion, and insertion length,
Timing and operation amount of the bending operation of the endoscope and the operation of rotating and advancing and retracting the insertion portion in the operation of inserting the endoscope into the gastrointestinal lumen using the image information data and the insertion state data regarding the characteristic amount as parameters. A navigation program in which the procedures regarding
The endoscope rotation and retreat of the bending operation and the insertion portion of the said endoscope derived by referring to the navigation program the parameters of the image information data and the insertion status data relating to the feature amount based upon insertion operations An endoscopic navigation device, comprising: a navigation function of presenting at least two or more procedures related to operation timing and operation amount with images or sounds simultaneously or successively. In the endoscopic navigation device,
A function of recording the insertion procedure as navigation data,
A function of recording the contents of the operation of the endoscope as operation data based on the navigation data,
A function to calculate effect evaluation result data that measures and evaluates the result of endoscope operation,
A function of associating the image information data, the insertion state data, the navigation data, the operation data, and the effect determination result data with each other in a database,
A function for machine learning the relationship between the data accumulated in the database,
Using the result of the machine learning, an endoscope navigation system according to claim 1, characterized in that a function of changing the navigation program. In the endoscopic navigation device,
A step of analyzing and detecting image information data relating to the feature amount of the target object which is an index of the endoscopic operation displayed on the endoscopic image in the endoscopic image of the endoscopic image;
Bending state of the endoscope, rotation of the insertion portion, a step of detecting insertion state data consisting of any one or more of the insertion length ,
Timing and operation amount of the bending operation of the endoscope and the operation of rotating and advancing and retracting the insertion portion in the operation of inserting the endoscope into the gastrointestinal lumen using the image information data and the insertion state data regarding the characteristic amount as parameters. A step of converting the procedure related to data into a navigation program in advance,
The endoscope in the operation of inserting the endoscope into the gastrointestinal lumen by referring to the navigation program based on the parameters of the image information data and the insertion state data relating to the feature amount at the time of the operation of inserting the endoscope A step for guiding the procedure regarding the operation amount of the mirror,
A step of presenting at least two or more procedures relating to the operation amount of the endoscope in the operation of inserting the endoscope into the lumen of the digestive tract simultaneously or continuously with an image or a sound. Endoscope navigation device.