WO2018046092A1

WO2018046092A1 - Method for operating an endoscope, and endoscope

Info

Publication number: WO2018046092A1
Application number: PCT/EP2016/071334
Authority: WO
Inventors: Thomas Engel; Maximilian Fleischer; Alexander Michael Gigler; Elfriede Simon
Original assignee: Siemens Aktiengesellschaft
Priority date: 2016-09-09
Filing date: 2016-09-09
Publication date: 2018-03-15

Abstract

The invention relates to an endoscope and a method for operating an endoscope, said method for operating an endoscope having the following steps: a) in situ detecting a measurement region by capturing at least one at least two-dimensional image of at least one tissue part of a patient, b) analyzing the image content of each individual captured image, c) detecting at least one significant structure for each captured image on the basis of the analysis, d) determining at least one significant structure which can be continually detected in a sequence of sequential captured images, and e) generating an image by joining at least parts of the sequence, in particular at least two captured images of the sequence, in order to form an image with a larger volume than the volume of the images of the individual captured images, wherein properties of the joining process, in particular the location and/or dimensions and/or number of overlaps, of sub-regions of the captured images are determined, in particular in the manner of a stitching process, on the basis of at least one of the specified significant structures, in particular in the manner of a registration process.

Description

description

Method for operating an endoscope and endoscope This invention relates to a method for operating a

Endoscope according to the preamble of claim 1 and an endoscope according to the preamble of claim 15.

It is a known motivation for a patient to make the surgical procedure as light as possible. Out of this motivation, the so-called minimally invasive surgery has developed, which is characterized by the smallest possible trauma during surgery.

In minimally invasive surgery, such as with laparoscopes, the doctor usually has no immediate insight into the site of the surgery. Nevertheless, to get the best possible spatial orientation, 3D endoscopes can be used. These endoscopes allow the situs to be at least partially detected in 3D, e.g. also for documentation purposes.

A well-known approach is the so-called NOTES (Natural

Orifice Transluminal Endoscopic Surgery), which can be seen as equal ^¬ synonymous with minimally invasive surgery without scars, represents a further development of laparoscopic minimally invasive surgery. This natural body openings are used to avoid visible scars after surgery.

For a physician, there is thus the technical problem of orienting oneself in the patient's body only through laparoscopic insight, reliably detecting spatial conditions and, if possible, excluding unintentional injuries to organs and vessels, for example with the sharp scalpel blades. With NOTES this question is exacerbated because there the access of places is selected, where one later no It does not necessarily mean a short and straightforward access to the site.

3D endoscopy is a very new technology for minimally invasive surgery, which is just beginning to be introduced. Thus, there is currently no solution that is outside of experimental laboratories in everyday medical use. For the introduction and acceptance of 3D endoscopy, ie the broadest possible approach in the medical field but also in particular for an extension of the application range of minimally invasive surgery to other clinical pictures with eg difficult accessibility to the site or with increased risk potential for unwanted injury to organs , Gefä ^¬ SEN or nerves, it is important to deal with the resulting technical challenges.

The object underlying the invention is to provide a solution which improves the known from the prior art solutions, in particular in which it ensures a further Traumare reduction.

This object is achieved by the endoscope method for operating an endoscope according to the preamble of claim 1 by its features and solved by the endoscope according to the preamble of claim 15 by the features thereof.

In the method according to the invention for operating at least one endoscope, a:

a) In situ detection of a measuring range by means Minim ^¬ least an at least two-dimensional image receiving at least a first tissue portion of a patient b) analysis of the image content of each image recording,

c) detecting at least one significant structure per image acquisition based on the analysis, d) determining at least one significant structure continuously detectable in a sequence of sequential image recordings,

e) generating an image by joining to-least parts, in particular as received at least two image ^¬, subsequently to an image larger volume than the volume of the images of the individual image recordings, wherein properties of assembling, in particular location and / or extent and / are turned number of overlaps, partial areas of the image recordings, particularly in the manner of a so-called "stitching", based on at least one of the ^¬ be voted significant structures, in particular in the manner of the so-called "Registration", fixed sets.

By means of the method according to the invention, the state of the art, in which an external localization of the endoscope is carried out to support the guidance of the endoscope, is improved by providing the surgeon or device control with a more comprehensive internal view at the site of the operation and the endoscope so unintentional injuries, for example by avoiding collisions can be prevented.

Further advantages are given by the ange¬ given in the dependent claims ^¬ developments.

The solution according to the invention is characterized, inter alia, by the fact that restrictions in the orientation are solved by the fact that natural markers of the body are found and used by the determined significant structures in order to join the field of view of the endoscope to a larger volume range. By Erfas- measurement and representation of the larger area, the Orientie ^¬ tion is made easier for the physician. According to the invention as a natural marker while Oberflä ^¬ chentexturen of organs, vessels such as blood vessels or lymph vessels or lymph nodes, nerves, tendons, muscle groups or packages, etc. are used..

The invention thus provides an assembly of individual regions, which are detected in 2D and / or 3D, into a larger measurement volume or volume range, ie in particular in the manner of the approach known in particular from stitching as a technique; In contrast, however, differing in particular by the fact that no rigid markers are used.

In order to be able to assemble 2D or 3D images or volume regions, it is provided according to the invention that overlapping parts of the images to be joined, i. the picture shots, on

Basis of the specific structures, so in the case of minimally invasive surgery, the natural markers to make, in order to provide this unique to each other attributable features. The joining of the individual recordings in 2D or 3D via the excellent features or markers takes place in particular according to the procedure known as registration.

With the invention, the peculiarity is that the "measuring object" human can not be regarded as a rigid body, but rather moves, be it by pulsation, which may also be slowed down during the operation, by breathing and breathing excursion or by muscle movements In addition, the situs can also change itself by the operating room itself, in that the surgeon himself moves the organs or changes them, for example through incisions in the topology, so that the invention in particular develops its advantages because it is known from the art method improved assembling allowed, because it is taught a new Registrierungspro- process, which is particularly adapted to the special needs of Medical Resident ^¬ rule application. In this case preferably at least tried with the endoscope a dreidimensiona ^¬ le image recording. Can thereby take place cavities ^¬ clear representation of the situs so that additional in ^¬ formations to assess the necessary and possible

Endoscope guide are available. The phrase "trying" brings it expressed that the endoscope is used fundamentally ^¬ additionally for recording three-dimensional image recording, but due to local conditions such as shadowing can occur that ultimately partly be present as a result, only two-dimensional image recordings.

According to a further development, the determination is performed such that at least a first parameter, insbesonde ^¬ re direction, speed, type of motion detected in each of certain significant structure to a possible full ^¬ continuous detection and description of the movement for each of the detected to obtain natural markers (specific significant structure). Alternatively or in addition at the same time each image receiving a parallel recording of vital signs, Example ^¬ as heartbeat, pulse, blood pressure, breathing state and / or Lun ^¬ genfüllung, the patient are performed, and a determination of at least a given based on the vital parameter characteristic, in particular a correlation Characteristic value per significant structure that reflects the first parameters. As a result, the movements can be related to bodily functions, which among other things is helpful for accurate predictions regarding a collision probability.

If, according to a further development, a selection of the significant structure based on the registration and the stitching based on the registration and stitching on the basis of the at least one first parameter and / or the characteristic characteristic value is selected, the result becomes more accurate. Preferably, the detection range of the endoscope, in particular ^¬ sondere by moving the endoscope is changed to detect other regions of the surgical site to increase, among other things, the volume of the detected range.

If, in the presence of a two-dimensional image recording resulting from a three-dimensional image acquisition inherent in the image, processing of the two-dimensional image recordings on the basis of triangulation methods using successive two-dimensional image recordings is carried out for a three-dimensional image acquisition, the movement of the endoscope may vary due to the fact that according to the invention calculated on ^¬ taken images and subsequently via triangulation methods in a time interval is known aufeinan ^¬ of the following images from a region of the operating site, the three dimensional shape of be, and so the above-mentioned be ^¬ einträchtigung three-dimensional images by shadowing, which have the effect of that not all pixels in the detection area can be detected directly in three dimensions, at least partially removed.

Preferably, the captured image recordings are displayed on a display device, so that the medical personnel can act directly supported thereby.

In this case, the volume regions joined together, in particular according to stitching and registration, are preferably displayed in two dimensions and / or three-dimensionally for display on a display unit.

Particularly suitable for documenting the OP is the development in which a recording of at least parts of the recorded image recordings and / or assembled volumes takes place.

If a recognition of the instruments used in the operation, in particular on shape, color, position, marker, also takes place a representation of their spatial position in the surgical field, in particular in relation to organs, on a display device, is a better differentiation of the objects of a displayed ^¬ th scene and thus a speedy response for a viewer, in particular operator possible.

A further development of the method according to the invention results if, on the basis of the recognition, an evaluation with regard to the presence of a risk of injury takes place and, on the basis of the evaluation, an issuing of warning messages if the presence, in particular by reaching a threshold value, is confirmed.

If, according to a further development, the endoscope is functionally connected and operated with a position sensor, for example an inertial sensor and / or a magnetic field sensor, a movement of the system can be detected and distinguished from a movement of the organs and other body components in the surgical site. If the position sensor is provided by inertial sensors and / or a magnetic field sensor, this is then designed for a detection of translations or rotations in at least one coordinate direction. In a further development of the method according to the invention, partial regions determined to generate the image of at least two endoscopes are used. As a result, the displayed image volume can be significantly increased again.

The endoscope according to the invention has means for carrying out the method according to one of the preceding embodiments and, by implementing the method, makes it possible to realize its advantages.

Further advantages and details of the invention will now be explained in more detail starting from the embodiment of the invention shown in the single figure. It shows the FIGURE is a flowchart with a simplified representation of an embodiment of Ver ^¬ procedure according to the invention

In the figure, the flow of an embodiment of the method according to the invention is greatly simplified, are illustrated in more detail by reference to two of which also features of the endoscope and the possible forms of design from ^¬ / alternative or additional further developments.

There is shown an embodiment of the invention which discloses a novel registration process that is particularly adapted to the particular needs of the medical application.

It takes place in a first step Sl detecting ei ^¬ nes measuring range by means of 2D / 3D image recordings. In a second step S2, determination of natural markers in the 2D / 3D image takes place and, in a third step S3, detection of the movement of the detected natural markers over a sequence of 2D / 3D images. This can be accompanied by one with parallel

Detecting the vital parameters of the patient, such as heartbeat, heart rate, blood pressure at the time of Bildaufnah ^¬ me, breathing state / lung filling, etc. in order to obtain as complete as possible detection and description of the movement for each of the recognized natural marker.

In a fourth step S4 follows a determination of at least one characteristic index for the movement for each of the natural markers, such as amplitude (1D, 2D or 3D if necessary direction), phase position of the movement to pulse or At ^¬ tion, frequency of movement - possibly in relation to pulse or respiratory rate. This is followed in a fifth step S5, a selection of the markers for a registration and a Stitching to be used, for example, due to the at least one measure.

A movement of the endoscope, so that other areas of the surgical site are detected in the detection area by means of the 2D / 3D image acquisition, promotes the process alternatively or additionally.

Due to shading, it may be that not all pixels in the detection area can be detected directly in 3D, but only as 2D. This circumstance should clarify the chosen formulation 2D / 3D in this figure description.

However, such a degradation can be remedied according to the invention if, as shown in the example, the movement of the endoscope between successive images from a region of the surgical site is known. In this case-housed ^¬ nen images is OF INVENTION ^¬-making according to the 3D shape of a lag also subsequently expects loading via triangulation methods are. This is achieved by a sixth step S6, in which a check is made as to whether an image capture is 2D or 3D, and a seventh step S7 in which it is the Triangula ^¬ tion method is applied is illustrated. It is advantageous if the previously sensed movement of the features or areas in Erfas ^¬ sungsbereich be included in this calculation to obtain the highest possible accuracy of the reconstructed 3D map. The above and also the following steps can be performed as further embodiments of the invention, partially overlapping or parallel so that the doctor can processing ^¬ th undisturbed and does not have to take on the operation of the 3D Endskops consideration.

In an eighth step S8, stitching of the at least two 3D image areas now present on the basis of the sixth and seventh steps S6 and S7 is then carried out, so that a larger amount of information is added. coherent area arises. An embodiment variant ^without the sixth and seventh steps S6 and S7 is also conceivable, in such a case there is a representation / processing of what can be detected, ie a 2D / 3D representation as described above.

There is otherwise in a ninth step S9 one depicting ^¬ len of, due to the seventh and eighth step S7 and S8 now present 3D images for the medical staff and represent the combined added volume regions in 3D representation.

In addition, a recording of the recorded images and / or the assembled volumes for documentation of the OP can take place.

In addition, recognizing the instruments used in the OP on shape, color, location, markers may be provided and a representation of their spatial position in the surgical field and in particular in relation to organs done, possibly issuing alerts when a risk of injury of Organs, vessels, nerves, tendons, etc. exists.

In the design of the system, it may be advantageous if the endoscope has a position sensor system to detect a movement of the system and to distinguish it from a movement of the organs and other body parts in the surgical site.

As position sensors, for example, an inertial sensor and / or a magnetic field sensor can be used, which are designed for a detection of translations or rotations in at least one coordinate direction.

A market could accelerate any, when the invention is developed such that methods are used for the instru ^¬ tenerkennung and collision warning, as in the so-called "3D Mine" project

(http: // www. miti. med. tum. de / index. php? id = 120 & L = 0)). gen and / or known presentation optimization of the 3D Info be applied.

Special advantages can be created by combining the mentioned with very compact chip-on-tip endoscopes.

The invention is not limited to the illustrated example of the method, or described (arrangement) variants ^¬ limits. Rather, it includes all combinations and excluded staltungsvarianten which are given by the claims and thereby have one or more of the following features / advantages ^¬:

Using proprietary markers to bring together individual SD images into larger areas.

Capture the movement of the natural markers and determine the markers' quality for subsequent stitching.

Possibility of use of the invention for both rigid as well as flexible endoscopes, which allows to operate even in hard to reach areas with 3D endoscopes.

Detecting movement depending on Vitalparame ^¬ tern of the patient.

Stitching with markers that are not rigid or stationary on the object, since the organs are not rigid bodies. So a stitching at local uncertainty of the registration points for stitching.

Claims

claims

1. A method for operating at least an endoscope ge ^¬ characterized by the following steps:

a) In situ detection of a measuring range by means of Minim ^¬ least an at least two-dimensional image pickup Minim ^¬ least a first tissue portion of a patient

b) analysis of the image content of each individual image acquisition,

c) detecting at least one significant structure per image acquisition based on the analysis,

d) determining at least one significant structure continuously detectable in a sequence of sequential image recordings,

e) generating an image by combining at least parts, in particular at least two image recordings, of the sequence into an image of larger volume than the volume of the images of the individual image recordings, wherein properties of the joining, in particular location and / or extent and / or number of Overlaps, of sub-areas of the image recordings, in particular in the manner of a so-called "stitching", based on at least one of the specific significant

Structures, in particular by the nature of the so-called "registration".

2. Method according to the preceding claim, characterized in that the detection takes place in such a way that a three-dimensional image recording is at least attempted.

3. The method according to one of the two preceding claims, characterized in that the determination is carried out such that at least one first parameter, in ^¬ particular direction, speed, type, a movement per specific significant structure is detected.

4. Method according to the preceding claim, characterized in that a) a simultaneous recording of vital parameters, for example heartbeat, pulse, blood pressure, breathing state and / or lung filling, of the patient is carried out simultaneously with each image acquisition,

b) determination of at least one characteristic value given on the basis of the Vi ^¬ talparameter, in particular a correlation to the first parameters reproducible characteristic value per significant structure.

5. The method according to the preceding claim, characterized in that a selection of the joining and fixing, in particular in the manner of the registration and the stitching, based significant structure due to the at least one first parameter and / or the characteristic value is carried out.

6. The method according to any one of the preceding claims, characterized in that the detection range of Endo ^¬ skops, in particular by moving the endoscope, for Erfas ^¬ sung other areas of the surgical site is changed.

7. A method according to any one of claims 3 to 6, characterized in that in the presence of an, in particular ^¬ fundamentally a stationary inherent degradation of the three-dimensional image recording, the resulting two-dimensional image recording, a treatment of the two-dimensional image recordings on the basis of consecutive two-dimensional image recordings used triangulation to a Three-dimensional image acquisition is performed.

8. The method according to any one of the preceding claims, characterized in that the captured image recording are brought to egg ^¬ nem display device for display.

9. The method according to one of the preceding claims, characterized in that, in particular according to stitching and

Registration, merged volume areas two-dimensional nal and / or three dimensions for display on a Anzeigege ^¬ advises be brought.

10. The method according to any one of the preceding claims, that a recording of at least parts of the captured image recordings and / or assembled volumes takes place.

11. The method according to any one of the preceding claims, characterized in that

a) recognition of the instruments used in the operation, in particular via shape, color, position, markers, b) a representation of their spatial position in the surgical field, in particular in relation to organs, on a display device.

12. The method according to the preceding claim, characterized in that

a) on the basis of the recognition, an evaluation is carried out with regard to the existence of a risk of injury and b) on the basis of the evaluation, an issuing of warning messages, if the existence, in particular by reaching a threshold value, is confirmed.

13. The method according to any one of the preceding claims, characterized in that the endoscope is functionally connected and operated with a position sensor, for example, an inertial sensor and / or a magnetic field sensor.

14. A method according to any one of the preceding claims, characterized in that partial regions determined to generate the image from at least two ^¬ endoscopes are used.

15. Endoscope, characterized by means for carrying out the method according to one of the preceding claims.