DE102012211396A1 - Endoscopic device for taking picture within blood vessel cavity of human, calculates distance between image pickup unit, striking landmark and distinctive landmark based on performed movement of image recording device - Google Patents

Abstract

The device (10) has a image recording device (20) which is connected with a image analyzing device (40) for recording position of the image pickup unit. The cavity (30) points the image along the optical axis of the imaging device. The displaced position of the image pickup unit is captured based on the presence of distinctive landmark relative to the optical axis. The distance between the image pickup unit, striking landmark and distinctive landmark is calculated based on the performed movement of the image recording device.

Description

The invention relates to an endoscopy device with an image recording device for recording an image within a cavity.

Minimally invasive surgery has become very widespread in recent decades and has become the standard procedure in many areas. Despite many advantages, the method also has some disadvantages, the limited depth perception is perceived by many surgeons as particularly critical. Although there are various approaches to solve this problem, such as directed shadow casting or the use of two optical channels for generating stereoscopic images. However, in particular stereoscopic techniques are associated with increased costs for the endoscope because of increased production costs. Furthermore, stereoscopic images in high quality are currently only to be considered with aids such as polarization or shutter glasses, which are perceived by the doctors as impractical.

In many cases, a full three-dimensional view is not necessary, but it is sufficient to show the surgeon the distance between the endoscope tip and tissue in the usual two-dimensional view to improve the orientation and thus significantly reduce the risk of injury to the patient.

The distance measurement should be done without contact in order not to damage the tissue and to keep the entire working space for the instruments free. The integration of typical non-contact distance meters such as ultrasonic or triangulation sensors is difficult due to the very tight space. In the future, the space will be even more scarce, as doctors prefer ever thinner endoscopes, which make it possible to operate difficult to reach or tiny regions. Furthermore, the filigree endoscopes require minor incisions during standard procedures with the result of a cosmetically almost invisible intervention. In laparoscopy, therefore, five-millimeter endoscopes are being increasingly used instead of the ten-millimeter endoscopes used to date.

Even in technical endoscopy, the knowledge of the object distance offers clear advantages. On the one hand, the navigation can be simplified for the operator, on the other hand, the size of objects can be better estimated if their distance is known.

The invention has for its object to provide an endoscopy device that allows better orientation inside a cavity than previous endoscopy devices.

This object is achieved by an endoscopy device with the features according to claim 1. Advantageous embodiments of the endoscopy device according to the invention are specified in subclaims.

According to the invention, it is provided according to the invention that an image evaluation device is connected to the image recording device and is suitable for transmitting a first image recorded at a first position of the image recording device and a second image shifted at a second position relative to the first position along the optical axis of the image recording device Position of the image pickup device taken second image on the presence of at least one distinctive landmark in both images towards and to detect a shift of the distinctive landmark relative to the optical axis or relative to another prominent landmark in the at least two images quantitatively and taking into account the shift of the Imaging device to calculate the distance between the image pickup device and the striking landmark or the distance between the one and the other distinctive landmarks. The term "landmarks" are characteristic image content, eg. As blood vessels or organ boundaries in the case of medical endoscopy to understand.

A significant advantage of the endoscopy device according to the invention is that it allows a non-contact distance measurement and thereby manages without additional components on the endoscope tip. To determine a distance, be it the distance between the image recording device and a striking landmark or the distance between two prominent landmarks, an endoscopic image is first recorded and examined by an image recognition algorithm for characteristic image content. Once at least one landmark has been found, then the endoscope can be moved by a defined path along the optical axis and another endoscopic image can be recorded. In this further endoscopic image, the landmarks identified in the first image are searched for and located. Subsequently, the distance between the landmark and the optical axis can be determined. alternative or in addition, the distance between landmarks can be determined when two or more landmarks have been identified in the two endoscopic images. The invention makes use of the fact that due to the changed between the two endoscope images object distance, a different magnification, so that the distances of the landmarks to the optical axis or between the same landmarks must be different in both images, the landmark or visualized by the landmark Object as such remains unchanged. The distance between the endoscope and the object and / or between objects determined in this way can be displayed to the user of the endoscopy device, that is, for example, to a surgeon, for better orientation within the cavity and for carrying out his surgical procedures.

For use in the medical field, it is considered advantageous if the image evaluation device is suitable for using a blood vessel or an organ border in a human or animal body cavity as a prominent landmark for distance measurement.

With regard to the configuration of the image evaluation device, it is considered advantageous if it comprises a computer device which is programmed in such a way that it can recognize distinctive landmarks in images and recognize a prominent landmark detected in an image in another image within the context of a pattern recognition process.

wobei a₂ den Abstand zwischen der markanten Landmarke und der Bildaufnahmeeinrichtung nach der Verschiebung, Δa die Verschiebung der Bildaufnahmeeinrichtung entlang der optischen Achse, y'₁ die Bildgröße vor der Verschiebung und y'₂ die Bildgröße nach der Verschiebung bezeichnen.Particularly preferably, the computer device is programmed in such a way that it can determine the distance between the distinctive landmark and the image recording device in accordance with:

where a ₂ denotes the distance between the prominent landmark and the image pickup device after the shift, Δa the displacement of the image pickup device along the optical axis, y ' ₁ the image size before the shift, and y' ₂ the image size after the shift.

In order to ensure that features recognized in the images actually refer to the same landmarks and indeed the same landmarks have been identified in the images, it is considered advantageous if a plausibility check is possible. According to a preferred embodiment, it is provided that the image evaluation device in each case examines connecting lines determined between at least three distinctive landmarks for an angular deviation or a deviation of the aspect ratios in the two images and exclusively in the case that the angular deviation and / or the extension deviation is a predetermined dimension falls short of or at least does not exceed the striking landmarks for distance and / or sizing.

In order to enable a defined displacement of the image recording device, it is considered advantageous if the endoscopy device has a displacement device which makes it possible to displace the image recording device along the optical axis.

The displacement device preferably comprises a hydraulic, pneumatic, electromagnetic or piezoelectric linear drive or a rotary drive with spindle.

With a view to a reproducible shifting of the image recording device, it is considered advantageous if the endoscopy device has at least one marking, at least one mechanical end stop, at least one limit switch or at least one displacement measuring system with which or which measure a displacement of the image recording device along the optical axis or limited to a predetermined level.

Moreover, it is considered advantageous if the image evaluation device is suitable for creating with the measured distances and / or object sizes a three-dimensional model of the interior of the cavity or of an object located in the interior of the cavity.

The invention also relates to an endoscopy method in which an image of a cavity is taken with an image recording device. According to the invention, it is provided that a first image which is recorded at a first position of the image recording device and which shows the cavity is displaced when the first image is moved along the optical axis of the image recording device Second position of the image pickup device recorded second image is evaluated for the presence of at least one distinctive landmark in both images out, a shift of the distinctive landmark relative to the optical axis or relative to another landmark in the at least two images is detected quantitatively and taking into account the shift the image recording device, the distance between the image pickup device and the distinctive landmark or the distance between the one and the other landmark is calculated.

Regarding the advantages of the method according to the invention, reference is made to the above statements in connection with the endoscopy device according to the invention, since the advantages of the endoscopy device according to the invention essentially correspond to those of the method according to the invention.

The invention will be explained in more detail with reference to embodiments; thereby show by way of example

1 An exemplary embodiment of an endoscopy device according to the invention, by means of which the method according to the invention is also explained by way of example,

2 an example of a first picture taken before the determination of landmarks,

3 in the first picture according to 2 recorded landmarks,

4 by way of example a second picture taken before the determination of landmarks,

5 in the second picture according to 4 recorded landmarks,

6 the distance determination as an example in the case that the distance of a landmark is evaluated to the optical axis for the distance determination,

7 the distance determination by way of example based on landmark distances among each other,

8th a plausibility check based on the connecting lines of landmarks and

9 A second embodiment of an endoscopy device.

For the sake of clarity, the same reference numbers are always used in the figures for identical or comparable components.

In the 1 one sees an embodiment for an endoscopy device 10 , This includes an image pickup device 20 for capturing an image within a cavity 30 For example, within a human or animal body 31 , To the image recording device 20 includes, inter alia, an optical system, for example in the form of a rod lens system (rod lens arrangement) 21 that with one end 21a in the cavity 30 is introduced and deals with its other end 21b outside the cavity 30 located. With the rod lens system 21 there is a camera 22 related to that via the rod lens system 21 rendered image from the inside of the cavity 30 receives. The rod lens system 21 is in an endoscope 11 the endoscopy device 10 That's about a trocar 12 in the body 31 or the cavity 30 is introduced.

With the camera 22 the image pickup device 20 is over a cable 23 an image evaluation device 40 in connection. The image evaluation device 40 preferably comprises a computer device 41 programmed to be at a first position of the image pickup device 20 recorded first image and at a second, opposite the first position along the optical axis of the image pickup device 20 shifted position recorded second image on the presence of at least one distinctive landmark in both images out.

The 2 shows by way of example a first recorded image before the determination of the landmarks. In the 3 are those of the image evaluation device 40 registered landmarks marked with the reference numerals L1, L2, L3 and L4.

The 4 shows by way of example the second recorded image before the determination of the landmarks. In the 5 are those of the image evaluation device 40 registered landmarks also with the Reference numerals L1, L2 and L3 are marked as they coincide with the landmarks L1, L2 and L3 in FIG 3 are identical. The landmark L4 is due to the reduced image section in 5 not visible.

The image evaluation device 40 then determines a shift of the landmarks relative to the optical axis or relative to the other prominent landmarks in the at least two images quantitatively and calculated taking into account the displacement of the image pickup device 20 the distance between the image pickup device 20 and the distinctive landmarks or the distance between the distinctive landmarks.

In the following, the determination of the distance is explained in greater detail by way of example. The 6 shows the example of the case that the distance of the landmarks to the optical axis O is used for the distance determination. The 7 exemplifies the case of the distance measurement based on the landmark distances among themselves. The distance of the landmarks to the optical axis O or two landmarks to each other is in the 6 and 7 each referred to as object size y.

The upper illustration in the 6 and 7 shows in each case the image y ' _{1 of} the object size y before the displacement of the image pickup device 20 and the lower diagram shows the image y ' _{2 of} the object size y after the displacement of the image recording device 20 , The construction beams for determining the image B are identified by the reference symbol K. One for the image recording device 20 belonging lens bears the reference numeral 25 , and the main plane of the lens 25 is marked with the reference H.

In the first position before the displacement of the image pickup device (hereafter referred to as position 1) applies to both 6 and 7 :

In this case, a ₁ denotes the object distance (distance between the image recording device 20 and the object), a ' ₁ the image size, y the object size and y' ₁ the image size.

Accordingly, after a shift of the image pickup device 20 for the second position (hereafter referred to as position 2):

In this case, a ₂ denotes the object width, a ' ₂ the image width and y' ₂ the image size, in each case after the displacement.

The endoscope displacement Δa is Δa = a ₁ - a ₂ (3)

For a fixed focus optics applies a ' ₁ = a' ₂ (4)

Formulas (1) and (2) can be switched to a ' ₁ or a' ₂ and then equated using (4). Substituting context (3) for a ₂ yields y ' ₁ a ₁ = y' ₂ (a ₁ -Δa) (5)

Transforming (5) returns the object distance in position 1:

The same applies to the object distance in position 2:

If fixed-focus optics are used, the object distance should be in the range of depth of field both before and after the shift. When using optics with variable focus, they should be refocused if necessary. This can lead to a small change in the focal length and thus the magnification. Depending on the accuracy to be achieved, this can be excluded when calculating the distance.

The endoscope displacement can be done manually or automatically, with each setting either a shift by a predetermined distance or the shift is freely selectable.

If a manual shift takes place by a predetermined distance, this can be achieved by mechanical stops or optical markings on the endoscopy device 10 , the trocar 12 or an additional component can be facilitated.

To measure a freely selectable displacement, a length scale readable by the surgeon in combination with manual value input or a suitable displacement measuring system with direct coupling to the image evaluation device 40 can be used, all types of path measuring systems are available, for example, resistive, inductive, optical or magnetic incremental method.

For automated endoscope displacement arbitrary actuators, for. B. linear direct drives, pneumatic actuators or electric motors in combination with a spindle can be used.

To achieve a fixed displacement path mechanical end stops or any form of limit switches or Wegmesssystemen can be used. For detecting an automatically driven freely selectable route, in turn, any suitable position measuring system can be used, in particular rotary encoder in the motor with known spindle pitch.

The path measuring system and possibly the drive can be part of the trocar 12 , the endoscope 10 or be executed as an independent module.

The landmark finding and retrieval algorithm should preferably be designed to provide more than just the at least required one or two landmarks so that the distance measurement function will be ensured even if some of the landmarks in the second image are not recognize. This can be the case, for example, if a landmark is located at the edge of the image and the image section is reduced by moving the endoscope in the direction of the object or the landmark lies in a sub-overexposed or overexposed region.

Furthermore, especially with manual displacement of the endoscope, the algorithm should preferably be robust against slight swiveling and tilting of the endoscope in order to ensure correct operation even if the displacement does not take place exclusively along the optical axis.

The correct assignment of the landmarks in both images can also be checked by an algorithm in which the connecting lines of the individual landmarks with the optical axis or the landmarks are analyzed with each other; this is exemplified by the 8th , You see in the 8th left three landmarks L and the connecting lines V that connect the landmarks L. In the middle of 8th one recognizes the same landmarks L as on the left in the case of a displacement of the image recording device without tilting, and in the 8th on the right you can see the same landmarks L in case of a tilt around the optical axis.

With purely axial displacement of the endoscopy device 10 all distances scale with the same factor and the alignment of the connecting lines V remains in both pictures (cf. 8th , Illustrations left and center).

With an additional tilt of the endoscopy device 10 around their longitudinal axis, all connecting lines V are rotated by the same angle, so that the angles between the connecting lines V remain constant (cf. 8th , Illustrations left and right). A slight swivel of the endoscopy device 10 results in a shift of all points in the image in the same direction and, if these have the same object width, by the same amount, so that here too the angles between the connecting lines are maintained with each other.

Preferably, connecting lines V determined between at least three prominent landmarks L in the images are examined for an angular deviation or a deviation of the aspect ratios in the two images and exclusively in the case that the angular deviation and / or the extension deviation falls below a predetermined level or at least does not exceed the landmarks used for distance and / or sizing.

If only one landmark and its distance to the optical axis or two landmarks and their distance from one another are used for the distance determination, only the object distance in the plane of the landmark or landmarks can be specified. However, since the surfaces of many organs are curved, it may be useful to use a variety of landmarks and specify separately the different depth planes calculated from them. It is also possible not only to display the depth levels, but to create a topographic map or a three-dimensional surface reconstruction. If necessary, the two-dimensional endoscope image can be laid on it as a texture.

Furthermore, with a known object distance from the image size, the object size according to the formula y '= y · a' / a be calculated. This makes it possible to measure the size of structures or to determine distances between objects. This information can be superimposed on the endoscope image or fed to further information processing. The selection of the objects to be measured can be done manually or automatically by image processing algorithms.

Instead of the rod lens system 21 It is also possible to use other types of optical systems, for example optical fibers or optical fiber bundles. A rod lens system, like the rod lens system 21 according to 1 is preferably used in rigid endoscopy devices; In the case of flexible endoscopy devices, an optical fiber bundle is preferably used as the optical system.

The 9 shows a second embodiment of an endoscopy device 10 , In this embodiment, the image pickup device is 20 through a camera 22 is formed at the end of the endoscope shaft 11 attached and thus located within the cavity 30 , The camera 22 is about an example electrical or optical signal line 26 with the image evaluation device 40 in connection. Otherwise, the explanations in connection with the 1 - 8th here accordingly.

The endoscopy device 10 and the corresponding endoscopy method have been described above using the example of medical endoscopy; the explanations apply to the use in the field of technical endoscopy, in which slidable cameras are used, accordingly.

LIST OF REFERENCE NUMBERS

10

 endoscopy equipment

11

 endoscope shaft

12

 trocar

20

 Image recording device

21

 Rod lens system

21a

 The End

21b

 The End

22

 camera

23

 electric wire

25

 lens

26

 signal line

30

 cavity

31

 body

40

 image analyzing device

41

 Computer device

a ₁

 object distance

a ' ₁

 image distance

a ₂

 object distance

a ' ₂

 image distance

.DELTA.a

 endoscope shift

B

 image

H

 main level

K

 construction beam

L

 landmark

L1

 landmark

L2

 landmark

L3

 landmark

L4

 landmark

O

 optical axis

y

 object size

y ' ₁

 image size

y ' ₂

 image size

V

 connecting line

Claims (10)

Endoscopy device ( 10 ) with an image recording device ( 20 ) for taking an image within a cavity ( 30 ), characterized in that with the image recording device ( 20 ) an image evaluation device ( 40 ), which is suitable for a first position of the image recording device ( 20 ), the cavity ( 30 ) and a at a second, opposite the first position along the optical axis (O) of the image pickup device ( 20 ) shifted position of the image pickup device ( 20 ) to evaluate the presence of at least one distinctive landmark (L, L1, L2, L3) in both images and a shift of the landmark (L, L1, L2, L3) relative to the optical axis (O) or relative to of another prominent landmark (L, L1, L2, L3) in the at least two images and, taking into account the displacement of the image recording device ( 20 ) the distance between the image recording device ( 20 ) and the distinctive landmark (L, L1, L2, L3) or the distance between one landmark and the other distinctive landmark. Endoscopy device ( 10 ) according to claim 1, characterized in that the image evaluation device ( 40 ), a blood vessel or an organ border in a human or animal body cavity ( 30 ) as a prominent landmark (L, L1, L2, L3) for distance measurement. Endoscopy device ( 10 ) according to one of the preceding claims, characterized in that the image evaluation device ( 40 ) a computer device ( 41 ) programmed to recognize landmarks (L, L1, L2, L3, L4) in images as part of a pattern recognition process, and a prominent landmark (L, L1, L2, L3, L4) detected in an image Recognize another picture. Endoscopy device ( 10 ) according to claim 3, characterized in that the computer device ( 41 ) is programmed such that it determines the distance between the prominent landmark (L, L1, L2, L3) and the image recording device ( 20 ) can determine according to:

where a _{2 is} the distance between the prominent landmark (L, L1, L2, L3) and the image recording device ( 20 ) after the displacement, Δa the displacement of the image recording device ( 20 ) along the optical axis (O), y ' ₁ denote the image size before the displacement and y' ₂ the image size after the displacement. Endoscopy device ( 10 ) according to one of the preceding claims, characterized in that the image evaluation device ( 40 ) investigated in each case between at least three landmarks marked (L) connecting lines (V) on an angular deviation or a deviation of the aspect ratios in the two images and exclusively in the case that the angular deviation and / or the extension deviation falls below a predetermined level or at least does not exceed the landmarks (L) for distance and / or sizing. Endoscopy device ( 10 ) according to one of the preceding claims, characterized in that the endoscopy device ( 10 ) has a displacement device, which is a displacement of the image recording device ( 20 ) along the optical axis (O). Endoscopy device ( 10 ) according to claim 6, characterized in that the displacement device comprises a hydraulic, pneumatic, electromagnetic or piezoelectric linear drive or a rotary drive with spindle. Endoscopy device ( 10 ) according to one of the preceding claims, characterized in that the endoscopy device ( 10 ) at least one marking, at least one mechanical end stop, at least one limit switch or at least one displacement measuring system, with or a displacement of the image pickup device ( 20 ) along the optical axis (O) or limit it to a predetermined amount. Endoscopy device ( 10 ) according to one of the preceding claims, characterized in that the image evaluation device ( 40 ) is suitable, with the measured distances and / or object sizes, a three-dimensional model of the interior of the cavity ( 30 ) or one inside the cavity ( 30 ). Endoscopy method in which with an image recording device ( 20 ) an image of a cavity ( 30 ), characterized in that - a at a first position of the image pickup device ( 20 ), the cavity ( 30 ) and at a position opposite to the first position along the optical axis (O) of the image recording device (FIG. 20 ) shifted second position of the image pickup device ( 20 ) is evaluated on the presence of at least one distinctive landmark (L, L1, L2, L3) in both images, - a shift of the landmark (L, L1, L2, L3) relative to the optical axis (O) or relative to another landmark (L, L1, L2, L3) is quantitatively detected in the at least two images and - taking into account the displacement of the image recording device ( 20 ) the distance between the image recording device ( 20 ) and the landmark (L, L1, L2, L3) or the distance between the one and the other landmarks (L, L1, L2, L3).

Images