WO2010124672A1

WO2010124672A1 - Video-based mono-camera navigation system

Info

Publication number: WO2010124672A1
Application number: PCT/DE2010/000440
Authority: WO
Inventors: Robert Haase; Hendrik Möckel; Ronny Grunert
Original assignee: Phacon Gmbh
Priority date: 2009-04-27
Filing date: 2010-04-20
Publication date: 2010-11-04
Also published as: DE102009019019A1; DE112010001790A5

Abstract

The invention relates to a method for producing a video-based mono-camera navigation system. The system is able to determine the position information of an object utilizing a single camera and markers provided with different geometries and colors. The system includes a video camera (5), an electronic data processing unit (8), a first marker (6), which has a specific pattern and a specific color and is located in a known fixed position with respect to a stationary or mobile object (10), and a second marker (7), which is provided with a different specific pattern and a different specific color and connected to an object (2) the position of which can be moved and the respectively assumed position of which is to be detected in a measuring range. Compared to known solutions, said system has the advantage that only one video camera (5) is required, and that the technical demands placed on said camera (5) are only such which can be fulfilled by nearly any video camera that is available for purchase.

Description

Video-based mono camera navigation system

description

The invention relates to a video-based camera navigation system, comprising a device and a method which allows the detection of objects provided with markers in space and their location with only one video camera.

It is known to determine the positional relationship of objects to each other with optical navigation systems. For example, in the fields of warehouse technology, in transportation, in metalworking and surgery. In surgery, these systems are used in the operating room to determine the position of instruments relative to the patient and thus to improve the chances of success of an operation. In "Khan M, corner U, man WJ .:" The application of an optical navigation system in endonasal sinus surgery ", HNO, 2003, rod T, Schultz-Coulon, Hans-Jürgen:" Clinical experience with an optical navigation system in Clinical Routine Operation ", 77th Annual Meeting of the German Society of Oto-Rhino Laryngology, Head and Neck Surgery, 2006; Nobuhiko Sugano:" Computer Assisted Navigation in Hip Arthroplasty ", Springer Berlin, 2001" describes optical stereo camera navigation systems. At least two or more cameras circularly arranged diodes emit infrared light. This is reflected by Markem, who are on patient and surgical instruments. The reflected infrared light is detected by at least two cameras. Software-supported triangulation determines the spatial coordinates of the markers in the room. The coordinate systems of the camera, the patient (CT sectional images) and the surgical instrument are transferred by registration into a uniform coordinate system. This makes it possible to display the surgical instrument or a so-called pointer in the CT slice images. The surgeon is thus able to determine the current position of the instrument on or in the patient in the CT slice images on a monitor. Based on the graphical representation of the instrument in the CT slice images, the surgeon recognizes the distance to the target area and the risk structures to be protected (eg nerves or vessels).

US 2004/0002642 A1 discloses an optical stereo camera navigation system which operates in the visible wavelength range. The localization is realized here by the recognition of black and white contrast markers with two cameras.

The disadvantages of these known solutions are that at least two cameras are needed in a defined distance from each other to capture objects in the room. It is problematic to accommodate the associated large housing or equipment cart in an operating room. Another aspect is the high cost due to the use of special stereo cameras.

The invention has for its object to develop a video-based camera navigation system that can be implemented with the use of a single video camera and no special requirements are placed on such a video camera. This makes it possible to use a commercially available video camera.

This object is achieved by an optical measuring system which consists of a device according to claim 1 and a method according to claim 5. Belong to the device

• a video camera,

• an electronic data processing system, • a first marker provided with a specific pattern and color, which is in a known fixed position with respect to a stationary or mobile object, and

• a second marker, which is provided with a different pattern and a different color, and which is connected to a variable object whose position is to be recorded in a measuring area.

The two markers thus together form a unit for detecting the spatial position and rotation in virtual reality environments of a first, fixed or movable object relative to a second, fixed or movable object and are connected as well as the video camera to the electronic data processing system.

No special requirements are placed on the video camera for monitoring the measuring range. For this purpose, a commercially available video camera can be used, which is connected via a USB or other data interface with the electronic data processing system.

On the objects, such as persons, instruments or tools whose location parameters are to be determined, a characteristic feature (marker) is attached. The marker is provided with various markings or geometries, which can be recognized by the fully automatic image processing by the video camera and assigned by the color-shape coding. The size, color and shape of the marker is arbitrary. The base color should differ only with the colored markings applied to it. The color tones of the markers can theoretically be chosen arbitrarily. For optimal image processing, however, strong color contrasts are preferable. For the position calculation, characteristic points of the geometry such as corner points, centers or centers of gravity are used. The shape, the color and the distance or the arrangement of the geometries on the respective marker is known to the mono-camera navigation system. - A -

A specially developed image processing software picks up the recorded images of the video camera and detects fully automatically the colored markings. By subsequently assigning the characteristic heavy or corner points of the markers in the recorded camera image to the geometries attached to the marker taking into account the camera properties, the position calculation of the object takes place in space. The image processing software does not require special hardware. It can be installed and run on a standard PC.

The navigation system operates according to the following method steps: a) placing a first marker in a fixed position on the first fixed or movable object, b) attaching a second marker on the second fixed or movable object, c) aligning the video camera on both markers, taking into account their possible Movement range, d) input of the position of the fixed target area, e) image acquisition via the video camera, f) image preprocessing, g) object recognition, h) assignment of the objects to the known geometries - classification, i) color determination of the geometries of the markings, j) assignment of the recorded characteristic pixels (for example, corner points) of a geometry to the markings attached to the real object, k) calculating the transformation T over the associated points of the markers and the camera parameters (focal length, image distortion),

I) displaying the image on the screen, m) capturing the new positions of the two markers by the video camera, n) sending the new camera parameters to the electronic data processing equipment, o) repeating steps e) to n). Due to the use of a commercially available video camera, the simple structure of the markers and the use of a standard PC, the user is offered a very economical navigation solution. The cost-effective hardware makes it possible to establish the technology in new market fields and to make them available to a larger number of potential customers.

Another advantage to previous optical navigation systems is that instead of two or more, only one camera is used for position determination. This novel implementation solves various problems:

- The structure of the system is kept very small. No need to place cameras in the room, which limits the flexibility and usability of the system.

- Any camera system can be used. Thus, an adaptation to the measurement requirement, state of the art camera technology and economic systems is possible at any time without major expenses.

- The visibility of the markers (line of sight problem) is better ensured, since the video camera, in contrast to conventional systems is not two to three meters from the area to be measured. The preferred example of a medical insert is either attached to the operating table or directly to the patient or surgical instrument.

- The camera position can be adapted more flexibly to the dimensions of the area to be measured. In a small area, the single video camera can be easily moved closer to the area to be surveyed. If a larger area needs to be monitored, the camera will be set up at a greater distance. The resolution of the video camera and the correlating measuring accuracy can thus be optimally adapted to the respective measurement situation. Higher measurement accuracy is smaller and coarser Accuracy achieved in larger areas.

embodiments

The invention will be explained in more detail below with reference to three exemplary embodiments and associated drawings, all of which relate to the use of the mono-camera navigation system in the operation of a patient. Show it:

1 shows the structure of a mono-camera navigation system in a schematic representation,

Fig. 2 possibilities for the construction of the marker in a schematic representation and

3 shows the sequence of the method steps for the image processing and tracking process.

The monaural camera navigation system illustrated in FIG. 1 is placed in an operating room for assistance and monitoring during an operation for the operating surgeon in such a way that on the screen 1, the position of his surgical instrument 2 to the location or target area 3 of the surgical procedure with his healthy and diseased tissues as well as his risk structures can see exactly. The screen 1 belongs together with the processor 4 and the keyboard and computer mouse not shown to the electronic data processing system 8. For the optical detection of the respective situation, a video camera 5 is provided, which is also connected to the electronic data processing system 8. Essential to the invention is the use of two markers 6, 7, which together form a unit for detecting the spatial position and rotation in virtual reality environments of a first, spatially movable object with respect to a second, stationary object. The one marker 6 is mounted in a fixed position on the patient 10, while the second Marker 7 is fixed in a specific position on the surgical instrument 2. Both markers 6, 7 are provided according to FIG. 2 with markings 9, which according to FIG. 2 have a specific color and a certain shape and are arranged in different fixed positions. The letters entered in the outlines are to be given an example of certain colors such as g for yellow, R for red, O for orange and so on. Due to the different colors and contours as well as the different arrangement of the markings, the respective position of the two markers 6, 7 relative to one another and the position and position of the surgical instrument 2 to the location or target area 3 of the surgical procedure can be detected with only one video camera 5 and on the Screen 2 are displayed.

1st embodiment

This first embodiment involves the use of the camera navigation system in the operation of a patient, wherein the operative target area occupies the same position and position on the operating table during the entire surgical procedure. It is of great importance to reach the operative target area while at the same time sparing the risk structures such as nerves or vessels. The assistance of the surgeon in the operation by the mono-camera navigation system consists in the optical navigation. If necessary, the surgeon can see exactly where he is with his surgical instrument 2 on screen 1 at any time. When the marker 7 attached to the instrument 2 moves with respect to the marker 6 fixedly attached to the patient 10, this is recognized by the video camera 5 as a result of the different patterns and colors and displayed on the screen 1, so that the surgeon can use the Position of his instrument 2 to the operative target area 3 can recognize. In this case, the coordinate systems of the patient (CT / MRI slice images), video camera 5 and instrument 2 are converted into a coordinate system by a point-based registration. While the surgeon leads the instrument 2, he can simultaneously recognize the actual position of the guided by him instrument 2 on a screen 1 in the CT Schnittbildem.

The procedure for using the video-based mono-camera navigation system in the operation is as follows:

1. attaching a first marker 6 is in a fixed position on the patient 10,

2. attaching a second marker 7 to the surgeon's instrument 2,

3. input the position of the operative target area 3,

4. input the position of the point of application of the surgical instrument 2,

5. image acquisition via the video camera 5,

6. image preprocessing,

7. object recognition,

8. Assignment of the objects to the known geometries - classification,

9. color determination of the geometries of the markings,

10. Assignment of the recorded characteristic pixels (for example corner points) of a geometry to the markings attached to the real object,

11. Calculate the transformation T over the assigned points of the markers and the camera parameters (focal length, image distortion),

12. Display the image on the screen 1,

13. Detection of the new positions of the two markers 6, 7 by the video camera 5,

14. sending the new camera parameters to the electronic data processing system 8,

15. Repeat steps 5 through 14.

2nd embodiment

In operations on the brain, its position is changed after cranial opening (brain shift). Since the MRI data was recorded preoperatively, the actual position of the brain tumor as operative target area 3 can no longer be determined exactly. A 2D ultrasound head scans the patient's brain intraoperatively. The described invention is combined with the 2D ultrasound system. Both the patient 10 and the 2D ultrasound head are markers 6, 7 different pattern and color. Point-based registration enables the patient coordinate systems (MRI systems) to

Sectional images), video camera 5 and 2D ultrasound head are converted into a coordinate system.

The location of the 2D ultrasound head is captured at the time the 2D ultrasound image is taken. It is thus possible to determine a 3D ultrasound image from the 2D. The intraoperative 3D ultrasound image is superimposed with the preoperatively acquired MRI slice images. Thus, the surgeon receives the information about the actual actual position of the brain tumor.

3. Application example

The mono camera system can be used as a direct 3D input device for various program and operating system surfaces. For this, the position information of the tracker is converted into actions and positions of the computer mouse. The marker 6 can be attached to the hand on the head or to various objects, in order, e.g. to take control of the computer mouse. Consequently, in 3D scenes or in 3D games, it is also possible to control the video camera 5 with, for example, the rotation or movement of the head. The direct link between real movement of the patient's head 10 and changing the virtual camera position gives the user a vivid and realistic impression of the 3D scene.

Claims

claims

1. Device for a video-based camera navigation system consisting of

A video camera (5),

An electronic data processing system (8) consisting of a keyboard, a computer mouse, a processor (4) and a screen (1),

A first marker (6) provided with two types of markings optically distinguishable from the video camera (5), which is in a known fixed position, and

A second marker (7), which is provided with two types of markings optically distinguishable from the video camera (5) and rigidly connected to a position-variable object whose position to be taken in each case relative to the first marker (6) is to be recorded in a measuring range,

2. Device for a video-based camera navigation system according to claim 1, characterized in that the two on the markers (6, 7) mounted and of the video camera (5) optically distinguishable types of marking consist of a two-dimensional license plate with sharp contours that are filled with a particular color.

3. Device for a video-based camera navigation system according to claims 1 and 2, characterized in that the markers (6, 7) are provided with different markings (9) consisting of several characters.

4. Device for a video-based camera navigation system according to claim 1, characterized in that it can be used as an input device for various program and operating system surfaces.

5. A method of using the device for a video-based camera navigation system according to claim 1, characterized by the following method steps: a) attaching a first marker (6) in a fixed position on the first object (10), b) attaching a second marker (7) on the second object (2), c) aligning the video camera (5) on both markers (6, 7), d) input of the position of the fixed target area, e) image acquisition via the video camera (5), f) image preprocessing, g) object recognition, h) assignment of the objects to the known geometries - classification, i) color determination of the geometries of the markings, j) assignment of the recorded characteristic pixels (for example corner points) of a geometry to the markings attached to the real object, k) calculation of the transformation T above the assigned points of the markers and the camera parameters (focal length, image distortion), I) display the image on the screen (1), m) acquire the new positives ones of the two markers (6, 7) by the video camera ^¬ (5), n) transmitting the new camera parameters to the electronic data processing installation (8), o) repeating steps e) to n).