DE102015212352A1 - Method, arrangement and computer program product for the position detection of an object to be examined - Google Patents

Abstract

The present invention relates to a method, an arrangement and a computer program product for detecting the position of an object to be examined (1, 2), on which a marker (3) with an optically readable structure (4) is fixed, at least dimensions and / or a Marked marker size (3). For this purpose, an optical recording (8) of at least one single image (21) of the object to be examined is carried out with an optical recording unit (5) and transmission of the at least one individual image (21) to an evaluation unit (7), performing a marker localization (9) in the at least one single image (21) by the evaluation unit (7), a definition of an image section (10) in the at least one single image (21) by the evaluation unit (7) in which the marker is located, performing a marker detection (11) the evaluation unit (7), in which the structure (4) arranged on or on the marker (3) is detected, and a processing of the structure (4) detected in the marker detection (12), wherein at least one position and / or an orientation of the marker (3) relative to the optical pickup unit (5) from the dimensions encoded by the structure (4) or the size of the detected marker (3) and the detected structure (4) in the at least one n single image (21) determined by the evaluation and stored in the evaluation unit (7) and / or displayed on an output unit (20).

Description

The present invention relates to a method, an arrangement and a computer program product for position detection of an object to be examined.

There are two different measurement principles for determining a patient position and an instrument position in an operating room or for determining a tool position in a workshop: optical tracking and electromagnetic tracking. Optical tracking uses an external stereo camera measurement system that measures an array of reflector spheres and determines a position of the instruments or patient. A disadvantage of this variant is that only a limited patient access of the medical staff in the operating room is possible in order not to cross a camera directive and thus to disturb the recording (so-called line-of-sight problem). In addition, high investment costs for the purchase of such a system and ongoing costs for the required disposable reflector balls must be taken into account. Moreover, each instrument in the operating room is assigned a unique array of reflector spheres that must be calibrated and specified.

The electromagnetic instrument tracking uses a magnetic field generated by a field generator for measuring a position of a sensor coil on the instrument or in an operating area. This coil makes it possible to obtain information about the position and orientation of the instrument. The main disadvantage of this method is a high degree of inaccuracy and susceptibility to interference with electrically conductive or ferromagnetic materials.

The present invention is therefore based on the object to develop a method and an arrangement which avoid the disadvantages mentioned, with which therefore a position of an object to be examined can be determined with high accuracy and reliability.

This object is achieved by a method according to claim 1, an arrangement according to claim 7 and a computer program product according to claim 10. Advantageous embodiments and further developments are described in the dependent claims.

A method for detecting the position of an object to be examined, on which a marker with an optically readable structure is fixed, which identifies at least dimensions and / or a size of the marker encoded, comprises a plurality of steps. In one of the steps, an optical recording of at least one individual image of the object to be examined is carried out with an optical recording unit and a transmission of the at least one individual image to an evaluation unit. Subsequently, a marker localization is performed in the at least one frame. In a further step, an image section in the at least one single image in which the marker is located is determined by the evaluation unit. A marker detection is performed by the evaluation unit, in which the structure arranged on or on the marker is detected. Finally, the structure detected in the marker detection is processed, wherein at least one position and / or orientation of the marker with respect to the optical acquisition unit from the dimensions or the size of the detected marker and / or the detected structure in the at least one single image the evaluation unit is determined and stored in the evaluation unit and / or displayed on an output unit. The dimensions and / or the size of the detected marker or the detected structure are in this case coded by the optically readable structure, i. H. especially in the structure by signs.

Characterized in that the marker is fixedly attached to the object to be examined and provided with the optically readable structure, information in the optically readable structure can be encoded, which describe the object to be examined in more detail and can be used for a position detection. Due to the fixed attachment can be concluded from a change in position of the marker on a change in position of the object to be examined. This is all the more easily possible, since the dimensions or the size of the marker are already known by the structure of the marker after the recording and can also be correlated with a determined in the marker detection size in the context of processing the detected during the marker detection structure , Since an optical recording technique is used, the position and the orientation can be reliably calculated with a sufficiently high accuracy by the already known size of the marker from the recording unit. In this case, the marker localization initially serves to detect the marker in the respective individual image, whereas the desired information is detected during the marker detection. While the marker localization thus has comparatively low accuracy requirements, the marker should be determinable only with the marker detection with increased accuracy. By outputting the information on the object to be examined obtained by the method on the output unit, a user can also perceive this information. The above steps do not all have to be done in the given order In particular, a position determination or orientation determination can already be carried out before the marker detection is carried out from the size of the marker.

Preferably, the structure of the marker indicates a size, a name, a type and / or a shape of the object to be examined. This information is used for a more accurate detection of the respective object, in particular if several objects to be examined are used, each with markers arranged thereon. However, a transformation from a centroid of the marker or of the object to be examined to a predetermined point in the structure of the marker may also be coded. Even previously unknown objects to be examined, for example, previously unused instruments can be detected and used so quickly with their properties. The method can thus be carried out faster and more reliably than comparable methods for position detection.

It can be provided that the distance between the marker and the optical recording unit is determined by the evaluation unit in order to obtain additional information about the arrangement of the marker in a spatial area detected by the optical recording unit.

Typically, the structure of the marker is formed by a bar code, a QR code (quick response code), at least one edge, and / or circular structures having at least two colors. The structures mentioned are optically simple and unambiguous to capture and offer a relatively large amount of information in a relatively small space. Alternatively or additionally, the structure of the marker may also contain the information in any desired encrypted form, so that the evaluation unit can access the information only if the key stored in the recording unit is known to decrypt.

In one embodiment, prior to the optical recording of the at least one individual image, a calibration of the recording unit is carried out in order to ensure a proper functioning of the recording unit.

It can also be provided to record a plurality of individual images one after the other and to record the marker and the structure in each of the individual images as described.

Typically, during processing of the structures, the evaluation unit determines a transformation between the marker and the optical acquisition unit in the at least one single image in order to determine and / or check the position of the marker and the orientation of the marker. This is preferably done by a homography transformation, ie a transformation between two planes, which in the present invention are typically given by the flat marker and a likewise typically flat recording plane of the optical recording unit, for example a camera chip.

In one embodiment, non-linear optimization may be performed by the evaluation unit in processing the structure to avoid backprojection errors.

An arrangement for detecting the position of an object to be examined, to which a marker with an optically readable structure is fixed, has an optical recording unit and an evaluation unit. The arrangement is adapted to carry out the described method. Accordingly, the method described above may be configured to be carried out on the assembly.

Typically, the arrangement is part of an equipment of an operating room, the method is thus also carried out in the operating room.

It may be provided that the optical recording unit is arranged on a light source of the operating room. Since the light source is usually present anyway in the operating room, the arrangement can be integrated to save space in this device. Typically, the evaluation unit is integrated into this light source.

Alternatively or additionally, the optical recording unit can be designed as a stereo camera in order to increase the accuracy and robustness of the marker detection. A stereo camera is to be understood here in particular as a camera with two lenses and two imaging elements, which are arranged in a single housing. The optical recording unit can also have two individual cameras in separate housings or more than two individual cameras in a housing or in individual housings.

A computer program product has a command sequence stored on a machine-readable carrier, preferably a digital storage medium, for performing the described method and / or for actuating the described arrangement when the computer program product runs on an electronic processing unit. Preferably, the computer program product can be loaded directly into an internal memory of the electronic processing unit or is already stored therein and includes typically parts of a program code for performing the described method or for driving the described device when the computer program product is executed on the electronic processing unit. The computer program product may also comprise a computer program which has software means for carrying out the described method and / or for driving the described device when the computer program is executed in an automation system or on the arithmetic unit.

As an electronic processing unit can also serve the evaluation.

Embodiments of the invention are illustrated in the drawings and are described below with reference to the 1 to 8th explained.

Show it:

1 a side view of a patient and a medical instrument, each provided with a marker, in an operating room;

2 a plan view of a QR code as a marker structure;

3 a perspective view of a provided with the marker instrument and a receiving unit;

4 a flowchart of a method for position detection;

5 a 1 corresponding view of a marker with a stereo camera system;

6 another flowchart of a method for position detection;

7 a perspective view of an instrument with a curved marker and

8th a flowchart with a more detailed representation of the marker localization.

In 1 is shown in a side view of an operating room in which a patient 1 lying on an operating table and on the patient 1 a marker 3 with a marker structure 4 is arranged. The marker structure 4 Contains information about the patient in the form of a QR code formed by black and white rectangles and squares 1 such as name or anatomical abnormalities. During an operation with a medical instrument 2 can over the marker 3 a position and orientation of the patient 1 be checked.

On the instrument 2 is also another marker 3 with another marker structure 4 fixedly arranged. The marker structure 4 is also a QR code of the instrument 2 and details of its dimensions and / or a manufacturer of the instrument 2 contains. Also, in the marker structure 4 Information about the dimensions of the marker structure 4 themselves or their size included, so for example, a size of 4 cm deposited on 4 cm in encoded by the QR code form. In further embodiments, instead of a QR code, a bar code or a two-color structure of any geometric shapes, for example, with circles, can be used. In addition, combinations of different forms of representation of the marker structure 4 possible.

Above the patient 1 is a surgical lamp 6 arranged as a light source, on which a camera 5 is attached, through which several individual images of the patient 1 and the instrument 2 with the respective markers 3 be recorded. These frames will be attached to one of the surgical lights 6 attached evaluation unit 7 transmitted, processed there and a result of the processing on an in 1 schematically shown output unit 20 , such as a display, to a user, such as a surgeon in the operating room.

This arrangement is used to carry out a method for instrument navigation, with the help of the in the surgical lamp 6 or surgical light integrated camera 5 as recording unit and the coded markers 3 as an instrument tracker or patient tracker navigated surgical interventions allows. The surgical lamp positioned over an operating area 6 with the integrated camera 5 captures the coded markers 3 with their marker structures 4 attached to one or more medical instruments 2 as well as on the patient 1 are attached.

From a projection of the marker structures designed as area markers 4 to the camera 5 can be a position and an orientation of marker structures 4 or marker codes are calculated algorithmically, whereby a required for the navigation coordinate transformation between a coordinate system of the patient 1 and a coordinate system of the instrument 2 is known. The location of the markers 3 is determined by their size, distortion and shape. The marker structure 4 With the marker information can also be encrypted in other embodiments, for example, by a hash code. From the determined position of the markers 3 in the camera image and their dimensions determined in the images can be compared with the example in the marker structure 4 deposited known dimensions algorithmically a distance between the camera 5 and the respective marker considered 3 be determined.

Integration into equipment already present in the operating room results in time and box savings and a reduction in a "one-of-sight" issue by positioning a measuring camera over the surgical field and intuitively releasing a field of view of the camera 5 , Thus, an instrument navigation is provided, which is a representation of the location of surgical instruments 2 in a preoperatively recorded three-dimensional data set, for example a volume data set of the patient obtained by computed tomography 1 , includes. This requires the positions of the instruments 2 and the patient 1 or the operating area are continuously determined metrologically and are related to the already existing three-dimensional data set in relation.

As an example of the marker structure 4 is one of the most on the in 1 shown markers 3 used QR codes in 2 shown in plan view. Recurring features are provided with identical reference numerals in this figure as well as in the following figures. The illustrated coding contains information about the size of the square in plan view marker 3 or its dimensions and the instrument type of the instrument 2 at which the marker 3 is attached. The in 2 The example QR code shown after reading is represented as "s50 pointer" and denoted by "s50" has an edge length of 50 mm, which due to the square configuration both length and width are known. "Pointer" refers to the tool on which the marker is located 3 with the illustrated marker structure 4 is attached. Optionally, the marker 3 or the marker structure 4 as coding also have a transformation to the instrument tip. The marker structures 4 and the markers 3 can be designed very flexible. For example, metal flakes or plastic flakes may be patterned by laser engraving or etching, or stickers may be printed and applied to the instrument 2 glued on.

Video image data in the operating room lamp 6 integrated camera 5 are used in the navigation method described in more detail below, wherein, as in 3 shown in a perspective view, from an image of the area marker in a frame of the video image data, the position and orientation of the marker 3 to be determined in the room. That from the camera 5 captured single image 21 is in 3 with an associated coordinate system between the camera 5 and with the marker 3 and the marker structure 4 provided instrument 2 shown schematically. The marker structure 4 appears in the frame 21 due to tilting of the instrument 2 and the marker 3 distorted. Because the marker 3 also well detectable outer edges as additional so-called "feature points" to the black and white transitions of the marker structure 4 may have the marker structure 4 with its distortion in the frame 21 be clearly recorded and by the evaluation unit 7 are processed. In addition, there may be a distance between the marker 3 and the camera 5 be determined.

A corresponding embodiment of a method is in 4 shown in a flow chart. The goal here is the position and orientation of the marker 3 or the marker structure 4 in the video stream from the frames 21 to reconstruct. For this purpose, first a camera calibration 8th for determining the internal camera parameters and eliminating optical distortions. Subsequently, a marker localization is carried out to determine the position data 9 , creating a picture detail in the frame 21 it is determined in which the marker 3 located. A subsequent marker detection 10 read those in the marker structure 4 contained information, such as the size or the transformation to the instrument tip. In the subsequent marker detection 11 there is a precise recognition of the marker structure 4 and thus the determination of point and line correspondences as the basis for the position and orientation calculation, the so-called "pose". This pose calculation 12 is transformed by a homography on the position and orientation of the marker 3 recalculated. The last step is through a non-linear optimization 13 minimizes a backprojection error and thereby determines the optimal pose In addition, as an optimization step using z. B. a Kalman filter stabilizes the illustrated method and thereby outliers are eliminated.

5 shows in one 1 corresponding view on the patient 1 attached marker 3 with marker structure 4 However, the now by a stereo camera system from the surgical lights 6 and 14 each with cameras arranged on it 5 and a common evaluation unit 7 is detected. Instead of two surgical lights 6 and 14 may in other embodiments, a one-piece device with two cameras contained therein 5 be used. Of course, it is also possible to use more than two cameras, which are arranged in individual housings or in a single housing.

The marker navigation can with any number of cameras 5 be implemented. When using more than one camera 5 Increases the accuracy and robustness of instrument navigation significantly. Furthermore, the probability increases that always at least one of the cameras 5 a clear field of view on the marker structures 4 as marker locators. As operating theaters increasingly with cameras 5 For training, visualization and documentation purposes, these cameras can 5 also be used for navigation purposes.

A corresponding process flow diagram when using two cameras 5 is in 6 in a 4 corresponding view shown. Starting with the calibration 15 the first camera 5 and the calibration 16 the second camera 5 the marker detection takes place 11 and the calculation of the position and orientation of the markers 3 So the pose calculation 12 , for each of the cameras 5 individually by the common evaluation unit 7 , From the transformation, ie translation and rotation, the marker 3 to each of the cameras 5 in turn, through a transformation calculation 17 the transformation of the two cameras 5 be calculated to each other. This can be achieved by stereotriangulation 18 the marker position will be determined and an accuracy check 19 as well as an optimization 13 the calculated position from the individual calculations, ie the orientation 12 the first camera 5 and the second camera 5 as well as the orientation by triangulation 18 , be performed.

The evaluation unit 7 is typically an electronic processing unit, such as a computer running on a computer program that performs the described method and / or controls the described arrangement. A computer program product is in the form of a sequence of instructions (or parts of a program code) stored on a machine-readable medium such as a hard disk or a compact disc, which performs the method as described or controls the arrangement when the computer program runs on the electronic processor.

7 shows a further embodiment of the marker 3 in a perspective view. On the instrument 2 is the marker designed as a flath marker 3 now three-dimensional, to ensure its visibility from different directions. This is the marker 3 composed of several mutually tilted platelets, on each of which the marker structure 4 is arranged. In further embodiments, only a portion of the marker structure may be on each of the platelets 4 be arranged. The marker 3 is thus the instrument 2 bent. This makes it possible to achieve an increase in accuracy of the described method for position detection, since now two or more surfaces are used for orientation.

8th shows a detailed embodiment of the method. Starting with a video image capture 22 through the camera 5 , which generates input data for the method, first becomes a frame 21 get in which the marker 3 next to a nonmarker environment 23 is shown.

Subsequently, in the illustrated embodiment, a coarse marker location 9 in which a picture section 24 around the marker 3 is determined by rough structures 25 or markers of the marker 3 as so-called "finder pattern". The enlarged image section 24 with the marker 3 is subsequently processed by a readout 26 the coded marker information from the marker structure 4 and a finer marker localization 27 in the captured image section 24 performed, in which the read marker information is already used and also a line mapping 28 from now determined fine structures and a detection or localization of the fine structures takes place. As a result, in the last step 29 an output of more accurate location data between the marker 3 and the camera 5 respectively.

The QR code shown reads "id = 1001 size = 54 mm Fraunhofer" and denotes an identification number of the instrument to which the marker 3 is attached, or an identification number of the marker itself and an indication of the edge length and an indication of the manufacturer of the instrument to which the marker 3 with the depicted marker structure 4 is attached.

Only features disclosed in the embodiments of the various embodiments can be combined and claimed individually.

Claims (10)

Method for detecting the position of an object to be examined ( 1 . 2 ), to which a marker ( 3 ) with an optically readable structure ( 4 ) is arranged, the at least dimensions and / or a size of the marker ( 3 ), comprising the steps of: a) optical recording ( 8th ) at least one frame ( 21 ) of the object to be examined with an optical recording unit ( 5 ) and transmitting the at least one single image ( 21 ) to an evaluation unit ( 7 ); b) performing a markerlocation ( 9 ) in the at least one single image ( 21 ) by the evaluation unit ( 7 ); c) definition of an image section ( 10 ) in the at least one single image ( 21 ), in which the marker is located, through the evaluation unit ( 7 ); d) performing a marker detection ( 11 ), in which the at or on the marker ( 3 ) arranged structure ( 4 ) is detected by the evaluation unit ( 7 ); e) processing the during marker detection ( 12 ) ( 4 ), wherein at least one position and / or orientation of the marker ( 3 ) with respect to the optical pickup unit ( 5 ) from the through the structure ( 4 ) encoded dimensions or the size of the detected marker ( 3 ) and the recorded structure ( 4 ) in the at least one single image ( 21 ) determined by the evaluation unit and in the evaluation unit ( 7 ) and / or on an output unit ( 20 ) being represented. Method according to claim 1, characterized in that through the structure ( 4 ) of the marker ( 3 ) a size, a name and / or a shape of the object to be examined ( 1 . 2 ). Method according to Claim 1 or Claim 2, characterized in that the evaluation unit ( 7 ) the distance between the marker ( 3 ) and the optical recording unit ( 5 ) is determined. Method according to one of the preceding claims, characterized in that the structure ( 4 ) of the marker ( 3 ) are formed by a bar code, a QR code, at least one edge and / or circular structures with at least two colors. Method according to one of the preceding claims, characterized in that the evaluation unit ( 7 ) when processing the structures, a transformation between the marker ( 3 ) and the optical recording unit ( 5 ) in the at least one single image ( 21 ) to determine the position of the marker ( 3 ) and the orientation of the marker ( 3 ) to determine and / or verify. Method according to one of the preceding claims, characterized in that a non-linear optimization of the evaluation unit ( 7 ) when processing the structure ( 4 ) to minimize backprojection errors. Arrangement for detecting the position of an object to be examined ( 1 . 2 ), to which a marker ( 3 ) is arranged with an optically readable structure, with an optical recording unit ( 5 ) and an evaluation unit ( 7 ), wherein the arrangement is arranged to perform a method according to any one of claims 1-6. Arrangement according to claim 7, characterized in that the optical recording unit ( 5 ) at a light source ( 6 ) is arranged an operating room. Arrangement according to claim 7 or claim 8, characterized in that the optical recording unit ( 5 ) is designed as a stereo camera or as more than two individual cameras. A computer program product comprising an instruction sequence stored on a machine-readable medium for carrying out the method according to one of claims 1 to 6 and / or for driving the arrangement according to any one of claims 7 to 9, when the computer program product runs on an electronic processing unit.

Images