ES2282048B2 - SPACE POSITIONING METHOD OF CYLINDRICAL OBJECTS THROUGH PICTURE ANALYSIS. - Google Patents

Abstract

Spatial positioning method of cylindrical objects by image analysis. This invention consists of a method for determining the spatial position of a cylindrical object, based on the processing of the image captured by a single camera. The sources of information are only the information of the captured video images, the field of view of the camera and the diameter of the cylinder. The 3D position of the cylindrical object is obtained by an analysis of the cross section of the object, and of the edges of the cylinder displayed in the captured image. This solution is interesting in the field of laparoscopic surgery, with three specific applications: automatic analysis of surgical interventions to obtain parameters for their evaluation; augmented reality applications that guide the surgeon; intelligent computerized surgery systems. The method may be useful in other sectors such as industrial or control.

Description

Object spatial positioning method cylindrical by image analysis.

Technical sector

This invention is part of the sector of artificial vision technologies. Present as principal application of the field of minimally invasive surgery (laparoscopy). However, it can be used in any other industrial sector or control in which the problem addressed is presented.

State of the art

The different techniques of object space tracking by means of sensors position. In this type of systems are the sensors optical, electromagnetic or mechanical. These present among their main disadvantages the apparatusity in its use (sensors mechanical), interference problems with metallic objects (electromagnetic sensors) or the need for a field of vision unobstructed (optical sensors).

To avoid the inconvenience of these systems, in recent years techniques are being developed of positioning of objects and tracking of their movements by means of analyzing video images, addressing aspects so disparate as tracking a ball on a soccer field or the face identification and tracking in sequences captured by A security camera.

In recent years, these analysis techniques video are suffering a great boom in the field of surgery minimal invasion, where it is intended to make a positioning and 3d tracking of the laparoscopic tool. Nowadays there are first approaches to the problem found in Recent scientific publications in this field. A first work empirically analyzes the problem of the 3d positioning of the tool in monoscopic video, and concludes that the information present in a laparoscopic image is sufficient to estimate the 3d coordinate of the tool end [1]. Specifically the information is based on an analysis of the size of the tool and the angle that define the edges of it in the image, but only offers a heuristic approach to the problem, without reaching Formalize a method.

In the scientific literature there is a work which raises a 3d positioning method of the tool laparoscopic using the information provided by the edges of the tool in the picture, but that is based on principles of display and illumination of computer generated scenes, something very different from what is proposed in this patent [2].

The authors of this patent have published first two solutions to this problem, based on two aspects of the two-dimensional image other than the one proposed here patent, as are the analysis of the visualized diameter of the tool or the calculation of the vanishing point of its edges in the image [3]. These methods have certain limitations as far as to the precision and stability achieved.

In the field of industrial property there are several object location patents in the field of surgical application, which use the information of some cameras of vision, but that require the stereoscopic information of two or more points of vision [4], [5], [6], [7]. Another alternative is the use of a monoscopic vision introducing a series of markers [8], thus estimating the position and orientation of the  instrumental through the situation in the image of said markers.

Detailed description of the invention

As mentioned in the previous section, there are different techniques that allow to obtain the 3d position of an object in space by means of position sensors. Without However, the application of such systems in certain fields may be unfeasible due to the main inconveniences that they present, as mentioned in the previous section. Therefore, a new form of determine the spatial position of a body through a analysis of images captured by cameras.

To date, location proposals spatial object making use of an analysis of the images give solutions only for images captured in vision Stereoscopic Which is inconvenient in certain applications in the presence of a single camera.

In this invention a method of spatial positioning of cylindrical objects by analysis of monoscopic video images. With this method you get determine the position of any point on the axis of a cylinder that appears in the image with respect to the camera or cameras that They acquire the image. Thus, this invention allows, to Unlike previous positioning methods, determine these spatial coordinates using as the sole source of Monoscopic video information information and no need for incorporate additional elements to the scene, such as markers.

The innovation of this positioning method spatial is the possibility of determining the position relative of any point on the axis of a cylinder with respect to to a camera, using only video images acquired by said camera. It is also necessary to know the field of vision of the chamber and the diameter of the cylinder. The orientation of the cylinder and its 3d position are determined through the edges of the cylinder in the image and of an analysis of the section transversal of it.

An application of the patent is its use as a method of spatial positioning of cylindrical objects in minimally invasive surgery The surgical instruments used in certain interventions, as in the case of laparoscopy, It has a cylindrical shank. This feature allows to apply the present positioning method to determine the spatial position of any instrument point surgical using the images captured by the endoscope.

Other potential applications of the patent are those related to sectors such as industry, robotics, control systems, etc., where location is necessary spatial cylindrical objects through images captured by one or Several cameras

Brief description of the drawings

Figure 1. Perspective of a cylinder (A) in the space and projection its edges in the image plane (B1) and (B2), fixing the optical center at point (C). Straight perpendicular to the axis of the cylinder that intersects it and that it contains the optical center, cuts to said axis at point (N). He point (P) is any point on the axis of the cylinder. The planes of view of the edges of the cylinder (\ Omega1) and (\ Omega2). These sight planes determine the projected edges of the cylinder in its cut with the image plane (\ Gamma).

Figure 2. Cylinder cross section contained in the plane orthogonal to the axis of the cylinder that passes through the optical center (C) and presents as center the point (N). The angle (2 \ lambda_ {N}) is the dihedral angle formed by the planes of view (\ Omega1) and (\ Omega2) in said section. Vectors (u \ {Omega_1}) and (u {\ Omega_2}) are the normal vectors to said view planes.

Detailed exposition of an embodiment of the invention

This section describes the steps main features of the spatial positioning method of cylindrical objects by image analysis.

First the geometry is described involved in the proposed positioning method. Stage of work consists of the spatial placement of a camera, known its optical center (C), and of a cylindrical object (A) whose position is within the field of view of the camera (See Figure 1). The camera acquires the movements in the space of the cylindrical object through sequences of video images.

In this way, the images acquired by the chamber show cylinder displacement. According to the projective geometry of images, view planes (\ Omega1) and (\ Omega2) containing each of the edges from the cylinder and to the optical center (C) of the chamber, cut to the plane of the image (\ Gamma), defining the projected edges of the cylinder, (B1) and (B2).

The innovation of the method is the determination of the position relative to the optical center of the camera of any cylinder axis point using information from the edges of the projected cylinder, two points of the edges of the cylinder and of any point of its axis in the image.

It is therefore necessary a stage of analysis of images that allow extracting 2d information from the image. In this stage, it is required to know the direction of the projections of both edges, the direction of the view beam of one point of each of the projected edges of the cylindrical object and the direction of the ray of sight of a point of the axis of the cylindrical object. The directions of both edges in the image are determined through the Image analysis On the contrary, the lightning direction of View of a cylinder point is calculated using the Field of View (FoV) parameter or field of view of the camera. This parameter allows to determine the direction of the sight ray (vector that joins the optical center of the camera with a point in space) from any point of the image.

With this data we proceed to the analysis of the cross section of the cylinder, to determine the position relative of a point of the cylinder axis (P) with respect to the center  optical camera It can be divided into two stages:

one.

Determination of the distance from the cylinder to the optical center of the chamber and its orientation , understood as the distance perpendicular to the axis of the cylinder that separates it from the optical center of the chamber.

Once known the beam director vector to a point on one of the edges and the direction of said border in the image, the vector is determined normal (u \ {Omega_1}) to the plane of view of the edge as a product vector of both vectors, since both are contained in said plane of view. Proceed in the same way to obtain from the normal vector (u \ {Omega_2}) to the plane of view of the second cylinder edge (See figure 2).

The address u orientation of the cylinder axis in space is given by the Vector product of normal vectors to the view planes. Similarly, the vector sum of these vectors indicates the direction of the line perpendicular to the axis of the connecting cylinder this one with the optical center of the camera (\ vec {u} _}).

Then the half angle (\ lambda_ {N}) defined by the planes of seen in the cross section, half of the dihedral angle, as show the following equation,

[Ecu. 1] so \ \ lambda _ {N} = \ frac {\ arrowvert \ vec {u} _ {\ Omega_ {1}} + \ vec {u} _ {\ Omega2 {}} \ arrowvert} {\ arrowvert \ vec {u} _ {\ Omega_ {1}} - \ vec {u} _{\Omega 2} \ arrowvert}

Distance between the optical center and the cylinder is determined by knowing the amplitude of said angle (λ N) and the diameter of the cylinder (\ Phi_ {CIL}). The equation for its determination is the which is shown below,

[Ecu. 2] \ arrowvert CN \ arrowvert = \ phi _ {CIL} / sen λ N

2.

Determination of the relative position of any point on the axis of the cylinder with respect to the optical center of the chamber .

In the image, set any point on the axis of the cylinder and known the direction of the beam of sight at that point (\ vec {u} _ {CP}), determine the position in space of that point by previously calculated parameters such as,

[Ecu. 3] \ overline {CP} = \ frac {\ arrowvert CN \ arrowvert \ cdot \ vec {u} _ {CP}} {\ vec {u} _ {CP} \ cdot \ vec {u} CN

In this way, the position can be determined relative of any point on the axis of a cylinder with respect to the optical center of the camera that acquires the recording of the movements, using exclusively the images that are captured by said chamber and knowing the diameter of the cylinder and the field of vision of it.

Industrial application

As already mentioned in the sections above, this invention can be applied as a system of location of cylindrical bodies in space.

This invention presents an interesting application in the field of minimally invasive surgery, specifically laparoscopy, where the feeling of depth is lost due to the intervention display on a screen. This method of positioning allows to determine the position of the end of the laparoscopic instruments (which has a cylindrical stem) with respect to the camera through image sequences acquired by it. In addition, the invention provides added value. by not requiring the incursion of additional elements in the scenario, in this case, in the patient's body. In this field Four specific applications are worth highlighting: the analysis Automatic surgical interventions to obtain parameters objectives for their evaluation; application development  augmented reality to guide the surgeon, in which the knowledge of the position of the tools is vital; he study of biomechanical properties of tissues in the field laparoscopic, by incorporating force sensors in the tools; the design of intelligent robotic systems of surgery.

In turn, this method of positioning cylindrical bodies allows its application in many other sectors as the industrial, in control systems, robotics, ...

       \ vskip1.000000 \ baselineskip
    

Bibliography

[one]

O. Tonet , TU Ramesh , G. Megali , and P. Dario , "Tracking endoscopic instruments without localizer: image analysis-based approach", Stud. Health Technol Inform., Vol. 119, pp. 544-549, \ underbar {2006}.

[2]

C. Doignon , F. Nageotte , M. de Mathelin . "Segmentation and Guidance of Multiple Rigid Objects for Intra-operative Endoscopic Vision". International Workshop on Dynamical Vision, in conjunction with ECCV 2006, Springer Graz (Austria), May \ underbar {2006}.

[3]

AM Cano , P. Lamata , F. Gayá , F. del Pozo , EJ Gómez . "New Methods for Video-Based Tracking of Laparoscopic Tools", Lecture Notes in Computer Science 4072, pp. 142-149, July, \ underbar {2006}.

[4]

MP Heilbrun , P. McDonald , JC Wiker , S. Koehler , W. Peters . "Medical instrument localizing apparatus for surgical applications, establishes 3D workspace coordinate framework, from calibration image pair generated by fiducial structure", US2001039421-A1; US6491702-B2.

[5]

S. Koehler , W. Peters , JC Wiker , P. McDonald , MP Heilbrun . "Photogrammetric surgical localization using two video cameras to provide two dimensional images with a fiducial structure used for calibration and a projection algorithm used to reconstruct the 3D framework", US6165181-A.

[6]

S. Koehler , MP Heilbrun , W. Peters , P. McDonald , JC Wiker , "Photogrammetric surgical localization appts. - defines location of medical instrument relative to elements in medical workspace". US5603318-A.

[7]

S. Lu , "3D position measuring system for movie animation, virtual reality - determines zero crossings of image lines using their derivative order, after which connecting points between zero crossings with respect to image lines is determined for judging pixel consistency". US5852672-A.

[8]

H. Mostafavi , "Surgical instrument tracking method eg for biopsy needle, involves determining if prior image data set obtained after calibrating video camera using marker block attached to the instrument, is within threshold", US2003063292-A1; WO2004023783-A2; AU2003268401-A1; EP1535457-A2; US6973202-B2.

Claims (6)

1. Method for determining the position in space of any point of the axis of a cylindrical object, of known diameter, that moves freely, characterized by the exclusive use of the analysis of a single video image acquired by a single camera in a given moment, whose field of vision is known, and in the absence of any type of external marker. 2. Method according to claim 1, characterized in that the analysis of the video image captured by the camera comprises the following steps:

-

2d information analysis of cylinder from the image consisting of obtaining the direction of the projections of the edges of the cylinder, the direction of the ray of view of a point on each of the projected edges of the cylindrical object and direction of the point of view ray any of the cylinder shaft

-

Analysis of the cross section of the cylinder consisting in determining the distance of the cylinder to the optical center of the camera and its orientation, and the determination of the relative position of any point on the axis of the cylinder with respect to the optical center of the chamber.

3. Method according to claims 1 and 2, characterized in that the following operations are carried out to determine the distance of the cylinder from the optical center of the chamber and its orientation:

-

Determination of the address u orientation of the cylinder axis using the vector product of normal vectors to the plane of the cylinder edges, u \ Omega 1 and u \ Omega 2, obtained from the 2d information extracted from image analysis

-

Determination of the address of the straight perpendicular to the axis of the cylinder and linking it with the optical center of the camera (\ vec {u} _}), by adding the normal vectors of the edges of the cylinder (u \ Omega 1 and u \ Omega 2)

-

Semi-angle calculation defined by the view planes in the cross section, angle (\ lambda_ {N}), by the following equation:

tan \ \ lambda _ {N} = \ frac {\ arrowvert \ vec {u} _ {\ Omega_ {1}} + \ vec {u} _ {\ Omega _ {2}} \ arrowvert} {\ arrowvert \ vec {u} _ {\ Omega _ {1}} - \ vec {u} _ {\ Omega 2} \ arrowvert}

-

Determination of the distance between the optical center (C) of the chamber and the axis of the cylinder, specifically at point (N), known its diameter (\ Phi_ {CIL}), by the following equation:

\ arrowvert CN \ arrowvert = \ phi CIL / sen \ lambda N 4. Method according to claims 1 to 3 characterized in that for the determination of the relative position of any point of the axis of the cylinder (P) with respect to the optical center (C) of the chamber the following operation is performed: \ overline {CP} = \ frac {\ arrowvert CN \ arrowvert \ cdot \ vec {u} _ {CP}} {\ vec {u} _ {CP} \ cdot \ vec {u} CN where \ vector {CP} is the vector unit of the view beam that joins the optical center (C) of the camera with the point P of the cylinder, obtained through the field parameter of vision of the camera. 5. Use of the method described in the claims 1 to 4 in laparoscopic surgery, specifically in the spatial positioning of laparoscopic instruments by analysis of images captured by an endoscope. 6. Use of the method described in the claims 1 to 4 in other fields such as production industrial, control systems and robotics.