AU2015203781B2

AU2015203781B2 - Gradient search integrated with local imaging in laser surgical systems

Info

Publication number: AU2015203781B2
Application number: AU2015203781A
Authority: AU
Inventors: Ferenc Raksi
Original assignee: Alcon Inc
Current assignee: Alcon Inc
Priority date: 2010-02-05
Filing date: 2015-07-07
Publication date: 2017-04-13
Anticipated expiration: 2031-02-04
Also published as: AU2015203781A1

Abstract

Techniques and systems for gradient search are provided based on sensing or measuring at selected locations of a target object without performing full-field sensing or measuring over the entire field of the target object. Search methods are provided to include 5 determining a coordinate of a boundary of a region in relation to a loop in a proximity of a first location, determining a direction of a gradient of the coordinate corresponding to the first location, and selecting a second location based on the determined direction. A search system can be implemented to include an imaging system to determine a coordinate of a feature of an object on a loop in a proximity of a first location, and a controller, coupled to the imaging 10 system, to determine a direction of a gradient of the coordinate corresponding to the first location, and to select a second location based on the determined direction.

Description

GRADIENT SEARCH INTEGRATED WITH LOCAL IMAGING IN LASER SURGICAL SYSTEMS

Technical field [0001] The present application is a divisional application from Australian Patent Application No. 2011212863, the entire disclosure of which is incorporated herein by reference.

[0001a] This patent document relates to gradient search techniques for various applications, including ophthalmic surgical applications.

Background [0002] There are many methods to image objects or portions of objects for a wide variety of purposes, including manufacturing, diagnostics, quality control, surgical applications and many more. Optical Coherence Tomography (OCT) is one of the many methods to create three dimensional images and extract structural information of materials. This is usually done by scanning an optical beam over a target region or surface and then analyzing the spatial, spectral and temporal characteristics of the scattered and returned light. Detailed imaging information can be obtained by scanning along a two or three dimensional grid pattern of a sufficiently high spatial density.

[0002a] The discussion of the background to the invention included herein including reference to documents, acts, materials, devices, articles and the like is included to explain the context of the present invention. This is not to be taken as an admission or a suggestion that any of the material referred to was published, known or part of the common general knowledge in Australia or in any other country as at the priority date of any of the claims.

Summary [0003] For many applications, however, there is no need to or it is not advantageous to create a complete image of an entire region or surface. One of these applications is the task to find specific points of an imaged surface or region. These specific points can be extrema of the imaged surface, features of the imaged object, or boundary points or edges of the object. This document discloses technique and systems for gradient search based on sensing or measuring at selected locations of a target object without performing the full-field sensing or measuring over the entire field of the target object. 1 [0004] Ophthalmic surgeries often encounter situations where the surgical procedure can be directed relative to a reference point, such as the apex of the cornea. In cataract procedures, the reference points can be the apex of the anterior or posterior surface of the lens, or its hardened nucleus. Reference points can be identified by, e.g., first scanning the target region with an imaging beam by a series of planar B, radial or cylindrical OCT scans. The scanned data can be analyzed and used to compose a full three dimensional image of the targeted region, surface or object. These image data then can be stored. Subsequently, search algorithms can be used on the stored image data to identify the maxima or minima of interest.

[0005] However, imaging the entire target region, especially with high density scanning to ensure good resolution, is unnecessarily wasteful as it stores lots of data points which are eventually not used. Further, taking these large amounts of data is time consuming and thus the process is slow. Thus, in applications where time is at a premium, these applications are at a disadvantage.

[0006] According to a first aspect, the present invention provides a computer-implemented method for searching surgical imagery data, based on sensing or measuring a function F(i') at selected locations of an imaged object comprising a surgical ophthalmic target, to search for the extrema of an ophthalmic layer of the imaged object without performing full-field sensing or measuring over the entire field of the target, wherein the method is performed in an integrated searching and imaging system comprising an imaging system and a controller, the method comprising the steps of: (i) selecting a first selected location (xl,yl) of the imaged object; (ii) selecting a first loop centered around the first selected location, the loop having a substantially elliptical or circular form; (iii) sensing or measuring values of the function with an imaging system at the set of locations along the first loop centred around the first selected location; (iv) determining, with a controller, a second location along the first loop where F(i) assumes a maximum or minimum value, and (v) determining the direction of a first gradient vector by considering any two of the first selected location (xl,yl), and the second location where F(i) has a maximum or minimum value, the first gradient vector being based on the values of the F(x) function at the determined second location along the first loop and the first selected location (x 1 ,y 1); (vi) updating the first selected location (xl,yl) along the first gradient to a second selected location (x2,y2); and repeatedly updating the first selected location by applying the steps (ii)-(vi) until the location of the maximum FTnax (x'rnax) or the minimum FTTIjn (5TIliT1) of the function F(x) is found. 2 [0007] In embodiments, the determined coordinate is one of a height, a depth, a z coordinate, and a coordinate along a reference line.

[0008] In embodiments, the loop is one of a closed loop, a loop in a plane, a loop on a predetermined surface, an ellipse and a circle.

[0009] In embodiments, the first location is one of a location in a plane transverse to an axis, a location in a plane transverse to the determined coordinate, a location in an (x,y) plane, a location in a radial coordinate system, and a coordinate on a predetermined surface.

[0010] In embodiments, the coordinate is determined by one of an optical coherence tomography, a depth measurement technology, an optical measurement, and a sensing technology of a merit function.

[0011] In embodiments, the determining a direction of a gradient step includes determining at least one of a direction of a maximum of the coordinate along the loop and a direction of a minimum of the coordinate along the loop, and determining the direction of the gradient based on at least one of the direction of the maximum and the direction of the minimum.

[0012] In embodiments, the selecting a second location step includes selecting the second location by shifting the first location by an increment vector, a direction of the increment vector being approximately parallel to the direction of the gradient.

[0013] In embodiments, the first location is retained when a magnitude of the gradient is smaller than an iteration-stop value.

[0014] In embodiments, the determining a coordinate of a boundary of a region in relation to a loop in the proximity of a first location step includes identifying rays around the first location, and determining the coordinate at ray-points.

[0015] In embodiments, the determining a direction of a gradient of the coordinate around the loop step includes determining a rate of change of the coordinate along the rays, selecting the ray along which the rate of change is maximum, and identifying a direction of the selected ray as the direction of the gradient, and the rate of change along the selected ray as the magnitude of the gradient.

[0016] In embodiments, the method includes shifting the first location by a shake-up vector having at least one of a direction making a non-zero angle with the direction of the gradient, or a magnitude substantially different from the magnitude of the gradient. 3 [0017] In embodiments, the search method is performed without determining the coordinate of the boundary of the region in one of a volume, along parallel lines, in a two dimensional raster, on a grid, and in a raster on a surface.

[0018] In embodiments, the boundary of the region is an ophthalmic layer of one of a cornea, a lens, and a cataract, the coordinate is a depth of the ophthalmic layer, and the search method includes determining an extremum of the depth of the ophthalmic layer.

[0019] In embodiments, the search method is capable of determining the extremum of the ophthalmic layer faster than one of 10 msec, 100 msec, 1 sec and 10 sec.

[0020] According to a second aspect, the present invention provides a search system, comprising: an imaging system that, based on sensing or measuring a function F(i) at selected locations of an imaged object comprising a surgical ophthalmic target, searches for the extrema of an ophthalmic layer of the imaged object without performing full-field sensing or measuring over the entire field of the target, and determines a coordinate of a feature of an object on a loop in a proximity of a first location; via a controller, coupled to the imaging system, the controller adapted to: (i) select a first selected location (xl,yl) of the imaged object; (ii) select a first loop centered around the first selected location, the loop having a substantially elliptical or circular form; (iii) sense or measure values of the function with an imaging system at the set of locations along the first loop centred around the first selected location; (iv) determine, with a controller, a second location along the first loop where F(x) assumes a maximum or minimum value, and (v) determine the direction of a first gradient vector by considering any two of the first selected location (xl,yl), and the second location where F(?) has a maximum or minimum value, the first gradient vector being based on the values of the F(>;) function at the determined second location along the first loop and the first selected location (xl,yl); (vi) updating the first selected location (xl,yl) along the first gradient to a second selected location (x2,y2); and repeatedly updating the first selected location by applying the steps (ii)-(vi) until the location of the maximum FmaX (x„mx) or the minimum Fmin (of the function F(s) is found.

[0021] In embodiments, the imaging system includes a scanner, to scan an imaging beam, and an image acquisition sub-system, to receive and to pre-process a returned imaging beam.

[0022] In embodiments, the imaging system is one of an optical coherence tomography system, a depth measurement system, an optical sensing system, and a sensing system of a merit function. 4 [0023] In embodiments, the controller is configured to determine at least one of a location of a maximum of the coordinate along the loop and a location of a minimum of the coordinate along the loop, and to determine the direction of the gradient by relating any two of the first location, the location of the maximum and the location of the minimum.

[0024] In embodiments, the controller is configured to select the second location by shifting the first location by an increment vector, a direction of the increment vector being substantially parallel to the direction of the gradient.

[0025] In embodiments, the controller is configured to retain the first location when a magnitude of the gradient is smaller than an iteration-stop value.

[0026] In embodiments, the imaging system is an ophthalmic coherence tomography system, the imaged object is an ophthalmic layer of one of a cornea, a lens, and a cataract, the coordinate is a depth of the ophthalmic layer, and the controller is configured to determine an extremum of the depth of the ophthalmic layer.

[0027] In embodiments, the search system is configured to determine the extremum of the ophthalmic layer faster than one of 10 msec, 100 msec, 1 sec and 10 sec.

[0028] In embodiments, a search method for surgical imaging includes the steps of: performing a local search of a merit function of an imaged region in a proximity of a first location, determining a preferred direction based on the local search, and selecting a second location based on the preferred direction.

[0029] In embodiments, the performing the local search step includes surgical imaging by one of an optical coherence tomography, a depth measurement technology, an optical sensing technology and a sensing technology of the merit function.

[0030] In embodiments, the performing the local search step includes determining one of a height, a depth, a z coordinate, a coordinate along a line of reference, an optical density, and an optical scattering intensity.

[0031] In embodiments, the performing the local search step includes determining the merit function along one of a closed loop, a loop in a plane, a loop on a predetermined surface, an ellipse, a circle, an essentially closed surface, an ellipsoid, and a sphere.

[0032] In embodiments, the performing the local search step includes determining the merit function along local rays. 5 [0033] In embodiments, the determining a preferred direction step includes determining a direction of a gradient of the merit function.

[0034] In embodiments, the determining the direction of the gradient step includes determining at least one of a location of a maximum of the merit function on an essentially closed surface or loop in the proximity of the first location, and a location of a minimum of the merit function on the essentially closed surface or loop in the proximity of the first location and determining the direction of the gradient by relating any two of the first location, the location of the maximum and the location of the minimum.

[0035] In embodiments, the selecting a second location step includes selecting the second location by shifting the first location by an increment vector, a direction of the increment vector being substantially parallel to the direction of the gradient.

[0036] In embodiments, the method includes retaining the first location when the magnitude of the gradient is smaller than an iteration-stop value.

[0037] In embodiments, the search method is performed without determining the merit function in one of a volume, along parallel lines, on aligned surfaces, in a two dimensional raster, and in a three dimensional raster.

[0038] A method for searching a gradient of a function over a target object is also described herein to include selecting a first location of the target object; performing a sensing or measuring operation at the first location to obtain a respective value of the function; selecting a first loop having multiple first loop locations of the target object that are different from the first location; and performing a sensing or measuring operation at the multiple first loop locations of the first loop, without performing the sensing or measuring operation at other locations, to obtain values of the function at the respective multiple first loop locations. One of the multiple first loop locations that has the maximum or minimum value for the function among the multiple first locations is selected. The first location and the selected first loop location that has the maximum or minimum value for the function among the multiple first loop locations, and corresponding values of the function are used to determine a first gradient between the first location and the selected first loop location. Next, a second location of the target object is selected along a direction of the first gradient and a sensing or measuring operation at the second location is performed to obtain a respective value of the function. A second loop having multiple second loop locations of the target object that are different from the second location is selected and a sensing or measuring operation at the multiple second loop locations, without performing the sensing or measuring operation at 6 other locations, is performed to obtain values of the function at the respective multiple second loop locations. One of the multiple second loop locations that has the maximum or minimum value for the function among the multiple second t loop locations, and corresponding values of the function are used to determine a second gradient between the second location and the 5 selected second 5 loop location. 2015203781 01 Mar 2017 [0038a] Where the terms "comprise", "comprises", "comprised" or "comprising" are used in this specification (including the claims) they are to be interpreted as specifying the presence of the stated features, integers, steps or components, but not precluding the presence of one or more other features, integers, steps or components, or group thereof. 10 [0039] The above and other aspects of the techniques and systems for gradient search based on sensing or measuring at selected locations of a target object are described in detail in the drawings, the description and the claims. 6a

Brief Description of the Drawings 2015203781 07 Μ 2015 [0040] FIG. 1A illustrates the principle of the gradient search algorithm.

[0041] FIG. IB illustrates a grid-scanning pattern used for imaging.

[0042] FIG. 2 illustrates an embodiment of a gradient search integrated with local 5 imaging method 100.

[0043] FIGS. 3A-B illustrate the local search step in association with a loop corresponding to a first location.

[0044] FIG. 4 illustrates the iterative aspect of the gradient search algorithm.

[0045] FIGS. 5A-B illustrate a ray-based local search step. 10 [0046] FIG. 6 illustrates a search algorithm based on a local search of a merit function 100’.

[0047] FIG. 7 illustrates an integrated searching and imaging system 200. 7

Detailed Description [0048] The challenge of finding the coordinates Xext of the extrema of a function

FeXt(XeXt) arises in various applications, where x is a position vector in a space of two or more dimensions, such as a position vector on an object surface or in an object volume of interest. The basis of the gradient, or steepest descent or ascent method is that if the function F(x) is real-valued and is differentiable in a neighborhood of a location a, then F(x) varies fastest in the direction of the gradient vector of F at a, VF(a) . Therefore, the

Xmax location of the maximum Fmax (-^max ) can be found efficiently by repeatedly updating an initial location in the direction of the gradient vector, corresponding to each update.

Similarly, the A'lllin location of the minimum Fmin (-Fmm ) can be found efficiently by repeatedly updating an initial location in the direction opposite to the gradient vector, corresponding to each update.

[0049] In an example of the maximum-searching algorithm the position vector is moved or updated from xl to x2 according to the relation:

χ2=Α+r^F(x1). CD

For a sufficiently small γ>0, the update increases the value of the function F(x):

Fix^ < F(x2 ) . Repeatedly updating the location x along the corresponding gradient vectors is therefore an iterative process which generates a sequence of locations x(l),x(2),...x(n) with the iterative relation: x(n +1) = x(n) + r(n)VF(x(n)). (2)

The above iterative relation generates a corresponding sequence of growing values of the function F(x):

Fm))<F(x(2))-<F(x(n)). (3)

These values converge to a maximum of F(x), at least locally. In a generic formulation, the starting coordinate 5:(1) can be chosen randomly, or based on some preexisting knowledge or on a guess of the location of the maximum. The step size, controlled by ]{n), can be a constant, or a variable that depends on n, chosen according to some suitable condition. 8 [0050] The analogous minimum-searching algorithm can update the location x according to: x(n +1) = x(ri) - y(n)VF(x(n)), (4) and proceed analogously otherwise.

[0051] FIG. 1A illustrates a gradient search algorithm. Here x is a two dimensional location (x,y) and F(x) is a scalar function. The closed loops are iso-value lines, or contour lines, connecting points where F{x) assumes the same value. The set of embedded closed iso-value lines enclose an extremum E which can be either a maximum or a minimum.

[0052] Referring to the maximum-searching algorithm, the gradient search algorithm can use a step-vector, or update-vector, which updates the initial location (xl,yl) with a vector parallel to the gradient vector at that point: VF(jd, yl), resulting in the updated location (x2,y2), as shown in Eq.(2). The length of the update-vector is controlled by }{n). FIG. 1A shows that the direction of the gradient vector at each location is orthogonal to the contour line going through the same location.

[0053] Repeated updating of the location through the sequence (xl,yl), (x2,y2), ...(xn,yn) generates a sequence of increasing function values: F(xl, yl) < F(x2, y2)... < F(xn, yn) (5) [0054] The gradient search method typically monitors the sequence of function values. When the increase of the function values falls below a threshold value, the algorithm presumably reached a close proximity of maximum of the function. The algorithm is then typically stopped and the location of this final point -£max = (xf,yf), as well as the corresponding maximum value of the function Fm.ix (xmait) are reported as the result of the algorithm.

[0055] Some algorithms can involve a step of shortening the length of the update-vector through decreasing y(n) when the increase of the function falls below the threshold value, to avoid overshooting, or the search entering into a non-convergent cycle.

[0056] The above described gradient search algorithm can be applied and implemented in several different ways. The function F(x) can be a wide variety of quantities, including, for example, a height, depth, temperature, optical or electromagnetic absorption, a 9 merit function for a search and a scalar function in various applications. Accordingly, the location X can be a multidimensional vector, including spatial coordinates in two or three dimensions, in a relative or fixed coordinate system. In a generic search for optimizing a merit function, the location can be an appropriate state vector.

[0057] The values of the wide variety of functions, F(x), can be acquired by a wide variety of suitable methods, often involving sensing or measuring operations on a target surface or within a target volume of an object. Examples of sensing or measuring operations for obtaining values of the function F(x) include: measuring the height or depth of a target object surface, a thickness of a target object tissue, volume or region by mechanical sensors or by optical imaging; measuring a refractive property, a density, or an other optical property; and measuring the temperature distribution by temperature sensors, e.g., direct contact temperature sensors, or by infrared sensors operating from a distance. The sensed values of the function F(x) can then be used to perform the rest of the gradient search method, as described above.

[0058] Many gradient search techniques perform a full-field sensing or measuring operation to obtain the values of the function F(x) for the entire target area or volume of an object. As an example, FIG. IB illustrates that some methods can perform the gradient search on a grid-scanned dataset. The scanning can be performed on an (x,y)-gnd, the scanning data stored, and the gradient search algorithm performed subsequently on the stored scanning data. FIG. IB shows a sparse grid for clarity. In many applications, a much denser grid can be applied to reach the desired precision.

[0059] Performing the gradient search on the full-field grid-scanned data may have drawbacks and limitations in certain applications. For example, sensing the values of the function Fix) of the entire target area or volume of the object can acquire a large number of data points which are often not used for the subsequent gradient search. As a result, such grid-scanning can be wasteful of the processing time. For another example, the grid-scanning method can be slow, since acquiring the unused scanned data takes time and this slow process can undermine the applicability of this method for time sensitive applications which require finding the extremum in a short time.

[0060] Search techniques described in this document can be implemented in ways that address these and other technical issues and challenges by an integrated gradient search which senses the function F{x) only locally, determines the gradient on this small set of 10 locally sensed values and updates the location x based on the determined gradient. Such an integrated technique has a built-in intelligence to select only a limited number of locations to perform the sensing or measuring the values of the function F(x) without performing the full-field sensing or measuring over the entire field of the target object area or volume. The selection of the limited number of locations to perform the sensing or measuring operation reduces the processing time and uses the system resources efficiently by only capturing and storing data that is relevant to the gradient search. This integration allows the scanning for the sensing or measuring operation to be local only without being performed globally in the entire field of the target object area or volume of interest. Since these search algorithms involve only local scans, they can be considerably faster than scanning the target over entire grids and thus reduce the scanning and computational effort and time.

[0061] Such implementations can be useful for imaging applications which do not require the full two or three dimensional grid-scanned information about the imaged object, but instead they are directed towards determining only certain features, for example the extrema of the spatial extent of the object or the location of a maximum of the optical absorption, or any other reference point.

[0062] Some embodiments of this integrated gradient search method can be implemented in ophthalmic applications, where this integrated gradient search method can be useful for several different functionalities: [0063] 1) Implementations can be used to find spatial extrema of boundaries of tissues in the eye. These extrema can be used as reference points for positioning and directing surgical procedures. Reference points can include, for example, an apex of a cornea or an apex of an anterior or posterior surface of a crystalline lens or a retina. Other characteristics, such as a thickness of the lens, or an axial length of the eye can be derived from these measurements.

[0064] 2) Implementations can be used to locate extrema of a cataract in the lens of the eye. The cataract may occupy only a portion of the lens, associated with the most hardened region of the nucleus.

[0065] 3) Implementations can be used to find an extremum of ophthalmic regions to guide surgical procedures where time is of essence. In ophthalmic procedures patients often lose their ability to keep their eyes motionless after 90 - 120 seconds. Therefore, fast searching for reference points to guide the ophthalmic surgery can be quite important for a timely completion of the ophthalmic surgery. 11 [0066] While below the implementations are explained primarily in surgical context, embodiments can be also used in the context of any of the applications listed above, further including any diagnostic procedures or manufacturing process as well, where any kind of imaging process is used, such as optical coherence tomography, or OCT.

[0067] FIG. 2 shows an embodiment of the Gradient Search Integrated with Local Imaging (GSILI) method 100. The GSILI method 100 first determines a coordinate of a boundary of a region in relation to a loop in the proximity of a first location in step 110.

Next, the method 100 determines a direction of a gradient of the coordinate corresponding to the first location in step 120. Finally, a second location is selected based on the determined direction in step 130.

[0068] In implementing the method 100, the boundary of the region can be a surface of an imaged three dimensional object. The boundary can be defined by expressing e.g. the z coordinate of the surface as a function of the corresponding (x,y) location. In the language of the above general description, this z coordinate or height is the F(x) function whose extrema are sought: F(x) = z(x, y).

[0069] The first location can be a location in a plane transverse to an axis, a location in a plane transverse to the determined coordinate, a location in an (x,y) plane, a location in a radial coordinate system, and a location on a predetermined surface. The above described Cartesian coordinate system is but one example where the (x,y) coordinates define the first location and the z coordinate is the determined coordinate.

[0070] In step 110 a first loop can be selected in relation to the first location. The first loop can be centered around the first location, or can be in the proximity of the first location. Next, the values of the function F(x) can be sensed or measuredat a set of locations along a first loop. Specific examples for implementing the method 100 are described below in terms of an algorithm searching for the extrema of a coordinate which can be a height, a depth, a z coordinate, and a coordinate along a reference line. The GSILI method 100 can be implemented for various other F(x) functions, including an optical property, a merit function, and a scalar function.

[0071] After the sensing or measuring operation is performed on the first loop around the first location, some implementations of method 100 select the loop location on the first loop that has the maximum or minimum value of the F(x) function. A first gradient can be determined based on the values of the F(x) function at this selected loop location and the 12 first location. Along this first gradient, a second location can be selected within the target area or volume of the object, in effect “updating the first location”. The cycle can be completed by selecting a second loop in the proximity of the second location.

[0072] The new cycle starts by sensing or measuring the values of the function F(x) along the second loop. Next, the loop location on the second loop is selected that has the maximum or minimum value of the F(x) function. A second gradient can be determined based on the values of the F(x) function at this selected loop location and the second location. Along this second gradient, a third location within the target area or volume of the object can be selected and the second location updated to this third location. The cycles of the above process can be repeated until the search for the maximum or minimum is completed.

[0073] In the above process, the use of a loop in connection with a location provides the local intelligence for selecting the next location for the gradient search. The sensing or measuring operation is performed locally along the loop rather than globally at all locations on the grid within the entire target area or volume of the object. Therefore, the local sensing or measuring operation and the updating of the location according to the local gradient are integrated and interleaved during the gradient search. These steps and their various implementations will be now described in more detail below.

[0074] The GSILI method 100 can search for the extrema of a boundary of the imaged object. An ophthalmic example is the apex of a section of the eye, such as the cornea. Another example is the apex of the hardened center of a nucleus of the lens.

[0075] FIG. 3A illustrates that in step 110 the coordinate, such as the z coordinate of the boundary of the imaged object, can be determined on a loop 112-1 in the proximity of a starting point (xl,yl). In an example the loop 112-1 can be centered around the starting point (xl,yl). In other examples, the starting point (xl,yl) can be off the center of the loop 112-1, or even outside the loop, in a predetermined relation with it. The loop 112-1 can be a closed loop, a loop in a plane, a loop on a predetermined surface, an ellipse and a circle. In some cases there can be one or more discontinuities along the loop 112-1.

[0076] In some embodiments of the step 110 the values of the coordinate can be determined not continuously, but only at a set of points along the loop 112. The points can be located densely or with some separation along the loop. 13 [0077] In step 110 the coordinate can be determined by an optical coherence tomography (OCT), a depth measurement technology and a sensing technology of a merit function.

[0078] In an example, a focal spot of an imaging beam can be scanned along the loop 112-1 by a scanner. The loop can lie in the (x,y) plane, centered at the point (xl,yl). The coordinate, such as the z = z(xi,yi) depth of the boundary of the imaged object, can be determined by sensing the imaging beam returned from the boundary by an image acquisition system. The z depth coordinate can be sensed essentially continuously, or at a set of points (xi,yi) as the focal spot of an imaging beam is scanned along the loop 122-1.

[0079] The surface z = z(x,y) can be distinguished e.g. by the optical scattering being measurably different on the two sides of the surface. Any optical imaging technology can be used for determining the coordinate, measuring reflectivity, scattering, transmittivity or absorptivity. The measurement can utilize any part of the electromagnetic spectrum, including the infrared and ultraviolet portions.

[0080] The overall application of the GSILI method 100 can be started by selecting a starting location (xl,yl). The starting location can be selected randomly, or by using some predetermined information, e.g. a result of a prior imaging, facts from general knowledge, or an educated guess.

[0081] FIG. 3B illustrates a possible result of step 110: the value of the coordinate determined along the loop, parametrized by the polar or azimuth angle φ. Sometimes this type of scanning is called creating a circular B-scan.

[0082] In step 120 a direction of a gradient can be determined by recalling that the direction of the gradient at a location is essentially the direction of the maximum change of the coordinate at that location. Thus, the direction of the gradient can be determined by determining at least one of a maximum of the coordinate Zmax along the loop and its direction relative to the center of the loop, characterized by the corresponding azimuth or polar angle φγηαχ, and a minimum of the coordinate zmin along the loop and its direction relative to the center of the loop, characterized by the corresponding azimuth or polar angle φηίη. From these values the direction of the gradient can be determined based on at least one of the directions of the maximum and the minimum. The direction of the gradient can be identified by different methods, e.g. as: 14 (i) the direction of the maximum, characterized by a vector originating at (xl,yl) and pointing in the direction of the azimuth angle </>max', or by (ii) the direction of the minimum, characterized by a vector parallel to the direction of the azimuth angle (f)min, pointing towards (xl,yl); or by (iii) directly connecting the points along the loop corresponding to φηαχ and φ,,,ι,,.

[0083] The magnitude of the gradient in the above three cases (i)-(iii) measured on a loop of radius r can be approximated by: (i) I Vz(*l, yl) I« (zmax - z(x 1, yl))/ r); or (ii) I Vz(*l, yl) I = (z(x 1, yl) - Zmin)/ r); or (iii) I Vz(*l, yl) I * (zmax - ) / 2r).

[0084] Since the gradient is a numerical derivative, or at least a finite differential of discrete data, typically it can have undesirably poor precision, or equivalently, high noise. To remedy this problem, the numerical noise in the values of the polar angles φΜαχ and φηύη can be reduced by averaging or curve fitting the polar angle φ along the loop. The numerical noise in the magnitude of the gradient can be reduced by accepting the direction of the gradient as any of the above three possibilities, and then performing additional scanning steps along this gradient direction at points at distances rn from the first location (xl,yl). The points at rn distances can be equally spaced on the ray connecting the first location at (xl,yl) and the point on the loop corresponding to Zmax, or selected by some other criterion. Labeling the newly scanned z values on the ray connecting the first location at (xl,yl) and the point on the loop corresponding to Zmax as Zmax(rl), ZmaJr2), ... zmax(rn) and the newly scanned values on the ray connecting (xl,yl) and the point on the loop corresponding to zmin as zmin(rl ’), Zmin(r2’), ... Zmin(rn’), approximations of the magnitude of the gradient can be obtained by various implementations: (f) I Vz(xU yl) I O) ~ Umax (rn) - z(x 1, yl)) / m); or (ii’) I Vz(x 1, yl) \ {n)~ (z(x 1, yl) - zmin (m’))/m'); or (iii’) I Vz(xl, yl) I (n)« (zmax (rn) - zmin (m'))/(m + rn'). 15 [0085] Any of these three implementations can then proceed by averaging over N of these discrete differentials: (iv’) <1 Vz(xL, yl) I) = -j- ^ I VzUl, yl) I (n) s A n-\ to obtain the magnitude of the gradient with improved precision and reduced noise. Some methods may mix the above implementations. These averaging methods can determine the magnitude of the gradient with increasing accuracy for increasing N. Other noise-reduction methods can be used as well.

[0086] Returning to FIG. 3A, in step 130 a second location can be selected by shifting the first location - in the illustrated case the initial location (xl,yl) - by an increment vector 114-1 along the direction of the gradient. The magnitude of the increment vector 114-1 can be proportional to the magnitude of the gradient, such as the magnitude times the step-size-control parameter γ.

[0087] In some implementations where the direction of the increment or update vector may not be entirely parallel to the gradient, but can be related to it. The increment vector can make a predetermined angle with the gradient vector, or it can be offset from the gradient’s direction by a random angle.

[0088] FIG. 4 illustrates that steps 110-130 can be performed repeatedly, incrementing the locations (xn,yn) progressing towards the final location (xf,yf), where the coordinate z assumes its maximum value: z(xf,yf) = z(Max). Note that zmax denotes the maximum of the coordinate along the individual search loops corresponding to a specific location, whereas z(Max) denotes the overall maximum of the z=z(x,y) surface being searched.

[0089] The magnitude of the increment vectors 114-n, also called the step size, can be proportional to the magnitude of the gradient vector. The constant of proportionality, % can be the same for all steps, or it can vary step-by-step: γ=Ί(η). The other parameters, such as the radius of the loop r, can be also selected by a user of the method 100, or by a controller computer, and can either remain the same or vary step-by-step. These parameters of the method 100 can be selected in order to achieve a fast convergence of the method 100. The parameter selection can depend on the steepness of the gradient, the complexity of the surface and the loop, the desired accuracy, the measurement noise and other considerations. 16 [0090] The GSILI method 100 can be terminated when a magnitude of the gradient is smaller than an iteration-stop value. In some cases the magnitude of the increment vector can be reduced gradually to avoid overshooting or getting into an oscillating or circulating cycle. In practice, when the increase of the z coordinate becomes smaller than a first value, then γ can be reduced to a yZ-cyvalue when continuing the method. This gradual reduction step can be repeated to increase precision.

[0091] FIGS. 5A-B illustrate another embodiment of step 110.

[0092] FIG. 5A shows that the coordinate z can be also determined “in relation to the loop” in the sense that the coordinate z is determined along rays, emanating from the first location and ending on or nearby the loop 112-1. Taking the gradient is a numerical derivation and can produce noisy results, possibly with low precision. This noise could lead to an imprecise determination of the direction and magnitude of the gradient. The averaging implementations (i’)-(iv’) above reduce the noise in the magnitude of the gradient, but still use only two points regarding the direction. Thus the direction of the gradient is determined only with a limited accuracy by the method 100.

[0093] FIG. 5B illustrates that this implementation can include the steps of: (a) identifying several ray directions 1, ... M (only rays 1, 2, and M are shown for clarity); (b) determining the value of the z coordinate at points on the first ray at: ri 1, ri2,...rin and ri 1 ’, ri2’,...rin’, up to points on the M-th ray at rMl, rM2,...rMn and ΓμΓ, Γμ2’, ...rMn’, where n can assume values up to N and n’ up to Ν’; (c) determining the rate of change along the M rays by any of the averaging methods (i’)-(iv’); (d) selecting the ray along which the rate of change is maximum; and (e) identifying the direction of the selected ray as the direction of the gradient, and the rate of change along the selected ray as the magnitude of the gradient.

[0094] In these implementations the number of ray-points N can be between 1 and 100, in some cases between 1 and 10, and the number of rays M can be between 1 and 100, in some cases between 1 and 10.

[0095] For relatively smooth and differentiable surfaces the GSILI method 100 converges to a local extremum of the boundary of the imaged region. However, it is possible 17 that the searched boundary has more than one extrema, only one of them being the global or overall extremum. However, the GSILI method 100 can very well get trapped in any one of the local minima. At the same time, some implementations of the GSILI method 100 are directed towards determining the global extremum of the boundary, and thus the search getting trapped in a local minimum poses a challenge.

[0096] In addition, if the imaging returns the z coordinates with lower precision, the corresponding noise may create lots of shallow apparent minima, again capable of trapping the search algorithm 100, in analogy e.g. to glassy problems in scientific applications.

[0097] To overcome these two types of trapping, implementations of the GSLI method 100 can be supplemented with known variations developed for optimization methods. Some optimization methods avoid the search getting trapped in a local minimum by applying occasional random jumps not controlled by the local gradient.

[0098] These random-jump implementations can shift, or update the location (xn,yn) by a shake-up vector, whose direction is not parallel with the direction of the gradient. The magnitude of the shake-up vector can be chosen to be sufficiently large to move the search out of the basin of attraction of the local minimum. In other embodiments, the direction of the shake-up vector can be related to the direction of the gradient, but the magnitude of the shake-up vector can be considerably different from the magnitude of the gradient.

[0099] Returning to the overall merits of the GSILI method 100, the search method 100 can be performed without performing the grid-scan of FIG. IB, or equivalently, without determining the coordinate of the boundary of the region in one of a volume, along parallel lines, in a two dimensional raster or grid, and in a raster on a surface. For this reason, the GSILI method 100 can be performed in a valuably short time.

[00100] In ophthalmic applications the imaged boundary of a region can be the ophthalmic layer of one of a cornea, a lens, and a cataract; the coordinate can be a depth of the ophthalmic layer; and the search method 100 can be directed to determine an extremum of the ophthalmic layer.

[00101] Since the GSILI method 100 performs local searches instead of grid scanning the entire target region, it can be performed considerably faster. In the above ophthalmic implementation the search method 100 is capable of determining an extremum of the ophthalmic layer faster than one of 10 msec, 100 msec, 1 sec and 10 sec. Given that ophthalmic patients often experience difficulties keeping their eyes under control past 90 - 120 seconds, and given the time constraints of the entire surgical procedure itself, the total 18 time of the search method being below 10 sec can allow an in-situ identification of a reference point on an ophthalmic layer as an introductory phase of the surgery. If the search method can be performed below 1 sec, then it can be performed repeatedly during the ophthalmic procedure, providing feedback about its progress. If the search method can be performed below 100 msec, then a limited resolution near-real time imaging of a reference point may be possible, and if the method can be performed below 10 msec, then a good resolution near-real time imaging of a reference point may be possible.

[00102] FIG. 6 illustrates yet another embodiment 100’ of the search method, possibly in a surgical context, including: performing a local search of a merit function of an imaged region in the proximity of a first location in step 110’; determining a preferred direction based on the local search in step 120’; and selecting a second location based on the preferred direction in step 130.

[00103] In FIG. 6, the local search in step 110’ can include imaging by one of an optical coherence tomography, a depth measurement technology, an optical sensing technology and a sensing technology of the merit function.

[00104] The merit function can be any spatial coordinate, a height, a depth, a surface z coordinate, any optical property, such as a scattering strength, a reflectivity, a transmittivity, an absorptivity, any of the above properties in any portion of the electromagnetic spectrum, such as in the infrared or ultraviolet portion, and any mechanical property. In other implementations, the merit function can be a combination of sensed quantities, e.g. the product or weighted sum of an optical reflectivity and an ultraviolet scattering strength. A large number of composite merit functions are possible, combining various performance parameters.

[00105] Any of these merit functions can be imaged if the merit function exhibits a detectable variation across the target layer. An example is the optical scattering strength exhibiting a rapid increase when the cataract region is reached during the scan of the nucleus.

[00106] The local search step 110’ can include searching in two or three dimensions in the proximity of the first location. Correspondingly, the merit function can be determined along a closed loop, a loop in a plane, a loop on a predetermined surface, an ellipse, a circle, an essentially closed surface, an ellipsoid, and a sphere, as well as along local rays.

[00107] In step 120’ the preferred direction can be determined as a direction of a gradient of the merit function. 19 [00108] Analogously to method 100 in FIG. 2, the direction of the gradient in FIG. 6 can include determining at least one of a location of a maximum of the merit function on an essentially closed surface or loop around the first location, and a location of a minimum of the merit function on the essentially closed surface or loop around the first location, followed by determining the direction of the gradient by relating any two of the first location, the location of the maximum and the location of the minimum. Any of the above methods can be applied to determine the gradient, including the various averaging methods.

[00109] Once the direction and magnitude of the gradient is determined, in step 130’ the selecting a second location step can include selecting the second location by shifting the first location by an increment vector, where a direction of the increment vector can be essentially parallel to the direction of the gradient.

[00110] After the steps 110’-130’ have been repeatedly performed, the search method 100’ can be stopped when the magnitude of the gradient becomes smaller than an iteration-stop value, since this signals that the method 100’ reached the immediate vicinity of the sought extremum.

[00111] Since the method 100’ performs the searches only locally, it does not involve determining the merit function of the imaged region in a volume, along parallel lines, on aligned surfaces, and in a two or three dimensional raster or grid.

[00112] FIG. 7 illustrates an Integrated Searching-Imaging (IS I) system 200. The ISI system 200 is capable of performing the methods 100 or 100’. The ISI system 200 can include a scanner 210 and an image acquisition and pre-processing unit 220, both being part of the imaging system 230. The scanner 210 is configured to scan a laser imaging beam in an imaged region. The image acquisition and pre-processing unit 220 can receive and pre-process a returned imaging beam. In implementations, the imaging system 230 can be configured in various forms, e.g., an optical coherence tomography system, a depth measurement system, or a sensing system of a merit function.

[00113] The imaging system 230 is capable of determining a coordinate of a feature of an object or a boundary of a region. The imaging process can be based on sensing any optical or mechanical property exhibiting a detectable change at the boundary.

[00114] A controller 240 is provided to control the scanner 210 and the image acquisition and pre-processing system 220. The communication between these subsystems can be bi-directional. The controller 240 can direct the imaging system 230: 20 (i) to determine the coordinate in a loop in the proximity of a first location; (ii) to determine a direction of a gradient of the coordinate corresponding to the first location; and (iii) to select a second location based on the determined direction.

[00115] The controller 240 can be configured to determine at least one of a location of a maximum of the coordinate along the loop and a location of a minimum of the coordinate along the loop, and to determine the direction of the gradient by relating any two of the first location, the location of the maximum and the location of the minimum.

[00116] The controller 240 can further select the second location by shifting the first location by an increment vector, where the direction of the increment vector can be essentially parallel to the direction of the gradient.

[00117] The controller 240 can be also configured to stop the search when a magnitude of the gradient becomes smaller than an iteration-stop value.

[00118] In ophthalmic implementations, the imaging system 230 can be an ophthalmic coherence tomography system; the imaged boundary of a region can be an ophthalmic layer of a cornea, a lens or a cataract, and the coordinate can be a depth of the ophthalmic layer. The scanner 210 can move a focal spot of an imaging beam along a loop in a surface, such as an (x,y) plane. The image acquisition system 220 can sense the returned portion of the imaging beam e.g., by a sensor or array of sensors to determine a coordinate of the imaged boundary, such as its z depth. In these implementations the controller 240 can be configured to determine an extremum of the ophthalmic layer.

[00119] In such ophthalmic applications the IS I system 200 can determine the extremum of the ophthalmic layer faster than one of 10 msec, 100 msec, 1 sec and 10 sec. Each of these values is associated with a unique functionality, as described earlier.

[00120] As repeatedly stated earlier, while some aspects of the implementations of the methods 100 and 100’ were described in relation to surgical procedures, these methods are equally applicable in any situation when the imaging of an entire surface or boundary is not needed, only the identification of their extrema or boundaries or some other features. As such, the methods 100 and 100’ can be applied for any material machining, diagnostic, quality control, and manufacturing applications.

[00121] While this document contains many specifics, these should not be construed as limitations on the scope of an invention or of what may be claimed, but rather as descriptions 21 of features specific to particular embodiments of the invention. Certain features that are described in this document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately 5 or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or a variation of a subcombination. 2015203781 07M2015 [00122] A number of implementations of image-guided laser surgical techniques, 10 apparatus and systems have been disclosed. However, variations and enhancements of the described implementations and other implementations can be made based on what has been described. 22

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:

1. A computer-implemented method for searching surgical imagery data, based on sensing or measuring a function F(x) at selected locations of an imaged object comprising a surgical ophthalmic target, to search for the extrema of an ophthalmic layer of the imaged object without performing full-field sensing or measuring over the entire field of the target, wherein the method is performed in an integrated searching and imaging system comprising an imaging system and a controller, the method comprising the steps of: (i) selecting a first selected location (xl,yl) of the imaged object; (ii) selecting a first loop centered around the first selected location, the loop having a substantially elliptical or circular form; (iii) sensing or measuring values of the function with an imaging system at the set of locations along the first loop centred around the first selected location; (iv) determining, with a controller, a second location along the first loop where F(x) assumes a maximum or minimum value, and (v) determining the direction of a first gradient vector by considering any two of the first selected location (xl,yl), and the second location where F(v) has a maximum or minimum value, the first gradient vector being based on the values of the F(-'c) function at the determined second location along the first loop and the first selected location (xl,yl); (vi) updating the first selected location (xl,yl) along the first gradient to a second selected location (x2,y2); and repeatedly updating the first selected location by applying the steps (ii)-(vi) until the location of the maximum Fniax (xms) or the minimum Fnini (xnini) of the function F(x) is found.
2. A search system, comprising: —T* an imaging system that, based on sensing or measuring a function Ffv) at selected locations of an imaged object comprising a surgical ophthalmic target, searches for the extrema of an ophthalmic layer of the imaged object without performing full-field sensing or measuring over the entire field of the target, and determines a coordinate of a feature of an object on a loop in a proximity of a first location; via a controller, coupled to the imaging system, the controller adapted to: (i) select a first selected location (xl,yl) of the imaged object; (ii) select a first loop centered around the first selected location, the loop having a substantially elliptical or circular form; (iii) sense or measure values of the function with an imaging system at the set of locations along the first loop centred around the first selected location; (iv) determine, with a controller, a second location along the first loop where F(%) assumes a maximum or minimum value, and (v) determine the direction of a first gradient vector by considering any two of the first selected location (xl,yl), and the second location where F(v) has a maximum or minimum value, the first gradient vector being based on the values of the F (x) function at the determined second location along the first loop and the first selected location (xl,yl); (vi) updating the first selected location (xl,yl) along the first gradient to a second selected location (x2,y2); and repeatedly updating the first selected location by applying the steps (ii)-(vi) until the location of the maximum Fniax (:?niax) or the minimum Fnini (xlllin) of the function F(x) is found.
3. The search method of claim 1 or the search system of claim 2, wherein: the function F(;c) is one of a height, a depth, a z coordinate, and a coordinate along a reference line.
4. The search method of claim 1 or the search system of claim 2, wherein: the first location is one of a location in a plane transverse to an axis, a location in a plane transverse to the determined coordinate, a location in an (x, y) plane, a location in a radial coordinate system, and a coordinate on a predetermined surface.
5. The search method of claim 1 or the search system of claim 2, wherein: the function F(x) is determined by one of an optical coherence tomography, a depth measurement technology, an optical measurement, and a sensing technology of a merit function.
6. The search method of claim 3 or the search system of claim 3, the determining a first gradient vector step comprising: determining a direction of the first gradient vector as a vector connecting the first location with the determined location along the first loop where F(i) assumes a maximum or minimum value; and determining a magnitude of the first gradient vector from a difference of the values of the F(5) function at the determined location along the first loop and at the first location.
7. The search method of claim 1 or the search system of claim 2, the selecting a second location step comprising: selecting the second location (X2, y2) by shifting the first location by an increment vector, a direction of the increment vector being approximately parallel to the direction of the first gradient vector.
8. The search method of claim 1 or the search system of claim 2, the selecting a first loop around the first location step comprising: identifying rays (ri 1, r\2, ... rin) emanating from the first location; and determining the function F(x) at ray-points.
9. The search method of claim 8 or the search system of claim 8, the determining a direction of a gradient vector step comprising: determining a rate of change of the function F(x) along the rays; selecting the ray along which the rate of change is maximum; and identifying a direction of the selected ray as the direction of the gradient vector, and the rate of change along the selected ray as the magnitude of the gradient vector.
10. The search method of claim 1 or the search system of claim 2, comprising: shifting the first location by a shake-up vector having at least one of a direction making a non-zero angle with the direction of the gradient vector; or a magnitude substantially different from the magnitude of the gradient vector.
11. The search method of claim 1 or the search system of claim 2, wherein: the ophthalmic layer is of one of a cornea, a lens, and a cataract; and the function F(x) is a depth of the ophthalmic layer.
12. The search system of claim 2, the imaging system comprising: a scanner that scans an imaging beam; and an image acquisition sub-system that receives and pre-processes a returned imaging beam.
13. The search system of claim 2, wherein: the imaging system is one of an optical coherence tomography system, a depth measurement system, an optical sensing system, and a sensing system of a merit function.
14. The search system of claim 2, wherein: the controller determines at least one of a location of a maximum of the coordinate along the loop and a location of a minimum of the coordinate along the loop; and determines the direction of the gradient by relating any two of the first location, the location of the maximum and the location of the minimum.
15. The search system of claim 2, wherein: the controller selects the second location by shifting the first location by an increment vector, a direction of the increment vector being substantially parallel to the direction of the gradient.
16. The search system of claim 2, wherein: the controller retains the first location when a magnitude of the gradient is smaller than an iteration-stop value.
17. The search system of claim 2, wherein: the imaging system is an ophthalmic coherence tomography system; the imaged object is an ophthalmic layer of one of a cornea, a lens, and a cataract; the coordinate is a depth of the ophthalmic layer; and the controller determines an extremum of the depth of the ophthalmic layer.