EP1929237A2

EP1929237A2 - Stereoscopic imaging through scattering media

Info

Publication number: EP1929237A2
Application number: EP06802527A
Authority: EP
Inventors: David Abookasis; Joseph Rosen
Original assignee: Individual
Current assignee: Individual
Priority date: 2005-08-26
Filing date: 2006-08-25
Publication date: 2008-06-11
Also published as: WO2007025278A2; WO2007025278A3

Abstract

A method for three-dimensional imaging of hidden objects in a scattering medium. Objects hidden between two biological tissues (T1 and T2) at different depths from the viewing system are recovered, and their three dimensional locations are computed. Analogous to a fly's two eyes, two microlens arrays (LL and LR) are used to observe the hidden objects from different perspectives. At the output of each lens array, constructed are objects from several sets of many speckled images by a technique previously suggested which uses a reference point. The differences of the reconstructed images on both arrays in respect to the reference point yield the information regarding the relative depth between the various objects.

Description

STEREOSCOPIC IMAGING THROUGH SCATTERING MEDIA

This application claims priority to U.S. Provisional Patent Application Serial No. 60/711,548, filed August 26, 2005. This Application is a continuation-in-part of Application No. 11/226,572, which claims priority to 60/609,643.

TECHNICAL FIELD

The present invention relates in general to imaging systems, and in particular, to a system for imaging a hidden object.

BACKGROUND INFORMATION

Medical tomography techniques such as X-ray Computed Tomography (CT) {see J. G. Webster, Minimally Invasive Medical Technology, (IoP, Bristol and Philadelphia, 2001), Chap. 4, pp. 46-58) offer great advantages and are still widely used despite the fact that they suffer from several drawbacks such as ionizing radiation, a complex structure and high-cost. An advantage of optical tomography over other medical tomography techniques is that it provides quantitative information on functional properties of tissues, while being non-harmful (the radiation is non-ionizing). Accordingly, in recent years, researchers have invested considerable efforts towards developing optical tomography systems that use near-infrared (NIR) light. A well-known system is the optical coherence tomography (OCT) {see A. F. Fercheri, W. Drexler, C. K. Hitzenberger and, T. Lasser, Optical coherence tomography- principles and applications," Rep. Prog. Phys. 66, 239-303 (2003)). OCT synthesizes cross-sectional images from a series of laterally adjacent depth-scans with high depth and transversal resolution. However, OCT involves a cumbersome interferometric process with complicated reconstruction algorithms to generate accurate cross-section images of various body parts.

In the NOISE system described in U.S. Patent Application Serial No. 11/226,572, which is hereby incorporated by reference herein, a hidden object is reconstructed from many speckled images formed by a microlens array (MLA). Each microlens from the array projects a small different speckled image of the hidden object onto a CCD camera. In the reconstruction algorithm, all the noisy images from the array are shifted to a common center and then accumulated to a single average picture thereby revealing the shape of the hidden object. In NOISE-2, also described in Serial No. 11/226,572, a different algorithm is implemented on the same optical system, alleviating a need to shift the speckled images to a common center. In addition to recording the speckled images of the object, recorded are speckled images of a point- like object generated by illumination through the medium with a point-source. Each sub-image of the speckled object with a corresponding sub-image of the speckled point-like object is placed together side by side in the computer. Then, by computing the autocorrelation of each joint sub- image, and averaging over all the autocorrelations, a cross-correlation between the object function and a narrow point-like function is reveled.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

Fig. 1 illustrates a system configured in accordance with an embodiment of the present invention;

Fig. 2 illustrates a perspective projection geometry of an object and a reference point through each channel;

Fig. 3 illustrates a summary of imaging results;

Fig. 4 illustrates an alternative embodiment of the present invention; and

Fig. 5 illustrates a block diagram of the computer in Fig. 1.

DETAILED DESCRIPTION

The present invention discloses an optical tomography technique that is based on speckled images. Embodiments of the present invention are an extension toward three dimensional (3D) imaging of the embodiments disclosed in Serial No. 11/226,572, referred to as Noninvasive Optical Imaging by Speckle E Ensemble (NOISE) (see J. Rosen and D. Abookasis, "Seeing through biological tissues using the fly eye principle," Opt. Exp. 11, 3605 - 3611 (2003); J. Rosen and D. Abookasis, "Noninvasive optical imaging by speckle ensemble," Opt. Lett, 29, 253-255 (2004)) and its modified version NOISE-2 (see D. Abookasis and J. Rosen, "NOISE 2 imaging system: seeing through scattering tissue with a reference point," Opt. Lett, 29, 956-958 (2004)). Therefore, embodiments of the present invention may be referred to as NOISE-3D.

In addition to reconstructing an object, its location in the 3D space is also extracted. Therefore, embodiments of the present invention reveal and acquire depth information of objects seen through a scattering medium. This technique differs from recently proposed depth extraction systems using lens array {see Y. Frauel and B. Javidi, "Digital three-dimensional image correlation by use of computer-reconstructed integral imaging," Appl. Opt, 41, 5488-5496 (2002); J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, "Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification," Appl. Opt, 43, 4882- 4895 (2004)) since it observes scenes hidden beneath a scattering media and uses a different algorithm for depth acquiring.

Fig. 1 is a schematic diagram of a 3D imaging system in accordance with embodiments of the present invention. The configuration consists of two MLAs accompanied by imaging lenses, a pinhole (implemented by an adjustable iris) placed behind the second scattering layer T₂ and conventional CCD cameras. Each path, left and right separately, is similar to that described in D. Abookasis and J. Rosen, "NOISE 2 imaging system: seeing through scattering tissue with a reference point," Opt. Lett, 29, 956-958 (2004). However, it should be noted that unlike NOISE-2, in the present invention, the point-source is placed in front of the scattering medium, and thus serves as a reference point instead of as a point-source of illumination. This improves the reconstructed image because now light from the pinhole is no longer scattered. Therefore, the pinhole's image is narrower, and so is its cross-correlation with the object's image. The idea behind this point technique is to ascribe the location of an object to a location of some known point in space.

The computational process at each channel is schematically described in the lower portion of Fig. 1. The computational process yields three spatially-separated terms at the output of each path. One term is the zero-order at the vicinity of the output plane origin. This term is equal to the sum of the pinhole autocorrelation and the object autocorrelation. The other two terms correspond to the cross-correlation between the object and the pinhole and thus, assuming the average pinhole image is close to a point, these terms approximately yield the object reconstruction. The image of the hidden object can therefore be retrieved by reading it from one of these orders. Note that in this scheme, the distance of the reconstructed object from the output plane origin is related directly to the transverse gap between the object and the reference point. To extract the depth information about the object, initially used is the principle of stereoscopic vision, ignoring, for the time being, the effect of the scattering medium T₂. In the diagram depicted in Fig. 2, the projection of the object and the point reference on the image planes of both arrays can be presented as follows:

X_R∞SΘ _„ B_R (x_R - d Jcosfl B_R - h

- = — , = (Ia)

J ^zp J Z₀

for the right channel, and

*_£cosfl „ Ih \x_L - d_L)cosθ ^ BL ^{+ h} (ib)

J Zp J Z₀

for the left channel. z₀ and z_p are the distances of the object and the pinhole from the center of the array, respectively. B_R and B_L represent the distances of each array from the z axis, d^ and JL are the gaps between the images of the reference point and the observed object, in the right and left channels, respectively; XR and *L are the displacements of the reference point images from the horizontal axis which crosses the center of each microlens, in the right and left channels, respectively, h is the transverse gap between the object and the pinhole, and θ is the tilt angle of each array. Solving the four equations of Eqs. (1) yields the distance between each array plane to the object, given by

B - f ^• z

Zo = ^■ p

B - f - z_D - D - cosθ (2)

where B=BR+BL is the baseline and D=d_R+d_L is the sum of the object-reference point gaps at the two channels. It is clear from Eq. (2) that different perspectives lead to a slight displacement of the object in the two views of the scene. Equation (2) has an advantage that one needs only to measure D, which is related to the sum of the distances of the reconstructed object from the output plane origin at the two channels. Accordingly, achieved is a simple method for obtaining the object's depth, since in the configuration the baseline B, the focal length/ and the pinhole distance from the MLA z_p are known parameters. Calculating D, as described below, estimated is the depth information Z₀ of an object from the MLAs in an imaged scene. Note that by using the pinhole there is no need to know the object and the reference point locations, which are difficult to estimate in such a noisy scattering system. This is an additional advantage for using a method with a reference point.

Since objects are covered under the layer T₂ with higher-than-one index of refraction, while the reference point is positioned in front of T₂, the obtained gap between each object and the reference point in each channel is different than the situation without the layer T₂. Considering the Snell law, and the fact that the layer T₂ is tilted relatively to the optical axis of each array, the corrected displacement D from the measured displacement D is

where D = JR ₊ d^ is the sum of the object-reference point gaps at the two channels measured after introducing the layer T_2; t and n are the thickness and the refractive index of the scattering layer, respectively, and ø is the angle between the z axis and the ray connecting the centers of the object and the MLA. The second term in Eq. (3) describes the size change of the image as a consequence of the layer T₂. Since ø and z₀ are unknown before calculating Eq. (1), ø and Z₀ are initially approximated as θ and z_p, respectively, in order to yield an initial estimation for D.

The next step is to extract the information about the displacement D from the reconstructed image. Referring to the flowchart in Fig. 1, for each path, each sub-image of the speckled point reference with a corresponding sub-image of the object (recorded by a different exposure where the iris is opened) is Fourier transformed and the square magnitude of each sub- spectrum is accumulated with all the other sub-spectrums. This average joint power spectrum is then Fourier transformed, which yields the output correlation plane. This plane in turn yields a symmetric image reconstruction around the plane origin. By taking one of the first orders, the number of pixels are counted that range from this order to the plane origin. The distance in pixels is converted to a real distance at the arrays' image plane by subtracting the sub-matrix width, by multiplying the pixel number with the size of the CCD pixel and by dividing it by the magnification of the imaging lenses L_R and LL. These operations are implemented on each path in order to estimate D. By calculating D using Eq. (3) the object depth can now be calculated by using Eq. (2). Figure 4 illustrates an alternative embodiment of the present invention where a set-up of a single laser, a single camera, and a single MLA are utilized to visualize a hidden object between two layers T₁ and T₂ using algorithms as described above with respect to Figure 1. In this alternative embodiment, either the hidden object and layers T₁ and T₂ are rotated relative to the camera, MLA, or laser, or the camera, MLA and laser are rotated around the hidden object. A benefit of this arrangement is that only a single laser, a single camera, and a single MLA are required. Thus with this alternative embodiment, the system can rotate around the sample and take pictures of the sample at different angles theta. In Figure 2, theta (θ) is indicated as the angle between a vertical line and the MLA. Because of the geometry, this angle is equal to the angle between the upper arm and a horizontal arm. The system in Figure 2 takes pictures using two sets of cameras, MLA, and laser, one oriented at θ degrees with the optical axis of the sample and one oriented at -θ degrees with respect to the optical axis of the sample. However, Figure 4 shows that it is possible to just utilize one setup that could be rotated around the sample and takes pictures and does reconstruction using the algorithms described herein for different angles of θ. For example, one embodiment would be to position the arm at θ degrees with respect to the optical axis of the sample, take a picture, and then rotate the arm, i.e. both the laser part of the arm (rail 1) and the pinhole, MLA, lens, camera part of the arm (rail 2) to an angle of -θ and then take another picture. However, real-time imaging is not possible as the whole arm has to rotate over two θ and take another picture. The information of both pictures can then be used in order to calculate the longitudinal distance between two imaged objects.

Experiments with two separated cylindrical sticks as observed objects were carried out using the configuration shown in Fig. 1. The sticks had a length of 20 mm and a diameter of 2.1 mm each. During the experiments, the left-stick was constantly attached to tissue T₁ while the right stick was moved longitudinally toward the MLAs, at three different positions. Thus, the relative longitudinal displacements between the sticks were: 0 mm (the objects in the same plane), 2 mm and 4 mm. The sticks were embedded between two slabs of chicken breast separated by a distance of 12 mm. This scattering medium is characterized by a scattering coefficient of μ_s =12SiP(Cm^'1) (see J. Rosen and D. Abookasis, "Noninvasive optical imaging by speckle ensemble," Opt. Lett, 29, 253-255 (2004)) and an absorption coefficient of μ_a~0.2(cm^"1) (see G. Marquez, L. V. Wang, S-P Lin, J. A. Schwartz, and S. L. Thomsen, "Anisotropy in the absorption and scattering spectra of chicken breast tissue," Appl. Opt, 37, 798-804 (1998)). The thicknesses of the rear tissues T₁ and the front tissue T₂ were about 3 mm and 4.5 mm, respectively. The reference point was created by placing a pinhole with an adjustable aperture at a short distance (22 mm) behind tissue T₂, between T₂ and the MLAs. The rear tissue Ti was illuminated by two diagonal collimated plane waves emerging from of He-Ne laser at 632.8 ran with 35 mw. The two MLAs (source: AdaptiveOptics, 0600-6.3-S) were placed at a distance of Z_p=162 mm from the pinhole. Each MLA comprises 42x42 micro-lenses, but 3x8 were used in this experiment. Using more than 3 columns per channel introduces a considerable different perspective of the object into the averaged image, and thus the reconstructed image is degraded. The diameter of each micro-lens is 0.6 mm and its focal length 6.3 mm. The image plane of each MLA is projected onto the CCD (PCO, PixelFly, with 1024x1280 active pixels and pixel size of 6x8x6.8 μm) plane by a single spherical lens LL and LR respectively, each with a focal length of 120 mm and a diameter of 150 mm. These lenses, with magnification of 1.3, matches the MLA size with the CCD size and are sufficiently large to cover the MLAs. At each channel the distance from the MLA to the imaging lens is 210 mm and the distance from the MLA to the CCD plane is 280 mm. The baseline B is 80 mm. After acquiring the sets of the observed image in each path, a computer program was employed to reconstruct the sticks and to determine their distance from the MLAs according to Eqs. (2) and (3).

A summary of the results is displayed in Fig. 3. Columns (a) and (d) show typical sub- images obtained from a typical microlens without using the averaging process. The reconstructed images derived from the averaging process on the images of the hidden sticks with different relative displacements between the objects are shown in column (b) for the left channel, and (e) for the right channel. Note that the reconstruction of one of the sticks (the right- one from Fig. 1) in the pictures is improved as a consequence of its closeness to the scattering layer T₂, while the other stick (the left-one from Fig. 1) remains far from T₂. Columns (c) and (f) show the same reconstructed images obtained by removing the second tissue T₂ on the same setup. The effect of stereoscopic vision is clearly demonstrated in these figures by observing that in the right path the relative distance between the sticks gets smaller while in the left path the distance grows as a consequence of moving the right stick longitudinally towards the MLAs.

Measuring corresponding distances in different figures can succeed when there are well- seen indicators on the objects that are viewed from the two channels. In the almost vertical sticks, one can refer to the central point of each stick as the object point for the distance measurements. Using these measurements, and applying Eq. (3) and Eq. (2), results indicate that the relative distances between the sticks without taking T₂ into account are 0.302 mm, 2.414 mm and 4.397 mm instead real distances of 0 mm, 2 mm and 4 mm while when taking T₂ into account the relative distances between the sticks are 0.24 mm, 2.358 mm and 4.548 mm. The average error in measuring z₀ is less than 0.1%.

Thus, disclosed is an optical tomography system which enables observation of the relative gap between objects hidden in a scattering medium. The system has been verified experimentally with a low error of deviation. The system operates with low computational complexity and is robust to the object depth variations.

Claims

WHAT IS CLAIMED IS:

1. A system for imaging an object hidden by a first scattering layer, comprising: a first coherent light source for emitting a plane wave; a first microlens array positioned to receive scattered light produced from the plane wave as it illuminates the object hidden by the first scattering layer, the first microlens array producing speckled images from the received scattered light; a first camera for collecting and storing the speckled images; and

A positioning system for enabling multiple speckled images to be collected at different angles around the hidden object.

2. The system as recited in claim 1, wherein the object is hidden between the first scattering layer and a second scattering layer, wherein the plane wave is directed through both scattering layers towards the microlens array.

3. The system as recited in claim 1, further comprising a focusing lens positioned between the first microlens array and the first camera.

4. The system as recited in claim 1, wherein the first camera comprises a charge-coupled device.

5. The system as recited in claim 1, further comprising a computer coupled to the first camera for displaying an image of the object recreated from the speckled images collected by the first camera.

6. The system as recited in claim 5, wherein the recreated image of the object is reconstructed by shifting the speckled images to a common center and accumulating the shifted speckled images into an average image.

7. The system as recited in claim 5, wherein the recreated image of the object is reconstructed by an algorithm operated as a computer program in the computer comprising the program steps of:

(a) Fourier transforming each of the speckled images jointly with an image of a speckled pointlike light source;

(b) accumulating a set of squared magnitude images created by step (a) to form an averaged image; and

(c) Fourier transforming the averaged image.

8. The system as recited in claim 1, further comprising a second coherent light source, a second microlens array, and a second camera all positioned at a location relative to the hidden object that is different from the first coherent light source, the first microlens array, and the first camera.