NL2010883C2

NL2010883C2 - Two-dimensional imaging system, device comprising an imaging system, and method for calculating a parameter for a two-dimensional range.

Info

Publication number: NL2010883C2
Application number: NL2010883A
Authority: NL
Inventors: Richard Johannes Cornelis Meester
Original assignee: Quest Photonic Devices B V
Priority date: 2013-05-29
Filing date: 2013-05-29
Publication date: 2014-12-08

Description

Two-dimensional imaging system, device comprising an imaging system, and method for calculating a parameter for a two-dimensional range.

Field of the invention

[0001] The invention relates to an imaging system. The invention further relates to a system and method for creating a two-dimensional (2D) image of an object or sample and associating spectral data with each point of the 2D image. The invention also relates to a device for tissue analysis, a device for fluorescent imaging, and a device for agricultural assessment. The invention further relates to a laparoscope. The invention also relates to a method for calculating a parameter for a 2D range.

Background of the invention

[0002] . Spectrometers have been around in various different shapes and forms to measure spectral responses from organic or inorganic samples, either through grating and lens devices as well as dispersion prism based devices. These solutions typically require a predefined light source to illuminate the sample and take either a point or line sample of the object under test.

[0003] A common drawback is related to the fact that the spectral samples are only point samples (spectrometer based on a grating) or a line (spectrometer based on a dispersion prism with 2D sensor placed behind it). When a real-time 2D (spatial) + spectral image of a sample is required, this point or line sampling is problematic. According to the prior art, one might use a push broom scan of an object with a line scan dispersion based spectrometer. Real care must be taken in stitching the scanned images together at the right position after scanning the object. Furthermore in these scanning applications, sample motion presents severe problems in accurately creating a 2D image.

[0004] Currently new sensors are starting to emerge on the market that can simultaneously sample multiple lines of spectral wavelengths. This is achieved by putting spectral line filters on top of the pixels of the sensor. This basically means that the filters are very narrow and sample only the wavelength that the filter lets through. Since the coatings are placed on the sensor after the lens, the filters only filter out on a line or pixel basis of the projection of the original image. This means that a physical part of the object of interest is sampled at one wavelength, and that a different portion of the object is sampled at another wavelength. These sensors thus do not sample the same line. In order to have co-registered images and take decisions based on spectral sampled data, it is very advantageous that one samples exactly the same physical location and get the complete spectral response (e.g. 400 to 1000 nm) in relatively good precision (for example, in 1 to 3 nm precision) of that pixel and not a spectral wavelength A of location one and wavelength B of a different location.

[0005] It is known from previous patents like US 5 772 589 A that by sampling certain wavelengths oxygenation and heart rate can be analysed. Modern oxygen saturation and tissue perfusion devices are all based on the principle that at certain wavelengths one can measure these variables. However, the known devices either use contact based point measurements (i.e. based on a fiber that is placed on a sticking plaster or clip which sends out light of a certain wavelength and this is measured with a spectrometer to get a good reading and accurate response) or based on three particular wavelengths for measuring oxygenation specifically, as in WO 01/15597 A1. In WO 01/15597, three 2D images are captured, each a at a specific narrow wavelength range. By combining the three 2D images an oxygenation image can be achieved. This device however has significant accuracy drawbacks because it is based, in the spectral domain, solely on a three point measurement (wavelengths 670, 810 and 940 nanometers), and thus uses implicit assumptions concerning which substances emit at these wavelengths.

[0006] In typical oxygenation or other parameter measurement applications, the actual signal with information is about 2% of the total measurement response. The rest of the response is caused by unknown substance (i.e. tissue, bone, fat, etc). Because in the described prior art approaches insufficient spectral data is available, it is generally impossible to determine which part of the spectral signal is due to bone, fat, skin, or any other substances because the spectral measured response is based on the total signal (for the wavelength range of interest) being measured.

[0007] Several imaging solutions are known from the prior art. US 2008 / 267 472 A1 discloses a system for splitting incoming light towards various sensors, using filters to select different wavelength ranges for the various sensors (e.g. visible light, near infrared, etc). However, the use of filters selects a relatively broad wavelength range and does not allow precision spectral measurements.

[0008] WO 2008 / 103 918 A1 discloses an imaging system wherein a micro-lens array is used to effectively divide the object area of a sample to a number of light points. This greatly reduces the spatial resolution of the measurements as well as spectral resolution and these are related. A higher spectral resolution can be obtained at the cost of spatial resolution and vice versa.

[0009] WO 2007 / 064 329 A1 discloses a system with a beam splitter and two CCD cameras. Pulsed illumination (10 microsecond pulses) at predetermined wavelengths is used to obtain measurements. The use of pulsed illumination complicates the design and use of the system.

[0010] US 2010 / 145 416 A1 discloses another imaging system with two light paths. Again, filters (e.g. in filter wheels) are used to select wavelength ranges for measurements.

[0011] There is a need for an improved 2D imaging solution that can make use of the additional information provided by real-time spectral data.

Summary of the invention

[0012] The invention provides an imaging system comprising - a beam splitting unit for splitting a beam of incident light into at least a first channel and a second channel; - a 2D image sampling unit arranged to receive light from the first channel and to sample a 2D image; and - a spectral sampling unit arranged to receive light from the second channel and to sample a spectral response for each point along a calibrated (straight) line in the 2D image.

[0013] The invention thus provides a 2D imaging system, suitable for real-time use with non-static samples. The system is based on beam splitting the light from a sample into at least two beams. Examples of beam splitters are dichroic prisms. A 2D image is generated by a 2D image sampling unit from the light along the first path (the terms channel and path may be used interchangeably), while a series of spectral images (that is, wavelength- or frequency-dependent images) is generated by a spectral sampling unit from the light along a second path. To this end, the spectral sampling unit may contain a slit and lens unit to restrict the light to a one dimensional sampling line. The spectral sampling unit can further comprise a means for spatially separating light depending on wavelength or frequency, such as a dispersion prism, and a 2D sensor, configured so that e.g. each point on the sampling line corresponds to a row in the 2D sensor array and each wavelength corresponds to a column in the 2D sensor array, or vice versa.

[0014] Both paths are spatially calibrated. In other words, a spatial sample location in one sampled image can be related to a spatial sample location in another sampled image. More in particular, the line (hereafter also “scan line”) along which spectral responses are measured has a known location in the 2D image, so that spectral measurements and 2D average intensity measurements from the same physical location in the sample can be combined.

[0015] Advantageously, both 2D sensors sample at least a percentage of overlapping wavelengths. Filters may be used to ensure this and/or be used to restrict certain wavelengths from reaching the 2D image sensor (i.e. placing a narrow band filter restricts the wavelengths that fall within the band to reach the sensor). This will aid calibration of the place of the scan line in the 2D image or can at least provide a way to check the calibration. In addition, the overlap can be used to extrapolate the spectral data to areas in the 2D image where the spectrum is not directly determined.

[0016] Advantages of the invention are apparent from the following considerations. When trying to reconstruct complete spectral information about the spectral response of a pixel location, in principle it is best is to measure the complete spectrum. This however is impractical to do for all pixels in a 2D image plane in real-time. The inventor has realized that it is usually enough to have this spectral information in only a subset of the spatial locations. Therefore, according the invention, it is possible to determine one or more sets of average intensities (as integrated over a predefined, possibly narrow, wavelength range) for the full 2D range (using the first channel) and spectral (that is, with detailed wavelength-dependent information over a relatively broad wavelength range) intensities for one or more lines in the 2D range. Full spectral information is thus only available at the locations of the one or more lines, but this information can be extrapolated to other parts of the 2D range.

[0017] In this known environment and with the direct reference between pixels out of the 2D image and an aligned spectral plot of a pixel, the calibration of the 2D image data using the spectral data can be achieved in real-time continuously and the accuracy of the values based on the 2D image data can be improved significantly.

[0018] One major drawback in the device described in WO 01/15595 is that it cannot be reliably used when chemicals are injected in the blood stream, like i.e. ICG (and possible other substances). Moreover, it does not reliably work in conjunction with fluorescent image guided surgery techniques where excitation sources of different wavelengths are present and where chemicals in the bloodstream are interfering with the read signal. Using an imaging system according the invention, one is able to clearly identify such chemicals and excitation sources because the full spectral range can be sampled and one can calibrate, normalize, and generally compensate for additional contributions to the measured spectrum from for example injected chemicals or fluorescence agents.

[0019] This invention thus overcomes the drawbacks associated with WO 01 / 15597 as discussed as part of the background, by having the full spectral plot of a certain pixel available which allows to perform an accurate spectral measurement. By determining for example the spectral response of tissue at a particular location it is possible, with the use of curve fitting available in spectrometry, to determine the exact amounts of hemoglobin or oxyhemoglobin contributing to the resulting spectral image. By curve fitting of the spectral measurements, an accurate determination of a spectral parameter of interest can be made.

[0020] By using a channel for generating a 2D image of for example light at 810 nm or 670 nm (or any other wavelength of interest) and another channel for determining spectra plots along a 1D line in the 2D spatial range of the 2D image, one can relate the pixel spectral plot measurement to the average 2D measurements, and overlay the 2D measurements with the parameter as determined from the spectral plots. By using all the pixel spectral plots as references one can relate and calibrate all the remaining 2D image pixels that lie outside the spectral scan line. A conversion map from pixel intensity in the 2D image can be created based on spectral pixel data from the spectral scan line and their equivalent intensity response in the 2D image at exactly the same position. After this calibration step, the parameters can be determined for all points outside the scan line locations as well, and the parameter data can be added to the 2D overlay. As such, a 2D overlay comprising directly measured parameters and extrapolated (calibrated) parameters can be determined and shown.

[0021] In clinical applications, the invention thus provides a more accurate 2D oxygenation image, as well as accurate readings of lung function or heart rate and can be used to assess the tissue perfusion, burn wound penetration, etc. by looking at the spectral difference between healthy known recordings and resulting spectral measurements.

[0022] At the location of the scan line the most accurate values are given, and outside the scan line an accurate assumption of the same pixel response can be given. A method according the invention thus provides the ability to link the spectral response of one or more than one pixel together on a line or a different configuration, to one or more selected wavelengths of 2D images, allowing one to overlay or extrapolate spectral response data in the 2D sampling region.

[0023] In an embodiment according the invention, the imaging system comprises a slit for creating an essentially one-dimensional light pattern as input for the spectral sampling unit.

[0024] In an embodiment according the invention, the imaging system comprises a filter in the first and or the second path for selecting a range of wavelengths.

[0025] In an embodiment according the invention, the selected wavelengths of the first path form a subset of the selected wavelengths of the second path .

[0026] In an embodiment according the invention, the beam splitting unit is a dichroic prism assembly .

[0027] In an embodiment according the invention, the imaging system comprises N 2D image sampling units and M spectral sampling units, wherein the beam splitting unit is configured to split the beam in at least N+M paths so that each image sampling unit and each spectral sampling unit receives light from a respective path of the beam splitting unit, N and M are integer number, and the sum of N and M is two, three, or more. In other words, an imaging system can be made which allows N 2D images to be sampled (for example, corresponding to N unique wavelength ranges) and M sets of spectral images (for example, corresponding to M unique paths in the 2D spatial range of the 2D images).

[0028] In an embodiment according the invention, the location of the at least one scan line is calibrated with respect to the at least one output 2D image.

[0029] In an embodiment according the invention, the location of the scan line relative to the sample can be moved. In an embodiment, the location of the scan line relative to the 2D image can be moved. The 2D image may be fixed relative to the sample, while the scan line is moved relative to both the sample and the 2D image. This allows to take spectral measurements at various locations in the 2D image without moving the main device. In an embodiment according the invention, the location of the scan line is moved by moving a slit.

[0030] The invention further provides an assembly of an imaging system as described in this application and a light source or light engine .In an embodiment according the invention, the light source or light engine is configured to be controlled for emitting selected wavelengths. In an embodiment according the invention, the light source is a ring light.

[0031] The invention further provides a device for tissue analysis (for example for the assessment of tissue composition, oxygenation, and other measureable quantities based on spectral decomposition of the signal) comprising a imaging system as described in this application. The invention also provides a device for fluorescent imaging comprising a imaging system as described in this application. The invention further provides a device for agricultural assessment comprising an imaging system as described in this application. Using a device for agricultural assessment, plants may be monitored to determine plant stress by measuring chlorophyll concentration in a manner analogous to the manner in which hemoglobin is measured. The device for agricultural assessment may also be used to determine vegetation indices and provide the determined chlorophyll concentration and vegetation indices in a combined overlay image. The invention further provides a laparoscope comprising an imaging system as described in this application.

[0032] Because the complete spectral response is measured, one can do a spectral analysis of the substance (i.e. bone, fat, tissue, skin, blood, and other substances.) and precisely assess which part of the response belongs to which part of the substance. In the prior art, the issue of accounting for other influences (other than the parameter of interest) is addressed by using calculation tables which are empirically determined from patient test data. Furthermore drawbacks in the above-mentioned prior art devices with sticking plasters is that they are contact measurements. This is done because surrounding light can influence the measured signal and cause deviations. The sticking plaster makes sure there is no external light interfering with the signal. However during surgery a sticking plaster cannot be attached to inner (wet) organs or burn wounds. The imaging system of the invention can be used in a contactless fashion and can also be calibrated with respect to external light (which would otherwise possibly interfere with the measured signal).

[0033] Since a complete spectral response with high resolution is available if an imaging system according the invention is used, one can calibrate the system for unwanted external signals. In contrast with the prior art systems described above (specifically WO 01/15597), the spectral region being sampled typically consists of a 10 or 15 nm wavelength range, which is averaged into a single average intensity value. Since the device in this invention is doing a complete spectral measurement (albeit at a subset of all locations in the sample), one can have a sample resolution ranging from 1.5 to 3 nm. Therefore the wavelength can be sampled with relatively high precision and the contributing wavelength to the total intensity value as registered in the 2D image pixel can be identified. The spectral images thus serve to provide a detailed interpretation of the significance of the total intensity values as registered in the 2D image.

[0034] The invention further relates to a method of data processing to reduce spectral waveforms. Data processing may be used to relate pixel spectral data (optionally in real time) to a database of known spectral responses of tissue, to assess the type of tissue. The invention also provides a method for assessing cancer tumors by reading the spectral response of the tumor and distinguishing this from healthy tissue. In fluorescent imaging, the invention provides a way to distinguish the difference between normal (healthy) tissue and tissue that has a fluorescent probe attached to the (tumor) tissue. Spectral responses of targeted probes are known and by having the spectral plot for a line of pixels in a 2D range, one can detect the probe and the quantity of probe substance by mapping the spectral plot to the database of known substances and tissues.

[0035] The invention further provides a method for calculating a parameter for a two-dimensional range, the method comprising: - sampling at least one set of average intensity values for the 2D range; - sampling spectral intensity values along at least one curve in the 2D range; - calculating the parameter values along the at least one curve from the spectral sampled intensity values; and - extrapolating the parameter for the 2D range based on the sampled average intensity values for the 2D range and the calculated parameter values along the curve.

[0036] In an embodiment according the invention, the sampled spectral intensity values are separated into a plurality of spectral contributions from respective components. In an embodiment according the invention, curve fitting is used to separate the sampled spectral intensity values into the plurality of spectral contributions from the respective components.

[0037] The method allows to predict the value of a parameter (e.g. P) at pixel locations outside the locations where a spectral scan line was determined. The parameter can be accurately determined at the locations where a spectral measurement is available (that is, at the locations along the curves or scan lines). The parameter can be determined based on the “raw” spectrum as measured. Alternatively, the spectrum may be first split into various spectral contributions from various (known) substances or compositions. Curve fitting is an advantageous technique for splitting spectra into various spectral contributions.

[0038] At the scan lines and also at all other locations in the 2D region, the 2D image measurement provides an averaged intensity value. By correlating the calculated parameter with the locally measured averaged intensity value, a calibration factor (the ratio between parameter value P and the averaged intensity value) can be determined. This locally determined calibration factor can account for any peculiarities of the sample, in particular the amount of non-related background radiation at the sampled wavelengths. Using the calibration factor, the value of P can be estimated for the parts of the 2D image where only an average intensity value but no spectral data is available.

[0039] The calibration factor may vary slightly as a function of position in the 2D range. It is possible to use a weighted calibration factor, wherein the weight is inversely proportional to the distance between the location where the calibration factor is determined and the position where the extrapolated value is to be determined.

[0040] In general, any type of functional relation can be used to correlate P to one or more average intensity values, not just a proportionality relation. In particular when there is a a relatively high level of stable background radiation, a linear relation (including offset) is more appropriate. In general, any linear or non-linear relation can be determined.

[0041] For example, it is also possible to calculate an calibration factor using the ration between P and the ratio between two averaged intensity values. In general, any curve fitting approach may be used on the available data of P in dependence of average intensities. Once the calibration factor or other correlation relation between P and the one or more average intensities is known, the pixels outside the scan line location can be converted to the normalized value and indicate e.g. the oxygenation, Sto2 or other data of interest.

Brief description of the Figures

[0042] On the attached drawing sheets, • figure 1 schematically shows a configuration of a 2D imaging system according to an embodiment of the invention; • figure 2 schematically shows a perspective view of a part of the 2D imaging system of figure 1; • figure 3 schematically shows a sample and measured image data according to an embodiment of the invention; • figure 4 schematically shows a further 2D imaging system according to an embodiment of the invention; • figure 5 schematically shows a sample and measured image data according to an embodiment of the invention; • figure 6 schematically shows a sample with scan lines according to an embodiment of the invention; • figure 7 schematically illustrates an application of an embodiment of the invention; • figure 8 schematically shows spectral curves for (oxy)hemoglobin, as used in an application according the invention; • figures 9a-9f schematically show measured and modelled spectra; • figures 10a-10d schematically illustrate the determination of a ratio according to respectively a prior art method and an embodiment of the invention; • figures 11a and 11b schematically show a measurement device according to an embodiment of the invention; • figure 12 schematically shows a laparoscope according to an embodiment of the invention; • figure 13 schematically shows a processing device according to an embodiment of the invention; • figure 14 schematically shows a method for determining a parameter according to an embodiment of the invention.

• figures 15a and 15b schematically show confidence areas of a parameter determined according to an embodiment of the invention; and • figure 16 schematically shows a method for determining an oxygenation ratio according to an embodiment of the invention.

Detailed description

[0043] Figure 1 schematically shows a 2D imaging system 10 according to an embodiment of the invention. A light source (LS) or light engine 13 lights a sample 12, with reflected light focused by lens 13 on the entrance of imaging system 10. The imaging system comprises a 2 channel prism assembly 14, comprising two prisms 15 and 16 configured to split the incident light from lens 13 into a first channel (C1, emerging from prism 15) and second channel (C2, emerging from prism 16). The light in the first channel C1 is filtered by filter 17 and detected in two dimensional (2D) sensor 18. The light in the second channel C2 is sent through slit 19, dispersion prism 20, filter 21, and finally detected in 2D sensor 22.

[0044] The 2D sensors 18, 22 have a 2D sensor array and can detect and output a 2D image. The filters can be configured to select a wavelength or wavelength range. In an embodiment, the filter 17 in C1 is configured to select a narrow wavelength range (e.g. around 670 or 920 nm in an oxygenation detection application) while filter 21 in C2 selects a broad range, e.g. 400 - 1000 nanometers (nm). In fact, a filter may not be needed in channel C2. The 2D sensor 18 in C1 is configured to generated a 2D (spatial, with coordinates x, y) image at the selected wavelength(s).

[0045] In the second path, slit 19 blocks all light except the light along a scan line. The one-dimensional light pattern is provided to dispersion prism 20, which spatially separates the various wavelengths in the light. The resulting light distribution is passed through filter 21 and imaged on 2D sensor 22. The sensor in C2 thus measures light frequency/wavelength in one direction and a spatial coordinate (x, if the scan line is in the direction of coordinate x) in the other direction.

[0046] The imaging system 10 is calibrated so that it is known which line in the image sensed by 2D sensor 18 corresponds to the scan line selected by slit 19. In other words, the spectral/spatial data measured by detector 22 can be mapped to the 2D spatial data measured by detector 18. If the wavelengths sampled by C2 detector 22 comprise all wavelengths sampled by C1 detector 18, then the calibration can be checked - the 1D (spatial) response obtained by integrating the spectral response as measured by C2 detector 22 over the range of wavelengths used by C1 detector 21, should, at least in shape, match the corresponding line in the 2D image ofC1 detector 21.

[0047] The sensors 18 and/or 22 (possibly combined with a filter 17, 21) may be glued to each of the output channels of the beam-splitter 14 respectively the dispersion prism 20. This arrangement provides mechanical stability.

[0048] The beam splitter, in the present example a prism assembly 14, splits up the light into at least, but not limited to, two channels (C1, C2) based on either an energy splitting beam splitter, or a dichroic coating. As was mentioned, C2 is aligned to C1 in such a predefined manner that it results in a calibrated coregistered image system that has a known registration between pixels in C1 and C2. It is for this registered (or calibrated) line of pixels that of every corresponding pixel in the 2D image a complete spectral response plot can be given.

[0049] In an embodiment, the slit 19 is motorized so that the position of the slit with respect to the sample, and thus the position of the scan line in the 2D image data, can be moved within a certain spatial range.

[0050] Figure 2 schematically shows a perspective view of an assembly of a slit 19, a dispersion prism 20, a filter 21, and 2D sensor 22 as may be used in C2 of figure 1. The slit, which may have a width of 50 - 200 urn, creates a horizontal line. The light along the line is guided through a dispersion prism 20 which separates the wavelength components and projects these vertically with blue on top and red on the bottom. A 2D sensor 22 (possibly preceded by filter 21) is then placed and aligned behind the dispersion prism 20 so that all the lines of the 2D sensor “sense” a different wavelength. The resolution of each line of the sensor represents about 2 nm, being a little less at the blue side and higher on the red/infrared side (4 nm per line).

[0051] Figure 3 schematically shows the resulting images of the configuration of figure 1. The C1 sensor 18 generates 2D image 31 at a selected wavelength. The C2 sensor 22 generates a set of spectra (intensity I versus wavelength λ) 32, each spectrum corresponding to a point along the scan line (coordinate x). Another way of describing data 31 and 32 is that data 31 is 2D spatial-spatial intensity (x, y, I) data, whereas data 32 is 2D spectral-spatial intensity (λ, x, I) data. Here (x, y, I) indicates a table of measured intensity values with the corresponding x and y coordinates. The table can be seen as sample points of the function l(x,y), indicating an intensity as a function of coordinates x and y. Likewise, tabular data of (λ, x, I) can be seen as sample points of the function Ι(λ, x), indicating an intensity as a function of wavelength and coordinate x. Similarly Ι(λ, y) is a function of wavelength and coordinate y The wavelength range of the samples may for example be 400 to 1000 nm. The intensities may be absolute, calibrated, values or may be expressed in relative (arbitrary) units.

[0052] In the overview of the sample 12 in figure 3, the dashed line 30 represents the scan line. The spectral data in set 32 corresponds to spatial points along this line.

[0053] Figure 4 shows an example of a four-way beam splitter 40 that may be used to generate four channels (C1, C2, C3, C4) in a imaging system according the invention. The beam splitter 40 comprises five prisms 41 - 45.

[0054] Figure 5 shows an example of resulting images in an imaging system using the four way splitter 40 of figure 4. Channels C1 and C2 are connected to spectral imaging units, to respectively measure a (λ, y, I) data along scan line 51 and (λ, x, I) data along scan line 52. Channels C2 and C3 are connected to 2D imaging units, to respectively measure (x, y, hR) 2D Infrared data and (x, y, lvis) 2D visible light data. In the example of figure 5, the scan lines 51, 52 are perpendicular so that a crosshair is formed.

[0055] Figure 6. shows a different configuration of sampling multiple dispersion lines close to each other, using two horizontal lines 61,62 with one or more 2D images of different wavelengths. The main difference with figure 5 is thus that now the scan lines 61 and 62 are parallel and not perpendicular.

[0056] Figure 7 shows an example of application on how an imaging device according to an embodiment of the invention can be used to assess tissue perfusion and oxygenation before, during or after a surgery. Image 70 shows a colon 71 with location 72 where the colon was re-attached after surgery. Images 73 and 76 are 2D images at visible wavelengths and at 810 nm, respectively. Data sets 77 and 78 comprise (λ, x, I), corresponding to scan line 75, and (λ, y, I) data, corresponding to scan line 74, respectively. It is possible to examine the blood component in each pixel of data sets 77, 78, and relate them to the 2D image at 810 nm, the isosbestic point as a reference channel, so that the oxyhemoglobin can be estimated in the entire 2D spatial range of 2D images 73, 76.

[0057] Figure 8 shows spectra for methemoglobin, oxyhemoglobin, and reduced hemoglobin as emission intensities (arbitrary units) as a function of wavelength (symbol λ, expressed in nanometers). The invention allows to sample such values for each pixel along a scan line, and to match these values with a 2D image of selected parts of the wavelength ranges (i.e. 2D image at 920 and 670).

[0058] It is known to determine the ratio R1/R2 of the intensities at 670 nanometers (R1) and 920 nanometers (R2) as a value indicating the oxygenation of blood. A value of 0.5 corresponds to approximately 100% oxyhemoglobin, whereas a value of R1/R2=2 indicates virtually no oxygenation. A value of approximately R1/R2=1 can indicate a reduced oxygenation.

[0059] It is noted that prior art document WO 01/15597 uses a third average intensity measurement at 810 nanometer as a reference signal for the other two signals, in order to increase accuracy. In a system according to the invention this is not required, since the available spectral data makes the 810 nanometer measurement redundant. The spectral data allows a complete curve fit of the whole spectral response, rather than a 3 point sample fit as used in WO 01/15597. This spectral fit improves the reliability of the resulting parameter determination and hence allows a more accurate quantification. With this information, a precise calculation table can be generated to calibrate the 2D images and relate them together without the need for the 810 nm 2D image.

[0060] Advantages of the invention are apparent from the example spectra of figures 9a-9f. Figure 9a schematically shows a spectrum as may be measured in a medical application. The spectrum contains not only contributions from (oxy)hemoglobin, but also shows an excitation peak at approximately 785 nanometers and a resulting emission in a broad profile with a maximum at 820 nanometers. This is typical for e.g. an ICG fluorescence applications. The ICG excitation peak and fluorescence peak are shown in isolation in figures 9b and 9c, respectively. Figure 9d shows the (oxy)hemaglobin signal in isolation from the additional signals. It is a particular advantage of the current invention that, along the scan lines at least, it is possible to separate the various contributions, so that it is possible to remove or at least reduce the influence of the excitation and fluorescence radiation (and, in general, any other contributing substance). In figure 9e, the three contributors, (oxy)hemoglobi, excitation, and fluorescence, are shown as three separate curves. Figure 9f shows the three curves added together, which matches the measured spectrum of figure 9a.

[0061] Figures 10a-10d illustrate the determination of the ratio R1/R2. If, in figure 10a, only the average intensity in the 670 nm wavelength range 81 and the average intensity in the 920 nanometer wavelength range 82 is used, then the presence of the fluorescence radiation around range 82 will cause an error in the determination of R1/R2. As can be seen in figure 10b, the radiation at 670 nanometers, with average intensity 83, is more or less correct but the radiation at 920 nanometers, with average intensity 84, is overestimated resulting in a R1/R2 ratio that is too low. In contrast, the invention allows that, along the scan lines at least, the influence of the additional spectra (e.g. ICG excitation and fluorescence) is removed. The intensities 85 and 86, corresponding to R1 and R2 respectively, are free from the disturbing influence of ICG, allowing an accurate determination of R1/R2.

[0062] Because along the scan lines both the “raw” R2 value 84 and the filtered R2 value 86 is determined, the fraction of radiation at 920 nanometers that belongs to hemoglobin is known (i.e. the peak of 84 divided by the peak of 86). Using this calibration value, the average (“raw”) intensities at 920 nanometers as measured over the entire 2D range can be corrected to remove or at least reduce the influence of the ICG spectra.

[0063] Figures 11a and 11b schematically show a measurement device 90 according to an embodiment of the invention. The device 90 comprises a lens 92 for receiving light from a sample. The lens 92 is surrounded by LEDs 91 forming a ring light for lighting the sample. Filters can be placed before the LEDs or output fibers to control the light that is sent to the studied sample. In an alternative embodiment, the light is provided by lasers or via light fibers which transport the light from a distant light source to the device 90. The LEDs or alternative light source(s) will emit light at a suitable wavelength for the application of the device. It is possible to provide multiple sets of light sources in the ring light for various applications. It is also possible to make the ring light module exchangeable, so that a suitable ring light module can be installed for an application of the device 90. The device 90 further has a housing 93 attached to the ring light and a handle 97 for holding the device 90. Inside the housing 93, the device comprises a imaging system 94 according the invention and a processing unit 95. At the back surface, opposite the lens 92 surface, the device may have an LCD display 95 (figure 11b) connected to the processing unit 95. The display 95 may be a touch panel, so that the user of the device can interact with the processing unit 95 via the touch panel.

[0064] In operation, light from the sample will be collected by lens 92 and sent to the imaging system 94. The processing unit 95 analyses the data collected by the sampling units of the imaging system, and provides an output picture. In the example of figure 11b, the system 94 comprises one 2D sampling unit and two spectral sampling units. The display shows the 2D image and the scan lines corresponding to the two spectral sampling units. In addition, the 2D image may show the extrapolated parameter value as calculated by the processing unit as overlay on top of the 2D image (for more details on the calculation see figure 14).

[0065] Figure 12 schematically shows a laparoscope 100 according to an embodiment of the invention. The laparoscope has an end 101a comprising a lens. Alternatively, a diagonal end surface 101b with lens may be provided (figure 10a). The laparoscope 100 has an elongate body 101 with a connector 102 for coupling in light from a light engine 108. The laparoscope 100 has a main housing 103 connected to the elongate body 101 and a handle 107 for holding the laparoscope. The housing 103 comprises an imaging system 104 according the invention and a processing unit 105 according the invention. The housing further comprises a connector 106 for connecting to an external display and a connector 109 for connecting a power source.

[0066] When connected to an external display via connector 106, the laparoscope 100 functions analogously to the measurement device of figure 11a and 11b, where the external display takes the place of display 96, the light engine 108 takes the place of the ring light, and the lens in ending 101a or 101b takes the place of lens 92.

[0067] «Figure 13 schematically shows a processing device according to an embodiment of the invention, such as may be used in the devices of figures 11a, 11b, and 12. The 2D sensor units 111 and 112 output an analogous signal which is digitized by Analog-to-Digital-Convertors (ADCs) 113 and 114 respectively. The digital signals are analyzed by processing unit 115, which is connected to a touch panel comprising display 116 and touch sensor unit 117. The ADC may be integrated in the sensor, as is for example done in CMOS sensors.

[0068] Figure 14 schematically shows a method 120 for determining a parameter P(x,y) as a function of location (x,y) in the sample, according to an embodiment of the invention. The parameter P can be any measureable quantity that can be determined from a spectrum measurement. An example that has been used throughout the application is the determination of oxygenation (see figures 8, 9a-9f, 10a-d). It is noted that the particular order of the steps in method 120 is generally not important. Many steps can be performed in arbitrary order, provided of course that the necessary data is measured before it is processed. An optional step, not shown in figure 14, is to perform a relative or absolute measurement for environment lighting, so that the influence of environment lighting on the determined (spectral) intensities can be separated in a further processing step.

[0069] The example of figure 14 focuses on an exemplary imaging system having one 2D sampling unit, providing (x, y, I), and two spectral sampling units, providing (λ, x, I) and (λ, y, I). The sampled data can be represented as a sampled mathematical functions l(x,y) (as sampled by the 2D sampling unit) and k(x,y0) and li(x0,y) (as sampled by the two spectral sampling units). Here the subscript λ in Ιλ indicates that the intensity is provided as a function of wavelength λ. Value x0 represents the x value of the vertical scan line (see e.g. line 51 in figure 5) and y0 represents the y value of the horizontal scan line (see e.g. line 52 in figure 5). In steps 121, 122, and 123 the sampled data for functions l(x,y), k(x,y0), and k(x0,y) is collected, respectively.

[0070] In step 124, data representing function I(x, y0) and P(x, y0) is calculated from k(x,y0)· l(x,y0) may be calculated by integrating function k(x,y0) over the range of wavelengths that is sampled by the 2D image sampler used to obtain l(x,y). In practice, the integral will be evaluated using a weighted sum of ^(x,y0) for a number of frequency samples. P(x, y0) is calculated according to the method for determining parameter P. An example method has been discussed in reference to figure 8. The calculation of P may comprise a spectrum separation (through e.g. curve fitting) as disclosed in reference to figures 9a-9f and 10c-d. In general, the calculation may comprise curve fitting and calculating relative and absolute peak intensities. For example, often the ratio between two peak intensities is a parameter to be determined. Step 125 is similar to step 124, except now l(x0, y) and P(x0, y) is calculated from L(x0,y).

[0071] In step 126, a first consistency check is done. The values l(x,yo) as calculated should, while accounting for any background radiation that may have been removed from the spectral measurements during processing, correspond to the measured values of l(x,y) along the line y=y0. The same holds for l(xo,y): these calculated values should correspond to the measured values of l(x,y) along the line x=x0. Step 126 is optional, but may advantageously serve to detect errors in measurements or calibration.

[0072] In step 127, a second consistency check is done. The correlation between the calculated l(x,yo) and P(x,yo) values and the calculated l(xo,y) and P(xo,y) values is checked. Like intensities I should give like parameter values P, otherwise the assumption at the basis of the extrapolation of P(x,y) is not valid. The correlation can also serve as input for a statistical analysis of the confidence interval of an extrapolated P(x,y) value.

[0073] The extrapolated values P(x,y), or dataset (x,y,P), is determined in step 128. Using l(x,y) as input, the correlation between l(x,yo) and P and/or the correlation between l(xo,y) and P is used to estimate P(x,y). Various methods can be used to determine P(x,y) based on l(x,y) and P(x,yo) and P(xo,y). In particular, the calculated values of P at points close to (x,y) can be given a greater weight than P values calculated for more remote points.

[0074] In optional step 129, a confidence value or interval is calculated, indicating the expected error in P(x,y). Depending on the statistical methods used, the skilled person can apply standard statistical techniques for calculating such a confidence value or interval.

[0075] In the above description of method 120, a cross-hair configuration of two scan lines has been used. It will be clear to a skilled person that the method may also be modified to be applied to calculating P(x,y) for any number of scan lines in any configuration.

[0076] It may be that more than one parameter P can be calculated from the spectral data Ιλ. For example, let intensity h be indicative of parameter P^ and intensity l2 indicative of parameter P2. An imaging unit that is configured to measure two 2D images (x,y,h) and (x,y,I2) and any number of scan lines Ιλ can then be used to calculate extrapolated values for Pi(x,y) and P2(x,y).

[0077] It is a major advantage of the method of figure 14 and its variants that it allows real-time display of calculated and measured parameters P on top of (as overlay of) visible light image data. The overlay is synchronized with the visible light image data, so that even samples with internal movement can be measured. In addition, the extrapolation is based on a analysis of the full spectrum, so that any background radiation not contributing to the peaks of interest can be effectively disregarded in the calculation of P.

[0078] To further illustrate the comments made in reference to steps 128 and 129, figure 15a and 15b schematically illustrate the estimated accuracy (confidence value) levels of the 2D parameter determination according to an embodiment of the invention. The drawn lines represent the scan lines, with figure 15a having the scan line configuration of figure 6 and figure 15b having the alternative scan line configuration of figure 5 and 7. The dotted lines, located closest to the scan lines, represent lines in the 2D range where there is relatively high confidence in the extrapolated parameters. The dashed lines represent lines in the 2D range where there is a reduced confidence in the extrapolated parameters. It will be clear to a skilled person that this is but one metric that can be used to determine confidence intervals. Other mathematical methods may be used to link a position (x,y) in the 2D range to a confidence value based on the relative location of the one or more scan lines. In addition, the internal consistency of the spectral measurements can be a factor in determining the confidence value. For example, if all spectral measurements are nearly identical, this is an indication that the 2D range is of a fairly homogeneous composition and confidence in extrapolated values will be high. In contrast, if the spectral measurements have a strong spatial dependency along a scan line, this can be an indication that the sample composition is spatially inhomogeneous, and the extrapolated values may have low confidence. For example, the standard deviation of a parameter as determined from the spectral measurements can be factor in determining the confidence value of the extrapolated parameter values - the confidence value can be taken to be inversely proportional to the standard deviation.

[0079] Figure 16 schematically illustrates a method 140 for calculating oxygenation ratios, which method can be seen as a special case of the more general method 120 of figure 14. In step 141 the system is calibrated for environment lighting. This calibration can comprise an absolute or relative spectral measurement of the environment light. In step 142, the 2D and scan line measurements are performed, as described earlier in this application. Any of the devices of figures 11-13 could for example be used. In step 143, the spectral scan line measurements are analysed, using application specific knowledge. In this case, known profiles for (oxy)hemoglobin, an excitation source, and icq emission signals are used to separate the various contributions, so that in particular the various hemoglobin profiles can be separated in step 144 (see also figure 8). As part of this step, the environment lighting can also be removed from the measurement data.

[0080] Based on the separated contributions, in step 145 a calibration factor is determined which correlates the required parameter (e.g. the 920 nm / 670 nm ratio of figure 9d as disclosed in reference to figures 8, 9a-9f, 10c-d) to the average intensity as measured in the wavelength range that is used for the 2D measurements. In step 146, if necessary, the average intensities as measured in the one or more 2D images are normalized, so that they can be compared with the calibrated intensities from the spectral measurements. Finally, in step 147, the required parameter (the mentioned ratio) is determined for the full 2D range by using the determined calibration factors.

[0081] In the foregoing description of the figures, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the scope of the invention as summarized in the attached claims.

[0082] In particular, combinations of specific features of various aspects of the invention may be made. An aspect of the invention may be further advantageously enhanced by adding a feature that was described in relation to another aspect of the invention.

[0083] It is to be understood that the invention is limited by the annexed claims and its technical equivalents only. In this document and in its claims, the verb "to comprise" and its conjugations are used in their non-limiting sense to mean that items following the word are included, without excluding items not specifically mentioned. In addition, reference to an element by the indefinite article "a" or "an" does not exclude the possibility that more than one of the element is present, unless the context clearly requires that there be one and only one of the elements. The indefinite article "a" or "an" thus usually means "at least one".

Claims

An imaging system (10) comprising: - a beam splitter unit (14, 40) for splitting a beam of incoming light from a sample (12) into at least a first channel (C1) and a second channel (C2); - at least a two-dimensional, 2D, image sampling unit (18) adapted to receive light via the first channel and to output a 2D image; and - at least one spectral sampling unit (20, 22) comprising a means for spatially separating light based on wavelength or frequency, the spectral sampling unit being adapted to receive light via the second channel and to sample a spectral response for each point along a scan line (English, scan line) (30, 51, 52, 61, 62, 74, 75) corresponding to a line in the 2D image; and - a slit (19) for forming an essentially one-dimensional light pattern as input for the spectral sampling unit (20, 22), the slit blocking the light in the second channel except for the light along the scanning line.

The imaging system (10) of claim 1, wherein the location of the scanning line (30, 51, 52, 61, 62, 74, 75) is calibrated relative to the 2D image.

The imaging system (10) according to claim 1 or 2, wherein the location of the scanning line (30, 51, 52, 61, 62, 74, 75) is displaceable relative to the sample.

The imaging system (10) of claim 4, wherein the slit (19) is movable to move the location of the scan line (30, 51, 52, 61, 62, 74, 75) relative to the sample.

The imaging system (10) according to any of the preceding claims, comprising a filter in the first and / or the second channel (C1, C2) for selecting a wavelength range.

The imaging system (10) of claim 5, wherein the wavelength range of the first channel (C1) is a subset of the wavelength range of the second channel (C2).

The imaging system (10) according to any of the preceding claims, wherein the beam splitter unit (14, 40) is a dichroic prism assembly (15, 16, 41-45).

The imaging system (10) according to any of the preceding claims, comprising N 2D image sampling units (18) and M spectral sampling units (20, 22), wherein the beam splitter unit (14, 40) is arranged to split the beam into at least N + M channels (C1, C2, C3, C4) such that each image sampling unit (18) and each spectral sampling unit (20, 22) receive light through a respective channel from the beam splitter unit (14, 40), where N and M are integers, and the sum of N and M equals two, three, or more.

An assembly (90, 100) of an imaging system (10) according to any one of the preceding claims and a light source (91) or light driver (108).

The assembly of claim 9, wherein the light source (91) or light driver (108) is adapted to controllably emit selected wavelengths.

An assembly according to claim 9 or 10, wherein the light source is a ring light (91).

A tissue analysis device comprising an imaging system according to any of the preceding claims 1-8.

An apparatus for fluorescence imaging (English: fluorescent imaging) comprising an imaging system according to any of the preceding claims 1-8.

An agricultural analysis apparatus comprising an imaging system according to any of the preceding claims 1-8.

A laparoscope (100) comprising an imaging system according to any of the preceding claims 1-8.

Method for calculating a parameter (P) in a two-dimensional, 2D, range, using an imaging system (10) according to any of the preceding claims 1-8, the method comprising: - sampling (121, 122) at least one set of average intensity values for the 2D range; - sampling (123) spectral intensity values along at least one curve in the 2D range; - calculating (124, 125) values of the parameter (P) along the at least one curve from the sampled spectral intensity values; and - extrapolating (128) the values of the parameter (P) for the 2D range based on the sampled average intensity values for the 2D range and the calculated values of parameter (P) along the curve.

The method of claim 16, comprising - separating the sampled spectral intensity values into a plurality of spectral contributions from respective components.

The method of claim 17, comprising the use of curvature approximations to separate the sampled spectral intensity values into the plurality of spectral contributions.