ZA200308144B - Spectrometry using broadbrand filters with overlapping spectral ranges. - Google Patents

Description

) SPECTROMETRY USING BROADAND FILTERS WITH OVERLAPPING SPECTRAL RANGES

SPECTROMETRIC DEVICES

THIS INVENTION relates to spectrometric devices. In particular, it relates to a spectrometer or spectro-radiometer or the like devices, and to a method of recovering a signal.

The conventional spectrometric devices known to the

Applicant all suffer from inherent disadvantages. Thus, a spectrometer using narrow band optical filters or prisms or gratings suffers from a poor signal to noise ratio and low throughput. Fourier Transform

Spectrometers, such as the Fourier Transform Infra-Red (FTIR) spectrometer, although possessing a multiplex advantage, and having a high throughput, are very expensive and not robust, requiring high tolerance components. Thus, the distance that the mirror moves relative to the beam splitter of a Fourier Transform Spectrometer is very small, in the order of nanometers, particularly if the spectrometer is intended for use in the visual light spectrum. As will be appreciated, this makes it very difficult to build a Fourier Transform Spectrometer for use in the “visual light spectrum. Furthermore, a Fourier Transform Spectrometer suffers from a field of view dependent wavenumber shift or phase difference among the different pixels in the field of view. A a. spectrometer using an acousto-optical tunable filter (AOTF) suffers from \ the disadvantage that the power of the radio frequency used to set up the acoustic wave in the crystal must be limited to inhibit unpredictable inter-modulation. An AOTF spectrometer is costly to build and a crystal for use in the 8 to 12 .m spectral radiation range has not been identified. .

According to one aspect of the invention, broadly, there is ) provided a spectrometer which includes a filter arrangement, in addition to any optional filters for limiting an operational wavelength range of the spectrometer, the filter arrangement comprising a plurality of broadband optical filters or broadband optical filter areas each of known transmission and being located or selectively locatable in a path of collected incident spectral radiation.

In this specification, the term spectrometer is intended to include spectrometric devices such as spectro-radiometers, spectrographs, and like devices, whether imaging or point devices and, for a spectrometer having an operational wavelength range from a lower operational wavelength A to an upper operational wavelength Ay and with a spectral resolution Al, a broadband filter is defined as a filter with a spectral transmission t(A) which is significantly greater than O over a wavelength region greater than the spectral resolution AA of the device. In other words, a measurement of the spectral radiation passing through the broadband filter does not allow direct measurement of the incident spectrum over the spectral resolution AA. Further, in this specification, a prime (‘) is used to indicate a vector or matrix transpose.

More particularly according to the invention, there is provided } a spectrometer which includes 4 a filter arrangement, in addition to any optional filters for limiting * an operational wavelength range of the spectrometer, the fitter :

arrangement comprising a plurality of broadband optical filters or a broadband optical filter areas each of known transmission and being located or selectively locatable in a path of collected incident spectral ) radiation; at least one detector arranged to measure the spectral radiation passing through at least one of the broadband filters located in the path of collected incident spectral radiation; and signal-processing means for recovering the spectrum of the collected spectral radiation from measurements by the detector.

The spectrometer of the invention thus does not require narrow band filters of any form, dispersive elements such as gratings or prisms, beamsplitters, or the like to function. Instead, in the place of these conventional spectrometer elements, a filter arrangement comprising a plurality of broadband optical filters or broadband optical filter areas each of known transmission is used.

Typically, the spectrometer includes collecting optics for collecting spectral radiation and focussing the collected spectral radiation on one or more of the broadband filters or filter areas. The spectrometer may also include focusing optics for focusing spectral radiation passing through the one or more broadband filters or filter areas on to the at least one detector.

The spectrometer may include a spectral band-pass filter between the filter arrangement and the at least one detector, typically , between the filter arrangement and the focusing optics, to limit the operational wavelength range of the spectrometer. Preferably, the spectral band-pass filter has practically zero spectrai radiation transmission outside the range A to hy

The filter arrangement may include a filter set with a plurality of unique and individual broadband filters or filter areas.

Preferably, in this case, all the broadband filters or filter areas are linearly independent, i.e. the transmission of any one broadband filter or filter area is preferably not a multiple of the transmission of any other broadband filter or filter area. [In one embodiment of the invention, the filter arrangement includes at least two filter sets arranged one in front of the other to provide broadband filter combinations. Most preferably, the filter arrangement includes only two filter sets, to ensure a high throughput through the filter arrangement, whilst retaining the advantage of reduced cost.

Each filter set may be in the form of a filter wheel.

The spectrometer may include locating means for selectively locating the broadband filters or filter areas in the path of collected incident spectral radiation.

When the filter arrangement includes two filter sets, the locating means may be configured to locate one filter or filter area on one filter set in the path of collected incident spectral radiation whilst successively locating each filter or filter area of the other filter set in the , path of collected incident spectral radiation, thereby forming a plurality of broadband filter combinations. Typically, this implies that if the filter )

sets are in the form of filter wheels, one filter set turns one index . position, and the other filter set turns through all index positions. : Each broadband filter combination may be unique and 5 individual and preferably each broadband filter combination is linearly independent of every other broadband filter combination. Preferably, the broadband filter combinations are selected to maximise the signal to noise ratio over the wavelength range A to Mu,

The spectrometer may be configured to measure N wavelength bands, i.e. it may have a spectral resolution A1 = Ay Y/N and the spectrometer may include J broadband filters or filter areas or J filter or filter area combinations, where J = N > 2. Preferably, J > N to improve the signal to noise ratio. When the filter arrangement includes two filter sets, each filter set may thus comprise M broadband filters or filter areas where J = M? or, in other words, the number of filters or filter areas equals 2M instead of J.

The filter arrangement may be configured to allow the spectrometer of the invention to operate in the visual and near infrared spectrums, e.g. 400 — 800 nm, 3 - 5 um and 8 — 12 um spectrums.

The filters or filter areas may be thin film filters or absorption filters. : 25

The spectrometer may include a detector associated with each broadband filter or filter area or with each filter or filter area combination which can exist simultaneously, allowing simultaneous sampling of each filter or filter area or each actual filter or filter area combination. Such a spectrometer provides the benefit of high speed measurement, in addition to the benefits of multiplexing and high throughput. Although the detectors each may have a slightly different ' point of view this may not be an issue if either the spectrometer or the scene viewed by the spectrometer is moving.

The signal-processing means may be configured, for J measured digital signals received by it and each digital signal corresponding to a measurement taken for a particular filter or filter combination, to arrange the J measured digital signals in a vector d, where the vector d=Bs+vand B is a matrix whose row vectors represent the effects of any hardware components, used to obtain the measured digital signals, on the measured digital signals, s is the incident spectrum and v is a noise vector, and to multiply a signal recovery matrix G with the vector dof the measured digital signals, to provide the measured incident spectrum S, i.e. § = Gd where

G = VAAD' or the pseudo inverse of B;

A, =diag{l/7,,...,1/7,,0....,0}, the subscript r indicating that only the r largest singular values are used; and

B=ULV' , the singular value decomposition of B, where

Uand V are JXN and NxN orthogonal matrices, the matrix X is a NxN diagonal matrix containing the singular values of B in descending order, i.e. Z=diagly,,...,7x}, #1 2%,...27y 20, and r is an integer in the ] range 1< r < N selected to minimize the measurement error.

According to another aspect of the invention, there is : provided a method of obtaining a measurement of an incident spectrum s over N wavelength bands or zones, the method including filtering the incident spectrum s through J optical broadband filters or filter combinations to provide J individual filtered spectrums; converting the J filtered spectrums to J measured analogue electrical signals; converting the J measured analogue electrical signals to a vector d of J measured digital signals, where the vector d=Bs+v and B is a matrix whose row vectors represent the effects of any hardware components, used to obtain the measured digital signals, on the measured digital signals, and v is a noise vector; and multiplying a signal recovery matrix G with the vector d of the measured digital signals, to provide the measured incident spectrum §, i.e. S = Gd where

G = VAU' or the pseudo inverse of B;

A, =diag{l/,,...,1/7,,0,...,0}, the subscript r indicating that only the r largest singular values are used; and

B =UZXV' , the singular value decomposition of B, where

Uand V are JxN and NxN orthogonal matrices, the matrix X is a NxN diagonal matrix containing the singular values of B in descending order, i.e. =diag{y,s....7n}, ¥12%,...27y 20, and ris an integer in the range 1<r < N selected to minimize the measurement error. . 25

The matrix B may include values or elements in its row : vectors reflecting the effects of any hardware components, used to obtain the measured digital signals, on the measured digital signals, the h effect being selected from the group of effects consisting of a detector ) pre-amplifier voltage gain, the transmission of any collecting optics, the

A transmission of any focusing optics, the throughput of optical elements, - the transmission of any filter or filters for limiting the wavelength range : of the incident spectrum s, the transmission of the broadband filters or filter combinations, a detector responsivity, or two or more of these.

The invention will now be described, by way of Examples, and with reference to the accompanying exemplary diagrammatic - drawings in which h

Figure 1 shows a block diagram of one embodiment of a : spectrometer in accordance with the invention; : Figure 2 shows a block diagram of another embodiment of a spectrometer in accordance with the invention, for use in the oo visual/near infrared spectrum;

Figure 3 shows a schematic representation of a noise model used by the spectrometers of Figures 1 and 2;

Figure 4 shows a schematic representation of a signal processing model used by the spectrometers of Figures 1 and 2;

Figure 5 shows a schematic representation of an alternate signal model used by the spectrometers of Figures 1 and 2, illustrating an estimation of an incident spectrum given a true spectrum and noise;

Figure 6 shows the transmission of filters on a first filter wheel, comprising eight filters, of the spectrometer of Figure 1; - Figure 7 shows the transmission of filters of a second filter wheel, comprising eight filters, of the spectrometer of Figure 1;

Figure 8 shows the transmission of the sixty four combinations of the filters of the first and second filter wheels of the spectrometer of

Figure 1; :

Figure 9 graphically illustrates the mean square error, : approximation error and transformed noise of a measured spectrum using the spectrometer of Figure 1, as a function of rank;

Figure 10 graphically illustrates an original or incident spectrum and a spectrum measured or recovered by the spectrometer of Figure 1;

Figure 11 graphically illustrates the transmission and harmonics of a first thin film filter of an example filter set for taking measurements in twenty spectral bands, with target and designed transmission being identical;

Figure 12 graphically illustrates the transmission and harmonics of a second thin film filter of the example filter set;

Figure 13 graphically illustrates the transmission and harmonics of a third thin film filter of the example: filter set;

Figure 14 graphically illustrates the transmission and harmonics of a twentieth thin film filter of the example filter set;

Figure 15 graphically illustrates an original or incident spectrum, a spectrum measured or recovered by means of a simulation of a spectrometer in accordance with the invention and in accordance with

Example 2, and a spectrum measured or recovered by means of a simulation of a narrowband prism or grating type spectrometer with the same noise as the simulated spectrometer in accordance with the invention; and

Figure 16 graphically illustrates the Instrument Line Shape (ILS) of the simulated spectrometer of Example 2. . Referring to Figure 1 of the drawings, reference numeral 10 generally indicates a spectrometer in accordance with the invention.

The spectrometer 10 can be used in the infrared spectrum, but the embodiment shown in Figure 1 is configured for use in the visual spectrum. :

The spectrometer 10 includes conventional collecting optics 12, a conventional modulator 14, a first filter wheel 16, a second filter wheel 18, a spectral band- pass filter 20, conventional focusing optics 22, a detector 24, a detector pre-amplifier 26, an analogue band-pass filter 28, an analogue/digital (A/D) converter 30, a demodulator 32, and a signal processor 34. The spectrometer 10 further includes three actuators 36, 38, 40 and a controller 42.

The modulator 14, filter wheels 16, 18, spectral band-pass filter 20, focusing optics 22 and detector 24 are in use arranged to be in the path of incident light collected by the collecting optics and indicated by arrow 44.

The spectral band-pass filter 20 removes light with wavelengths less than 400 nm and more than 800 nm. This limits the operational wavelength of the spectrometer to the visual spectral range of 400 to 800 nm.

The first filter wheel 16 comprises eight optical broadband filters obtained commercially from Schott Glaswerke, Germany. The eight filters each has a particular transmission as shown in Figure 6 of the drawings, over the spectral range of 400 to 800 nm. Similarly, the second filter wheel 18 comprises eight optical broadband filters obtained commercially from Schott Glaswerke, Germany and the . transmissions of the eight filters on the second filter wheel are shown in Figure 7 of the drawings, over the spectral range of 400 to 800 nm.

The filters on the first and second filter wheels 16, 18 have not been : selected for optimum performance, i.e. to maximize the signal to noise ratio over the wavelength range of 400 to 800 nm. In Figures 6 and 7 of the drawings, the Schott Glaswerke filters are identified by their

Schott Glaswerke identification codes. Although the filters have not been optimized, each filter on a particular filter wheel 16, 18 is linearly independent from all the other filters on the filter wheel 16, 18.

The controller 42 and the actuator 38, 40 are configured to maintain the first filter wheel 16 in a particular index position whilst rotating the second filter wheel 18 stepwise through 360° or eight index positions. In this fashion, eight combinations between a particular filter on the first filter wheel 16 and the eight filters on the second filter wheel 18 are obtained. Once the second filter wheel 18 has rotated through 360°, the first filter wheel 16 rotates through 45° or one index position to bring another filter on the filter wheel 16 into position in the path of the collected incident spectrum, and the second fitter wheel 18 again rotates stepwise through 360° to provide eight further filter combinations. In this fashion, the first filter wheel 16 and the second filter wheel 18 provide sixty-four broadband filter combinations.

As is indicated in Figure 1 of the drawings, the controller 42 also controls the modulator 14 by means of the actuator 36, and the demodulator 32. The modulation/demodulation functions are known to those skilled in the art and are not discussed in any further detail.

With reference to Figure 2 of the drawings, a spectrometer for use in the visual/near infrared spectrum is generally indicated by reference numeral 100. The spectrometer 100 1s similar to the spectrometer 10 and unless otherwise indicated, the same : reference numerals used in relation to the spectrometer 10 are used in relation to the spectrometer 100, for the same parts or features.

Unlike the spectrometer 10, the spectrometer 100 does not include the modulator 14, actuator 36 or demodulator 32. As is known to those skilled in the art, these features are not necessary for a spectrometer used in the visual/near infrared spectrum.

In the discussions that follow, reference is made only to the spectrometer 10; however, it is to be understood that any . reference to the spectrometer 10 hereinafter is also a reference to the spectrometer 100, unless inapplicable.

The signal processor 34 is configured to recover the spectrum of incident or collected spectral radiation from measurements by the detector 24 of the spectrums passing through the sixty-four broadband filter combinations. The configuration or programming of the signal processor 34 is based on certain models and assumptions, which are described in more detail below. The spectrometer 10 is configured to provide measurements over N spectral bands and uses J measurements to achieve this, where J > N.

For an optical/radiometric model, it is assumed that the collecting and focusing optics 12, 22 have spectrally flat transmissions 7, and 7,. Each filter wheel 16, 18 is modelled as having . transmission, 0<7,,(4,m )<1 and 0S Try, (1,m,)<1 for m,,m,=1,...,.M .

The transmission of the filter wheels 16, 18 is thus a function of . wavelength A and an angular position index m of the filter wheels 16, 18. The transmission of the chosen substrate of the filters of the filter wheels 16, 18 should be insensitive to temperature variations over the operational temperature range of the spectrometer 10. Some constraints on the average combined transmission may be required over the spectral band of interest so that a multiplex advantage is not lost. Specifically, it is required that the filters be non-orthogonal, i.e.

Fs [f Trt (Aa, V0 ma (Asm JAA >To > 0, Viy,m,=1,...,M.

The combined transmission is 7.(1, j)= Tuy, (4,12, ty, (2,m,) Where j= a measurement index of j=1,...,J measurements and the filter wheel combinations or indexes are described by m, = ceil(j/M) and m, =((j —1)mod M)+1 for j=1,...,J or alternatively j=(m ~1)M +m,.

This means that if the first filter wheel 16 turns only one index position, the second filter wheel 18 turns M index positions, where

M = 8 in the case of the spectrometer 10.

The filter wheels 16, 18 are followed by the spectral band-pass filter 20 having transmission 7e(1) to remove out-of-band light. This filter should have zero transmission outside the band : A, £A<A4,. The detector 24 responsivity is R(A) with zero mean

Gaussian noise n(t) and the detector pre-amplifier 26 gain is 4, . To and 7,, are the transmission of the collecting optics 12 and the focusing optics 22 respectively. The output of the detector 24, for a certain filter-wheel combination measurement ;=1..../ and as a function of time t is then given by dt, j)= 4 ff PLC) (AR(A)TL(A)dA + 4,n(t). ]

Here Tis the throughput of the spectrometer 10 and L(1) is the spectral radiance which is being measured.

For a physical signal model, the output of the detector 24 given above can be approximated as

N dt, j) ~ 4,77, TAL) 7c (4, iy) (4, JR(A, L(A, )+ A(t) i=1 with the wavelength range A4,.....4, divided into N bands, centred at

A, (a discrete wavelength measured in meter) , i=1,...,N, each band of width AA. If required more accurate integration methods can be used, such as Simpson's method. After sampling, a detector measurement can be written as

N d, = 4,7,7,TALY 1c(4, jYer (4 )R(A)L(4)+v, where v; is sampled noise. i=]

A noise model is shown in Figure 3 of the drawings. In

Figure 3, n(f) is band-limited white noise and n,(r) is quantisation noise, which is assumed to have uniform distribution over the quantisation interval. The sampled noise is denoted by v, with noise power oc’. It is required that the noise be independent and identically distributed between samples. In other words the noise covariance is of the form Efvv'}=0’1, where I is a JxJ identity matrix.

The spectral bands are modified by the optical : transmission of components in the path of incident spectral radiation and the responsivity of the detector 24. In order to model this, a set of {(1xN) spectral sampling vectors, b,, that summarise conversion from spectral radiance to voltage for each measurement, is defined. Each element of this vector is given by b, = Ayr, 7, TAA, (4 5 J Jer (4 R(A, ) .

The detector signal can now be represented, using vector notation as d;=bs+v, = where s=| : |. (Nx1)

L(2)

The non-linear detector responsivity and filter wheel transmission is captured by the vector b; which allows a simplified linear model to be used.

The signal processing problem is to determine the unknown radiance vector s using the signal model presented above.

To achieve this, the following matrix and vector forms are used: b, e A spectral sampling matrix, B={ : |, {(JxN matrix) . d, . 20 e detector measurements, - : | and (Jx1) d,

e noise 7 | (Jx1)

V,

The physical signal model presented above can then be represented in ) compact form as d=Bs+v.

In the case where J > N, the estimate or measurement of the spectrum can be formulated as an optimisation problem § = argmin £{d - Bx] [d — Bx].

The solution to this problem, as described in L.L. Scharf, Statistical

Signal processing: Detection, Estimation, and Time Series Analysis,

Addison-Wesley Publishing Company, 1991, is i=[B'B]'B'd.

Numerically stable ways of calculating this are available in the art, e.g. as described in chapter 2 of L.L. Scharf, Statistical Signal processing:

Detection, Estimation, and Time Series Analysis, Addison-Wesley

Publishing Company, 1991. Performing the operation B'B is described in more detail later.

Since spectral sampling vectors are constant for a given set of filter wheels 16,18, the matrix

G =[B'B]'B' (NxJ matrix) (the so-called pseudo-inverse) is constant and can be pre-calculated. In the case where J=N, the recovery matrix is the usual inverse (provided it exists) i.e. G=B". The spectral estimate or measurement can be obtained more succinctly from s=Gd.

It is to be appreciated that, if this estimate is to be used in further linear processing (such as might be the case with some of the conventional hyper-spectral detection algorithms), this processing could be combined to reduce computational time.

A spectrometer signal-processing model is shown in

Figure 4 of the drawings. It shows the physical signal model d=Bs+v that describes how measurements are related to the incident spectrum and the recovery process §=Gd. if the physical signal model is substituted into the recovery equation, the following is obtained

S=s+Gv =s+u where the transformed noise is defined as u=Gv. This is an alternate signal model, illustrated in Figure 5 of the drawings. It describes how an estimate or measurement of the spectrum is obtained given the true spectrum and noise. The noise, v, is essentially of unknown distribution. For the spectrometer 10, it is required that the noise be independent and identically distributed between samples. In other words the noise covariance is of the form E{vv'}= oI, where I is the

JxJ identity matrix.’

The covariance of the noise that is transformed into the estimate of the spectrum is

Flau'{= EIGvwW'G' =GE ad. =o (B'B)" and it will be useful in the following paragraphs (numerically stable ) ways of calculating this are described below). It is to be appreciated that the transformed noise covariance can be calculated without estimating the spectrum or calculating G.

A useful measure of the performance of the spectrometer is the signal to noise ratio (SNR). The SNR can be considered at the detector 24 and after recovery. These results are summarised in Table 10 1, Table 2 and Table 3.

Table 1 Spectrum and Detector Signal Power.

Sonal [Power

NI LI

Noise-less detector signal | s'B'Bs

Bs

Table 2 Noise and transformed noise Power.

ET i Bl ON

Table 3 Detector and estimate SNR.

I CL

’ Jo?

Estimate SNR (SNR, ) S's ss

ETE)

Performing ‘the operation B'B is essentially a squaring operation which doubles the dynamic range of floating point representation. When such a matrix is inverted, numerical inaccuracies may occur if this product is ill-conditioned. There are at least two ways around this: e Using the QR decomposition and e Using the singular value decomposition (SVD).

A background discussion of both methods can be found in chapter 2 of L.L. Scharf, Statistical Signal processing: Detection, Estimation, and

Time Series Analysis, Addison-Wesley Publishing Company, 1991 and

F. Deprettere, SVD and Signal Processing: Algorithms, Applications and Architectures, Elsevier Science Publishers, 1988. The SVD is the preferred method as it offers the following additional advantages over the QR decomposition: e It avoids the inversion of non-diagonal matrices. e It can be used to determine the condition number of B, a measure of performance of the given set of filters. The smaller the condition number the better the choice of filters. e |t allows a trade-off of an approximation error against SNR.

The SVD can be used to represent the (UxN) spectrai sampling matrix, B, as a product

B=UXLV’ where Uand V are JxN and NxN orthogonal matrices. An orthogonal ) matrix has the property that U' =U’ or UU =1. The matrix L is a

NxN diagonal matrix containing the singular values of B in descending order. Represented mathematically, x = diag{y,,...,7n} and

Yi2Z¥s..2yy 20.

The quantities of interest are expanded using the SVD, by

BB)" = | @wzv) (uzv)] = [very = VA?V' where A =X or alternatively

A = diagll/r,..... ry}

This reduces the matrix inversion to taking the reciprocals of each singular value. The signal reconstruction matrix (pseudo-inverse) is given by

G=[BB['B =VA*V'(UZV) = VA’ZU = VAU

For the transformed noise power, :

tr{Efun’}} = o* tr{B'B)} =o? tr{VA?V'} =o’ {A> V'V} } =o’ ir{a?}

The rotation rule for the trace has been used in the third step. If the trace is taken, then i=t Vi which is the sum of the detector noise power scaled by the square of the singular values. The largest singular value makes the smallest contribution to the sum and vice-versa. If the spread of singular values are small, the lowest transformed noise power is obtained. This characteristic is used to increase the SNR.

In order to further clarify this characteristic and its application, the SNR loss is defined as the ratio of SNR of the estimated spectrum to the SNR at the detector 24. A SNR loss lower bound is derived in terms of the condition number as

SMR, 1 wv

SNR, ~(cond(B)} #7} where the condition number, given as cond(B)= 11

Yn is an indication of transformed power. The derivation of the SNR loss lower bound is described in more detail below. If the condition number ’ 20 is large, the transformed noise power will be large and consequently the SNR loss will be high. By definition of the singular values, ¥,2¥,..-2)y 20 it is noted that for minimum SNR loss, it is required that cond(B)=1 or equal singular values with a minimum SNR loss of

Re

SNR, when B is orthogonal. Therefore, whatever the SNR is at the detector, one cannot expect it to get better after recovery, but it could get much worse if B is ill-conditioned. This is one of the reasons why the multiplex advantage of the spectrometer 10 is of importance since in practice cond(B)>1.

The effects of encoding by means of filters and the conversion process (collecting and focusing optics, spectral band-pass filter and the detector) on the signal to noise ratio (SNR) can be examined by factorising the B matrix as B=B_B., where

B. = 4,77, TAA diagr. (4 )R(4,),...,7.(4, )R(4,)} and in =

B= : : . 74.) 7c(Ay, J)

The NxN diagonal matrix B. describes the optical transmission and conversion of the optical signal with the exception of the spectral encoding filters. These are described separately in a JxN spectral encoding matrix B;. The condition number of B relates detector SNR to the estimate SNR (recovered signal). This result is now extended to the factorisation. It can be shown that cond(B)< cond(B;)cond(B,.).

When considering a diagonal matrix A eR™ such that

A =diag{a,,...,a,} and using the definitions hereinafter used in the derivation of SNR lower bound, then cond(A) =A],

As it is known that (see the derivation of SNR lower bound } hereinafter) [A], = maxi) and hence - 1

Ja, = max =——

SE [4 min|a, so that max|a, cond(A)=—1——. min|a, and using the result above for the condition number of a diagonal matrix, maxjz(4,)R(2 ) cond(B,)= — = minfz,.(2,)R(4)

Ideally cond(B)=1 can be achieved if both cond(B.) and cond(B,) are 1. The former requirement is equivalent to saying that the combined transmission of the collecting optics, spectral band-pass filter and detector must have a flat spectral response.

This result separates the SNR loss into two components: the encoding loss and the conversion loss. It can be used to determine the optimum detector operating bandwidth for a specified SNR using the condition number.

Discarding some of the smaller singular values may result in a reduction in transformed noise power, or a corresponding increase in SNR. The A —r smallest singuiar values are discarded in the following manner:

Xx, = diag{y,,....7,,0s...,0} and

A, =diagll/7,....1/7,.0,....0}.

The question is how well can the spectrum still be measured or estimated. Based on what is set out above, the recovered signal s,=G,d and G,=VAU'. The subscript indicates that only the r largest singular values are used. Expanding, §,=G,(Bs+v) =(VA,UXUZV' ls +G,v =VA,XV's+u, where u, =G,v.

If r=N, the signal would be perfectly reconstructed, since

AX=1 and then

Sy=VIVs+u, =s+u=s which illustrates that the results are consistent.

For r<N, an approximation error (bias) is made. It is necessary to quantify this error in order to trade it off against the SNR improvement. The mean square error can be used as a metric of the total error, defined as

MSE(r)= Ells -5,][s - 5) = Efg'V({I-A Z)V's} +o? wr{a?}

The derivation of the second step is described in more detail below.

The first term is the bias squared while the second term is the transformed noise variance. The approximation error is the result of not completely projecting the true spectrum into the orthogonal space defined by V. The approximation effectively reduces the rank of

B from N to r.

For system level design of a spectrometer such as the spectrometer 10, it is important to have some measure of computational, memory and data rate requirements for an image of size X xY pixels with N spectral bands. Only the signal recovery part of the algorithm is considered for real-time implementation. If it is assumed that the area detector 24 has a frame rate of f;, then for a full data cube: e Every J/f; seconds a hyper-spectral image (HSI) is captured. e A total of JNXY floating point multiply-accumulate (MA) operations are required. e The computational rate is NXYf, floating point operations per second (FLOPS), counting each MA as one operation. e The area detector data rate is XYf, words per second. e The data cube occupies 4NXY bytes where it is assumed a single precision floating point and that intermediate results are stored in the same memory as the final result. e The hyper-spectral camera data rate is 4NX1f,/J bytes/second.

If a typical example with the parameters indicated in

Table 4 is considered, the requirements in terms of processing rates ) 25 and memory shown in Table 5 are obtained, which are quite feasible with contemporary digital signal processors and memory technology.

Table 4 Example parameters.

Parameter | Value

EN

Table 5 Performance for example parameters — point and image spectrometers. (x,r)=(1,1) | (x.Y) = (512,512)

Total MA operations 5000 FLOP 1310 MFLOP bli Cel

Detector/Area detector data | 25 6.6 rate words/second | Megawords/second

HSI data rate 50 B/s 12.5 MB/s

It is preferred to select or design the filter transmissions of the filters of the spectrometer 10 for each measurement to maximise the signal to noise ratio over wavelength. The sampling vectors b, are a non-linear function of the filter transmissions. The noise is transformed by G, which is a function of the matrix B. The optimisation problem can be stated mathematically as . Eis’s

A, j)= < 7c(t.)) amg 208 or in words: find the set of filters or filter transmissions : from the space ®, and hence B, so as to maximise the signal (including any wavelength dependent losses} to noise ratio of the estimate.

Clearly the space ® will be large and ways of reducing this space based on physical and mathematical requirements and performance requirements should be implemented. A first requirement is that the transmission is constrained to 0<7.(4,/)<1. The next requirement is that the product B'B must be invertible. This requires that B have at least rank(B)=N. In other words, N linearly independent vectors are needed as a basis — all the row vectors can then be formed from linear combinations of these. With the SVD algorithm described above, this requirement can be relaxed from N to r.

Furthermore, if an orthogonal matrix B can be found, then cond(B)=1, and the abovementioned optimisation problem is solved. As a result of the constraints on the transmission functions, it is only possible to approach the minimum, and not to achieve cond(B)=1.

When considering the implementation of the filter transmissions using optical thin film technology, there are two routes for determining the filter transmissions: : 25 e Select a set of orthogonal functions that maximise the estimate

SNR and attempt to implement these or e Optimise the thin film design parameters so as to maximise the estimate SNR directly.

in the first case it is assumed that the implementation technology has an underlying parametric form that can produce the desired transmission functions. This may only lead to an approximation of the required transmission functions. It is clear that from an implementation point of view functions with discontinuities should be avoided. In the second case, the filter transmission is not specified explicitly. This approach requires an intimate knowledge of thin film theory and is not discussed in any further detail.

A set of functions in terms of wavelength is presented hereunder. It should however be noted that these could just as well have been formulated in terms of wavenumber. In fact, it might be desirable to do so from the point of view of thin film design.

The continuous spectrum is sampled at N discrete points, each band of width AA. It is necessary to be able to reconstruct the original signal from the sampled version. Spectral frequency is first defined as the number of cycles of a certain quantity (transmission, radiance, etc.) occurring over a given wavelength band. Note that is not necessarily the same as the wavenumber for a given wavelength. The spectral sampling frequency is thus f;=1/A4.

From Nyquist’s theorem, it is known that in order to meet the objective of reconstructing the spectrum from the sampled version, there should not be any signal component above half the spectral sampling frequency f<f;/2.

For simplicity it is assumed that J =N. There are two cases:

e One filter per measurement (High performance).

An encoding transmission of the form 7A, J) = a+ Pp(A, j) then exists, where 0<a <1 and 0< #<1 and ¢(4,) is the function to be determined.

From Fourier theory it is known that sine and cosine functions can be used. Furthermore, it is possible to construct a set of functions, some of which are orthogonal, using only sine or only cosine or both. The latter two are used because they minimise the number of transmission variations with respect to wavelength. The first set of cosine functions is , To. , $01) = cof Zii-1) j=L...,J.

N

. a N

For the mixed set, the definition of M = floor Ek The set of orthogonal functions is given by 27 . cos| Z=i(j 1) j=l. ,M+1 . N #4, 7) = 27 sin 2-0-1) j=M+2,..,N

N

These functions represent a set of orthogonal functions that can be used and are by no means the only ones. There may be other types of functions, which are also suitable such as wavelets. The sine and cosine set forms a complete orthogonal space. If only the cosines from this set are used, sine functions cannot be represented since these are orthogonal. This means that additional higher frequency cosines must : be used to achieve the same SNR - this is typically a limitation of

FTIRs. It is to be noted that the cosine set are at half the frequency when compared to the cosines in the mixed set.

e One filter combination per measurement (Reduced cost).

A combined transmission of the from (A. j)= A VT aA, ) exists in this situation, where

Tey (Amy) =a+ fe(4, m,) ,

Tomah, ) = a+ Py (A,m,), 0<a<l and 02 4<1 and #,(A,m,), é,(1,m,) are the functions to be determined.

The following set of functions é (4, ,m;)= cof 22 im, +6, (m) ,(4,,m,)= cof 22 im, —1)+8,(m, ) are not orthogonal but can yield a fairly low condition number if the phase vectors are chosen such that lo |-eminfela’}:

This last condition is equivalent to maximising the multiplex advantage. It should be noted that for the double filter wheel more measurements than spectral bins are required, i.e., J > N. lt is possible to meet this requirement and still have less than N filters in total.

The encoding functions above represent transmission targets for the design of optical thin film filters. In practice, one will be able to achieve transmissions that are close but not exactly the same. In fact, the transmissions will contain spectral frequency components at the desired frequencies and other additional frequency components or harmonics. These harmonics cause aliasing. This problem can be : overcome by spectral over-sampling. Such over-sampling may also be required if the approximations in the physical signal model and the spectrum estimation optimisation problem set out hereinbefore are not sufficiently accurate.

The B matrix can be over-sampled in the ratio 1: Q by inserting 0-1 evenly spaced points in between the original points at wavelength intervals of = This spectrally over-sampled matrix is denoted as

B, and is a JxN, matrix where N, =QN and J<N,. The wavelength increment changes from AA to 4 for B,. This over- sampling is based on measured values of the optics, encoding filters and responsivity and is thus an extension of the calibration process that would have been required at N points for B anyway.

Using the over-sampled system matrix and the N, x1 over-sampled signal vector s,, the detector output can be more accurately calculated with d=B,s,+v.

The spectrum can be recovered using 5, =G,d where the reconstruction matrix is given by

G,=(B,B,)'B,. (N,xJ matrix)

This is calculated in a similar way to the calculation of B’B described ) earlier, although the SVD matrices may be different sizes. It should be pointed out that this is an under-determined system and in fact the mean square error described earlier in relation to the SVD method is relevant here. Clearly, rank(B,)<J and forms a “reduced” rank system with ==. . Thus there will be some approximation error. but also some gain in SNR - an indirect consequence of over-sampling.

In order to get back to the original N points it will be necessary to sub- sample §,. This can be done, with reduced computational time, by extracting every Q rows of G, to form a new matrix. This will also result in reduced approximation error.

In practice the transmission of the optics of a spectrometer such as the spectrometer 10 is not known exactly, and the broadband optical filter transmissions and the detector responsivity have some variation. Three forms of calibration can be distinguished: e Dark offset calibration. e Coarse or model calibration. In this case it is assumed that the models are very inaccurate models or that the spectral sampling matrix B, is unknown. e Fine calibration. The purpose of this calibration is to make the instrument traceable to some standard. It assumes that the spectral sampling matrix is known approximately.

Dark Offset Calibration: The purpose of this calibration is to determine the offset introduced by the detector 24, the amplifier 26, filters and the A/D converter 30, with zero spectral radiance. This can be expressed as follows:

Set s =0 and average over a large number of detector measurements — this will be referred to as the dark offset d,. This can also be written as a vector do d, =|: (Ix 1} .

Model Calibration: The aim of model calibration is to characterise the entire system in a single model — the spectral sampling matrix B given a specific instrument. Clearly B contains JN unknowns. However, since control is exercised over the source during calibration, the problem can be split into J independent problems. The task at hand is then to determine b, given a number of known sources whose spectrum is known.

Several reference sources s, for k=1,...,K, with known spectrum are required. At least as many sources are required as there are spectral bands i.e. K=N. A source matrix S=[s, -- sg] (NxK matrix) then exist.

From the physical signal model, for a given filter combination j, dy =bs, +v, and in matrix notation d=bS+v where the detector measurement vector and noise vector are (Kx1) vectors. The spectral sampling vector is b,=S(SS)d.

The requirement for the reference sources s,, is only that each be different spectrally, with known spectrum. A monochromater or some other means could generate these. There is not a specific need for s, to be narrow band signal.

Fine Calibration: It is possible that the “reference sources” used in the previous section may not be accurately calibrated, and therefore a method for fine calibration may be needed.

A traceable reference source s=s, can be used. Next, an averaged estimate of the spectrum § is obtained, assuming a spectral correction of the form 5,=x5,, Vi=L..,N. where § is the calibrated estimate and the calibration factor is given by 5;

The derivation of SNR lower bound is now described. The following metrics are helpful with reference to G.H. Golub and C.F von

Loan, Matrix Computations, John Hopkins University Press, 1983:

N 12 * a Euclidian norm fx, [3] for xeR", i=l a matrix norm [A], BL , for AcR™ x=0 Ix],

The relations . x'x = |x|; and e A generalised form of the Cauchy-Schwartz inequality, lax, <{Al kf, are also used.

As set out above, SNR at the detector is

SNR, = BBs _ [Bd],

Ev EM; and the estimate SNR is ovr, 55 _ Ish lau} Ej);

The SNR loss is defined as g SNR. _ Ish EM. Is EM, 1 2 2 2 2 2 2 2 2

SNR, Elul;, [Bs], GI, Elv[, |BL,Is; IGILIBl;

The condition number for a rectangular matrix is defined as cond(B) = B,J,

Simplifying, IB], =[U=V, = |=], =» and similarly for , 1

Gl, = [vat = [A], =—.

Yn so that cond(B)=-2L.

Yn

So, clearly the SNR loss is bounded by the reciprocal of the square of the condition number

SNRg, 1 _ 7

SNR, ~ (cond(B)f #7

By definition of the singular values,

V2 2¥y 20 it is clear that for minimum SNR loss, it is required that cond(B)=1 or equal singular values. . If it is assumed that B=Q an orthogonal matrix i.e. QQ=I and

Q'=Q', then the minimum SNR loss is ‘ 2 2 2 oo Se IE, Ie A ' 2 2 *

SNE, E|QV; [Qs], Ef]; |],

The derivation of mean square error as a function of the : rank r is now described. The difference s—§ =s~(VA,ZV's+u,) =(I-VAZV')}-u, =V(I-A,Z)V's—u, can be considered and then the following product, substituting u, = VA, Uv in the cross-terms, can be formed (s- 3.) (s-5,)=[sVA-A ZV -u][V(I- AZIV's—u,] =sV{I-AZfVs—sV(I-AZ)Vu, —u VI-A Z)V's+uu, =sV(I-A,Z)Vs-sV({I-AZAUv-vUA, (I-A Z)Vs+uu,

The middle two terms are zero because

A, E=diag{l,...,1,0,...,0}, (r ‘'1’'sand N-r 'O's) and therefore

I-A XT =diag,....01,...1}, (r ‘O'sand N-r ‘1's.

Therefore (I-AA, =0.

Taking the expectation of the product,

MSE()=Efs-3,) (5-3, )}= EV(I- A, E)V's}+ Efuln,} and, as

Efa'n,}=tr{E{u,u }= o> wr{A? } and substituting, the final result is MSE(r)=EEV(I-AX)V's}+o? w{a2}.

The QR decomposition for calculating BB)" is now described. The spectral sampling matrix can be written as a product of two matrices

B=QU ‘ where Q is a JxJ unitary matrix and Uis a JxN upper triangular matrix. This is called the QR decomposition. In fact, the latter matrix can be partitioned as wf] 0 if rank(B)=N so that U, is a NxN upper triangular matrix. The matrix product can be expressed as

BB=U'QQU=UU=UU, and its inverse as

Lo ER-@u) - 7 (o7)

Since U,is upper triangular, its inverse can be more easily calculated.

EXAMPLE 1

A simulated example with reference to the spectrometer 10 using commercially available filters from Schott Glaswerke is provided. The example illustrates the general concept of the invention and does not provide an optimum design for the spectrometer 10.

Hence, some parameters in Table 6 were chosen arbitrarily. The transmission of the filters on each wheel 16, 18 is shown in Figures 6 and 7, while Figure 8 is the combined transmission of the sixty-four filter combinations. The approach described above has been used to recover the signal. There has been no attempt to use the orthogonality } conditions described - this would result in better performance without any approximation error. In the recovered signal shown in Figure 10, r = 26, roughly corresponding to the minimum of the mean square error in Figure 8. This shows that even if a less than optimal set of filters Is used, the spectrum can still be estimated.

Table 6 Parameters used in the example.

J Number of measurements used to | 64

MI ee

EC LC LS LI

I iS J

I li Coo a

EI li a

EI eR UR boo wo 1 le LL A

EXAMPLE 2

In this example one thin film filter is used per measurement. The cosine function set described earlier has been used as a design target for the thin film transmissions. The parameters used for this spectrometer are listed in Table 67. The Essential Macleod V8.03e software was used to design the thin film filters. ‘

Table 7 Parameters used in the example ’ Number of spectral bands.

Number of measurements used to 20 . estimate the spectrum.

Upper and lower operational wavelength. | 600-800nm

Wavelength resolution.

Sampling frequency 100c/pm

Over-sampling factor

Transmission of collecting optics.

Transmission of focusing optics.

Spectral band-pass filter.

Radiometer throughput.

R(A) Detector responsivity (V/W).

Detector Pre-amp voltage gain (V/V). a | Transmission function offset 05

The condition number of the over-sampled matrix is cond(B,) =27.3 for this example. The reference SNR, SNR, = 118.2 while the recovered signal to mean square error is 533.7 for the results shown in Figure 15, with a resulting gain of 4.5. This is related to SNR; /SNR, but also includes the approximation error or bias term of the estimate.

The predicted SNR gain (SNR; /SNR, ) is 3.1. The average signal to mean square error gain over 1000 runs was 3.3 with a standard deviation of 1.2.

The Instrument Line Shape (ILS) is the response of a spectrometer to a spectral impulse or dirac-delta. It gives an indication of leakage into adjacent spectral bins caused by making only a small number of measurements. It is normal to define a ripple ratio as the ratio of the maximum side-lobe amplitude to the main-lobe amplitude. The ILS for the spectrometer of this example is shown in Figure 16. The non-zero a ~- width of the input spectrum, A. (at 715nm), has not been corrected for. The main lobe to side lobe ratio is 6.2 without any window functions.

While 21 filters were designed, only the first three and the last filter are shown in the drawings. The design for the third filter is given in

Table 8. The design consists of the chemicals, and corresponding thickness, that are deposited on a substrate.

Table 8. Example of thin film design for j=3.

Layer Material Thickness (nm) incident Medium Air 25102 | 205.06 oz | tore fgsio2 | 4rd

La I tqioz | 8138 148102 | 351.61

Substrate BK?

Advantages of the spectrometer of the invention, as illustrated, include that it has a multiplex advantage and a good signal to noise ratio as a result of the high throughput of the broadband filters employed to obtain i measurements of an incident spectrum. It is also expected that the spectrometer of the invention, as illustrated, will be much less costly to manufacture and more robust than conventional spectrometers such as

FTIR spectrometers, since it contains no high tolerance components.

Furthermore, there are no field of view dependent wavenumber shifts as with FTIR spectrometers.

Claims (18)

CLAIMS:

1. A spectrometer which includes a filter arrangement, in addition to any optional filters for limiting an operational wavelength range of the spectrometer, the filter arrangement comprising a plurality of broadband optical filters or broadband optical filter areas each of known transmission and being located or selectively locatable in a path of collected incident spectral radiation; at least one detector arranged to measure the spectral radiation passing through at least one of the broadband filters located in the path of collected incident spectral radiation; and signal-processing means for recovering the spectrum of the collected spectral radiation from measurements by the detector.

2. A spectrometer as claimed in claim 1, in which the filter arrangement includes a filter set with a plurality of unique and individual broadband filters or filter areas.

3. A spectrometer as claimed in claim 2, in which all the broadband filters or filter areas are linearly independent.

4, A spectrometer as claimed in claim 2 or claim 3, in which the filter arrangement includes at least two filter sets arranged one in front of the other to provide broadband filter combinations.

5. A spectrometer as claimed in claim 4, in which each filter set is in the form of a filter wheel.

6. A spectrometer as claimed in claim 4 or claim 5, which ‘ includes locating means for selectively locating the broadband filters or filter areas in the path of collected incident spectral radiation, the locating means being configured to locate one filter or filter area on one filter set in the path of collected incident spectral radiation whilst successively locating each filter or filter area of the other filter set in the path of collected incident spectral radiation, thereby forming a plurality of broadband filter combinations.

7. A spectrometer as claimed in any one of claims 4 to © inclusive, in which each broadband filter combination is unique and individual and each broadband filter combination is linearly independent of every other broadband filter combination.

8. A spectrometer as claimed in any one of claims 4 to 7 inclusive, which is configured to measure N wavelength bands and which includes J broadband filters or filter areas or J filter or filter area combinations, where J > N > 2, the filter arrangement including only two filter sets, each filter set comprising M broadband filters or filter areas where J = M2

9. A spectrometer as claimed in any one of the preceding claims, in which the filter arrangement is configured to allow the spectrometer of the invention to operate in the visual and near infrared spectrums.

10. A spectrometer as claimed in any one of the preceding claims, in which the filters or filter areas are thin film filters or absorption filters.

11. A spectrometer as claimed in any one of the preceding claims, which includes a detector associated with each broadband filter or filter area or with each filter or filter area combination which can exist simultaneously, allowing simultaneous sampling of each filter or filter area or each actual filter or filter area combination.

12. A spectrometer as claimed in any one of the preceding claims, in which the signal-processing means is configured, for J measured digital signals received by it and each digital signal corresponding to a measurement taken for a particular filter or filter combination, to arrange the J measured digital signals in a vector d, where the vector d=Bs+vand B is a matrix whose row vectors represent the effects of any hardware components, used to obtain the measured digital signals, on the measured digital signals, s is the incident spectrum and v is a noise vector, and to multiply a signal recovery matrix G with the vector dof the measured digital signals, to provide the measured incident spectrum §, i.e. § = G d where G =VAA' or the pseudo inverse of B; A, =diagfl/y,,...,1/7,,0,...,0}, the subscript r indicating that only the r largest singular values are used; and B=UXV' , the singular value decomposition of B, where Uand V are JxN and NxN orthogonal matrices, the matrix X£ is a NxN diagonal matrix containing the singular values of B in descending order, i.e. Z=diagly,,....¥y}, 7,2%,...2%x 20, and r is an integer in the range 1<r < N selected to minimize the measurement error.

13. A method of obtaining a measurement of an incident ‘ spectrum s over N wavelength bands or zones, the method including filtering the incident spectrum s through J optical broadband filters or filter combinations to provide J individual filtered spectrums; converting the J filtered spectrums to J measured analogue electrical signals; converting the J measured analogue electrical signals to a vector d of J measured digital signals, where the vector d=Bs+v and B is a matrix whose row vectors represent the effects of any hardware components, used to obtain the measured digital signals, on the measured digital signals, and v is a noise vector; and multiplying a signal recovery matrix G with the vector d of the measured digital signals, to provide the measured incident spectrum §,

i.e. S = Gd where G = VA/U' or the pseudo inverse of B; A, =diagfl/y, ,...,1/7,,0,...,0}, the subscript r indicating that only the r largest singular values are used; and B=UXV’, the singular value decomposition of B, where Uand V are JxN and NxN orthogonal matrices, the matrix X is a NxN diagonal matrix containing the singular values of B in descending order, i.e. T =diagly,,.... 7x} 7,27,...2%y 20, and r is an integer in the range 1<r < N selected to minimize the measurement error.

14. A method as claimed in claim 13, in which the matrix B includes values or elements in its row vectors reflecting the effect of any hardware components, used to obtain the measured digital signals, on the measured digital signals, the effect being selected from the group of

SR WO 02/088646 PCT/IB02/01323 ; effects consisting of a detector preamplifier voltage gain, the transmission of any collecting optics, the transmission of any focusing : optics, the throughput of any optical elements, the transmission of any filter or filters for limiting the wavelength range of the incident : spectrum s , the transmission of the broadband filters or filter combinations, a detector responsivity, or two or more of these.

15. A spectrometer which includes a filter arrangement, in addition to any optional filters for limiting an operational wavelength range of the spectrometer, the filter arrangement comprising a plurality - of broadband optical filters or broadband optical filter areas each of : known transmission and being located or selectively locatable in a path of collected incident spectral radiation.

16. A spectrometer as claimed in claim 1 or claim 15, substantially as herein described and illustrated.

17. A method of obtaining a measurement of an incident spectrum, as claimed in claim 13, substantially as herein described and

20 . illustrated. -

18. A new spectrometer or a new method of obtaining a measurement of an incident spectrum, substantially as herein described.