WO2018065756A1

WO2018065756A1 - Apparatus and method for removing noise from measured data

Info

Publication number: WO2018065756A1
Application number: PCT/GB2017/052923
Authority: WO
Inventors: Mark EDGERTON
Original assignee: Renishaw PLC
Current assignee: Renishaw PLC
Priority date: 2016-10-03
Filing date: 2017-09-29
Publication date: 2018-04-12
Anticipated expiration: 2019-04-03
Also published as: GB2568190A; GB201616787D0; GB201903332D0

Abstract

This invention concerns a method and apparatus for removing noise from measured data, such as spectral data. The method comprises receiving (101) measured data comprising a plurality of sequential observations of discrete variables, carrying out (102) a method of blind signal separation to separate the measured data into a plurality of components, generating (105) at least one modified component by removing at least a portion of a noise signal present in at least one of the components and generating (107) modified data from ones of the components including the at least one modified component.

Description

APPARATUS AND METHOD FOR REMOVING NOISE FROM

MEASURED DATA

Field of the Invention

This invention relates to measurement apparatus and methods. It is particularly useful in spectroscopy, such as Raman, narrow-line photoluminescence, fluorescence, cathode-luminescence, UV visible (UV Vis), nuclear magnetic resonance ( MR), mid infra-red (mid-IR) or near infra-red (NIR) spectroscopy.

Background to the Invention

The Raman Effect is the inelastic scattering of light by a sample. In Raman Spectroscopy, a sample is irradiated by monochromatic laser light and the scattered light is then dispersed by a dispersive device, such as a diffraction grating, e.g. in a monochromator, to generate a spectrum called a Raman spectrum. The Raman spectrum is detected by a detector such as a charge- coupled device (CCD). Examples of Raman spectroscopy apparatus are known from US Patents Nos. 5,442,438 and 5,510,894, which are incorporated herein by reference.

Different chemical compounds have different characteristic Raman spectra. Accordingly, the Raman effect can be used to analyse chemical compounds present in a sample.

The detected spectrum comprises the Raman spectrum together with noise generated by the system and the surrounding environment. In particular, electronic switching within the Raman apparatus and in surrounding electronic equipment can result in noise in the detected spectrum.

It is known to provide Raman systems for mapping a sample, for example as described in PCT/GB2016/050630. As the speed of mapping increases, a ratio of noise to the Raman signal increases. It is desirable to provide a system for removing noise from the detected spectrum/spectra. The accurate removal of noise from the detected spectrum/spectra is particularly desirable in high speed mapping applications which have a high noise to Raman signal ratio.

Summary of Invention

According to a first aspect of the invention there is provided a method of removing noise from measured data, the method comprising receiving measured data comprising a plurality of sequential observations of discrete variables, carrying out a method of blind signal separation (also referred to as "blind source separation") to separate the measured data into a plurality of components, generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and generating modified data from ones of the components including the at least one modified component.

The noise may be more easily identified and removed from components of the blind signal separation method than the raw measured data, as such methods attempt to separate a set of source signals, such as a source of noise, from a set of mixed signals, e.g. a plurality of spectra. This may be particularly the case for sources of fixed pattern and burst noise, which may have readily identifiable noise signatures. For example, fixed pattern noise is likely to result in at least one component describing a sinusoid and/or peaks at certain frequencies in the frequency domain. It has been found, particularly for systems with a high noise to signal ratio, a component of the blind signal separation method can comprise both noise signals and useful information pertaining to the property being measured (referred to herein as the "measurement signal"). Therefore, to rebuild the measured data without the components that contain noise can result in portions of the measurement signal being lost. It has been found that by modifying, rather than removing, a component identified as containing a noise signal, loss of useful portions of the measurement signal can be mitigated whilst noise is reduced. Furthermore, the method of removing noise from the measured data is scalable with readout speed. In particular, a position of the measurement signal in the frequency domain is dependent on the spectral resolution of the spectroscopy/metrology apparatus and independent of the readout speed.

In one embodiment, the measured data is spectral data comprising a plurality of spectra captured sequentially using a spectroscopy system. In such an embodiment, the method comprises carrying out the method of blind signal separation to separate the spectral data into the plurality of components, generating the at least one modified component by removing at least the portion of the noise signal present in at least one of the components and generating modified spectral data from ones of the components including the at least one modified component. The plurality of spectra may be obtained from multiple points on a sample (so called mapping) and/or from a plurality of samples, for example which are sequentially analysed by the spectrometer as part of a manufacturing process.

In another embodiment, the measured data may be data from a metrology apparatus, for example a coordinate positioning machine (such as a machine tool or coordinate measuring machine (CMM)), a measurement probe (such as a touch trigger probe, scanning probe or optical probe), a rotary probe head, a sensor of an additive manufacturing (AM) machine (for example, sensors in the AM machine for monitoring a melt pool through an optical train of an optical scanner and/or acoustic sensors), an encoder system (such as an interferometric, magnetic or optical encoder system), a spatial laser measurement system and/or the like.

The method of blind signal separation may comprise principal component analysis. Principal component analysis generates a set of principal components ordered such that the first principal component describes a coordinate that gives rise to the greatest variance in the measured data, with each succeeding component having the next highest possible variance under the constraint that it is orthogonal (independent) to the preceding components. For measured data comprising a high noise to measurement signal ratio, a noise signal can be present in a high order principal component, as a significant proportion of the variance in the measured data can be apportioned to noise. A high order principal component is likely to also contain portions of the measurement signal valuable for later analysis. The method may comprise modifying one of the principal components to remove at least a portion of the noise signal identified as present in the principal component and generating modified data from ones of the principal components including the modified principal component. Modification of the principal component may comprise modifying loadings of the principal components identified as comprising noise.

Other suitable methods of blind signal separation may be used, such as singular value decomposition (SVD), factor analysis, canonical correlation analysis, independent component analysis, multivariate curve resolution alternating least squares (MCR-ALS), and non-negative matrix factorization.

The noise signature(s) may be particular patterns, such as sinusoids, identifiable in the components, certain known operating frequencies of electronic components in the spectroscopy apparatus/metrology apparatus/surrounding environment, and/or signatures identified by a user. The signatures may be determined empirically, for example by experiments carried out by a user, the user inputting the determined noise signatures into an apparatus for carrying out the method, or the apparatus may be pre-programmed with typical noise signatures, for example as determined from theory and/or operating frequencies set for electronic components of the spectroscopy apparatus/metrology apparatus.

The known operating frequency may be an operating frequency of a switched- mode power supply and/or a camera that captures the measured data, such as a readout frequency. If possible, devices of the spectroscopy/metrology apparatus, which result in noise in the spectral data, may be arranged to operate at frequencies away from the frequencies at which the measurement signal appears in the components of the blind signal separation in the frequency domain. This may simplify removal of the noise signal and reduce the risk of useful measurement information, such as spectral signals, being removed. Ideally, the frequencies should be sufficiently separated such that the noise and the measurement signals do not overlap in the frequency domain. The devices that are a source for noise in the spectral data may be synchronised with the readout from the camera such that the fixed pattern noise is at the same phase as the readout. This may reduce the number of loadings in a component at which a given noise signal is present. Generating the modified component may comprise mathematically transforming the component to remove the identified noise. More than one component may be modified to remove noise.

Modifying the component may comprise generating the components in a frequency-domain using a time to frequency transform, such as a Fourier transform, and modifying the component in the frequency-domain to remove at least a portion of the noise signal. It will be understood that "time" and "time- domain" as used herein is not limited to data that is a function of time but refers to the domain of the original function before transformation into frequencies contained within the original data by the time to frequency transform.

The step of modifying the components may comprise removing peaks attributable to noise from the components in the frequency-domain. A "peak-flattening" method by used to modify the components in the frequency-domain. For example, data points of peaks of the component(s) in the frequency domain may be removed or, preferably, replaced with substitute data points interpolated from data of the component.

The method may comprise fitting an analytical curve, such as an nth order polynomial, typically a 4^th to 6^th order polynomial, or set of spline curves, particularly cubic splines, to the data points of the component in the frequency domain and removing ones of the data points that are more than an allowable deviation above the fitted analytical curve. The allowable deviation may be a multiple of an average deviation of the data of the component in the frequency- domain from the fitted analytical curve, such as a multiple of a root mean square (RMS) of the data. The multiple may be greater than 1 and typically less than 3. The method may comprise replacing the removed data points with substitute data points, for example substitute data points based upon the analytical curve.

The peak flattening method may comprise carrying out an iterative process, in which each iteration comprises fitting an analytical curve to reference data, determining the allowable deviation of the reference data from the analytical curve and removing data points of the reference data or of the component in the frequency-domain that are more than the allowable deviation above the analytical curve to provide the reference data for the next iteration until a termination criterion is met, wherein the reference data is initially the data of the component in the frequency-domain. The reference data of the final iteration may be used as the modified component in the frequency-domain. The iterative process may be carried out on each component identified.

Fitting of the spline curve may comprise identifying a location of the anchors (knots). Identification of the locations of the anchors may be based upon the reference data or component. The locations of the anchors for each iteration may be identified automatically using an algorithm. The number of anchor points predefined by a user. The number of anchor points may be predefined based on an expected width of noise peaks in the component.

The polynomial/cubic splines tends to be fitted much more closely to the slowly varying spread of frequencies, such as those generated by non-repeating measurement peaks, for example spectral peaks, rather than the sudden spikes of frequencies generated by fixed pattern or burst noise. Accordingly, clipping data points of the reference data or component that are more than the allowable deviation above the analytical curve may remove the sharp noise peaks from the reference data whilst retaining the more slowly varying data attributable to measurement information, such as spectral signals.

To generate the modified measured data, the components including the at least one modified component may be transformed back into the time domain using a reverse time to frequency transform and the modified measured data built from the components in the time domain.

Alternatively or additionally, generating the modified component may comprise modifying the component in the time-domain. For example, a sinusoid at the same frequency and phase of the noise signal identified as present in the component may be subtracted from the component in the time domain to remove the noise signal.

The method may comprise receiving a user input identifying portions of the components attributable to noise.

The method may comprise an iterative process in which the method as described above is carried out in each iteration, wherein the measured data for each iteration, except for a first iteration, is the modified measured data generated in a previous iteration.

According to a second aspect of the invention there is provided a method of identifying noise in measured data, the method comprising receiving measured data comprising a plurality of sequential observations of discrete variables, carrying out a method of blind signal separation to separate the measured data into a plurality of components, identifying a noise signal in one or more of the components and generating an output identifying a characteristic of the noise signal. The measured data may be spectral data comprising a plurality of spectra captured sequentially using a spectroscopy apparatus. The characteristic may be a frequency of the noise signal. The user may be then identify a source of noise from the characteristic.

The method may comprise generating further measured data using the spectroscopy/metrology apparatus after the spectroscopy/metrology apparatus and/or localized environment to the spectroscopy/metrology apparatus has been modified to remove a source of the noise signal based upon the characteristic. For example, the identified characteristic of the noise signal may enable the user to search for and/or shield the spectroscopy/metrology apparatus from or change electronics that operate at a frequency commensurate with the noise signal. Shielding the spectroscopy/metrology apparatus may comprise relocating the source of noise. Changing the electronics may comprise changing an operating/switching frequency of the electronics or switching off the electronics.

The method may comprise generating further measured data using the spectroscopy/metrology apparatus after the spectroscopy/metrology apparatus has been modified to change a resolution, such as spectral resolution, based upon the identified noise signal(s). For example, if a noise signal is found to be located in a component in the frequency domain amongst the measurement signal, the resolution of the measured data may be increased (thus decreasing the range (spectral width) of the measured data) such that the measured signal occupies a smaller region of the component in the frequency domain that is free from the noise signal. Conversely, the resolution of the measured data may be decreased (and the range increased) if the noise signal is identified as being spaced away from the measurement signal.

The method used for generating the modified measured data may depend on the width and/or position of the noise signal to be removed. Broadband noise may require modification of the localized environment to the spectroscopy/metrology apparatus, whereas narrow band noise may be removed using post-processing techniques such as the peak flattening or filtering techniques as described above. The method may comprise identifying the noise signal from the components in frequency-domain. The noise signal may be identified based upon peaks in the frequency domain having a defined characteristic, such as a peak having an amplitude above a certain threshold and/or having a defined width.

According to a third aspect of the invention there is provided a data carrier having stored thereon instructions, which, when executed by a processor, cause the processor to carry out the methods of the first and/or second aspects of the invention.

The data carrier may be a non-transient data carrier, such as volatile memory, eg RAM, non-volatile memory, eg ROM, flash memory and data storage devices, such as hard discs, optical discs, or a transient data carrier, such as an electronic or optical signal.

According to a fourth aspect of the invention there is provided apparatus comprising a processor, wherein the processor is arranged to carry out the methods of the first and/or second aspects of the invention. According to a fifth aspect of the invention there is provided spectroscopy apparatus comprising a light source for illuminating one or more samples, collection optics for collecting light transmitted from the one or more samples when illuminated by the light source, a detector for generating signals based upon the light collected by the collection optics, wherein readout of signals from the detector is synchronised with a frequency of a source of noise in the signal.

The source of noise may be a switched power supply of the spectroscopy apparatus. According to a sixth aspect of the invention there is provided apparatus for removing noise from measured data comprising means for receiving measured data comprising a plurality of sequential observations of discrete variables, means for carrying out a method of blind signal separation to separate the measured data into a plurality of components, means for generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and means for generating modified measured data from ones of the components including the at least one modified component.

According to a seventh aspect of the invention there is provided apparatus for removing noise from measured data comprising a receiving module for receiving measured data comprising a plurality of sequential observations of discrete variables, an analytical module for carrying out a method of blind signal separation to separate the measured data into a plurality of components, a component modification module for generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and a modified measured data module for generating modified measured data from ones of the components including the at least one modified component.

According to an eighth aspect of the invention there is provided apparatus for identifying noise in measured data comprising means for receiving measured data comprising a plurality of sequential observations of discrete variables, means for carrying out a method of blind signal separation to separate the measured data into a plurality of components and identifying a noise signal in one or more of the components and means for generating an output identifying a characteristic of the identified noise signal.

According to a ninth aspect of the invention there is provided apparatus for identifying noise in measured data comprising a receiving module for receiving measured data comprising a plurality of sequential observations of discrete variables, an analytical module for carrying out a method of blind signal separation to separate the measured data into a plurality of components and for identifying a noise signal in one or more of the components and an output module for generating an output identifying a characteristic of the identified noise signal. The modified measured data, such as spectral data, may be used for identifying structural information of samples, for example structural information which gives rise to low level spectral signals.

For example, the samples may be samples used in as bioprocessing, protein/peptide structural analysis, microbiology, drug delivery in vitro and in vivo, cancer research/pathology, redox biology, regenerative medicine, ageing and neurodegenerative diseases, biofuel and agricultural research, lipidomics, metabolomics, developmental biology, reproductive biology and virology. In such analysis the target of interest may be present in very low quantities and therefore give rise to low level spectral signals. Accordingly, removal of noise from the spectral data may be required to identify the spectral signals of the target. The spectral analysis of the samples may be used in the diagnosis of patients.

The samples may include 2D materials and nanotechnology, including graphene, carbon nanotubes (CNT), graphite, diamond, and diamond-like carbon (DLC), MoS2, hBN, and WSe2. The method may be used in the control of processes used to manufacture such materials. For example, the number of graphene layers, and their defects, doping and strain, diamond Like Carbon (DLC) thickness, hybridised composition (sp2 and sp3), carbon nanotube (CNT) diameter and functionalisation, diamond stress, purity and origin (synthetic or natural), the properties of C60 and other fullerenes and the structural composition of amorphous carbons. Such materials consist of single, or just a few, atomic layers such that high sensitivity of the spectroscopy system, and therefore, spectral data with a low noise signal, is required.

The samples may include photovoltaics (PVs), semiconductors and/or catalysis materials. The PVs may include Si-based, CIGS, CdTe, organics, III-Vs, PV material. The structural information that may be determined is alloy fraction, electronic efficiency, strain/stress, thin film thickness, crystal structure type and orientation, crystal quality, sample uniformity and purity (e.g. defects and contaminants). The spectroscopy system may be used to map the samples to determine how the material properties vary across the sample, for example entire semiconductor wafers. As the speed of mapping increases, the noise to spectral signal ratio increases, requiring a method of noise removal for high sensitivity in the mapping routine. This structural information may be used is a feedback mechanism in a manufacturing process.

Most catalysts are fundamentally unstable in air. The spectral data of the atmosphere of suitable cells for storing the catalysts can be used as part of a feedback mechanism to ensure that the catalysts are maintained in a suitable environment. For example, temperature, humidity, and gaseous environment can be controlled within the cell.

The samples may be materials used in battery technology, such as carbon materials, metal oxides, polymers and electrolytes. Mapping and imaging enables the study of the distribution of materials on electrode surfaces, or across cross- sections. As the speed of mapping increases, the noise to spectral signal ratio increases, requiring a method of noise removal for high sensitivity in the mapping routine.

Raman mapping and imaging enables the study of the distribution of materials on electrode surfaces, or across cross-sections. The resulting data can be quantified, giving metrics such as fraction estimates and particle statistics. Brief Description of the Drawings

Embodiments of the invention will now be described, by example only, with reference to the accompanying drawings, in which: - Figure 1 shows apparatus according to one embodiment of the invention;

Figure 2 is a flowchart illustrating a method of identifying and removing noise form spectral data according to an embodiment of the invention;

Figure 3 is a flowchart illustrating a method of "peak flattening" according to an embodiment of the invention;

Figure 4a shows an initial principal component (PC) and corresponding modified PC in the "time" domain and Figure 4b shows the initial PC and the corresponding modified PC in the frequency domain; Figure 5 is a schematic drawing of a processing apparatus according to an embodiment of the invention; and

Figure 6 is a manufacturing plant comprising apparatus according to the embodiment shown in Figure 1.

Description of Embodiments

Referring to Fig. 1, apparatus according to the invention comprises a Raman spectrometer connected to a computer 25 that has access to memory 29.

The Raman spectrometer comprises an input laser beam 10 reflected through 90 degrees by a dichroic filter 12, placed at 45 degrees to the optical path. Alternatively a holographic dichroic filter may be placed at a low angle of incidence such as 10 degrees. The laser beam then passes to an objective lens 16, which focuses it to a spot at its focal point 19 on a sample 18. Light is scattered by the sample at this illuminated spot, and is collected by the objective lens 16 and collimated into a parallel beam which passes back to the dichroic filter 12. The filter 12 rejects Rayleigh scattered light having the same frequency as the input laser beam 10, and transmits the Raman scattered light. The Raman scattered light then passes to a Raman analyser 20.

The Raman analyser 20 comprises a dispersive element such as a diffraction grating. The light from the analyser 20 is focused by a lens 22 onto a suitable photo-detector 24. A photo-detector array is preferred. In the present embodiment the detector 24 is a charge-coupled device (CCD), which consists of a two- dimensional array of pixels, and which is connected to the computer 25 which acquires data from each of the pixels and analyses it as required. The analyser 20 produces a spectrum spread out in a line along the CCD 24.

Samples 18 may be mounted on an X-Y table so that the focal point 19 can be scanned across it in X and Y directions, e.g. under control of the computer 25. A plurality of spectra can then be gathered, each spectrum corresponding to a different point on the sample. A map can then be generated based upon properties of the sample determined from the collected spectra.

The computer 25 is programmed with software code on a suitable medium, such as memory 29, comprising instructions, which when executed by a processor of computer 25 cause the computer 25 to perform the analysis routines described below. Alternatively, the data on the Raman spectrum/spectra obtained may be transferred to a separate computer having such software for this analysis. In either case, as the analysis proceeds, the values determined are stored in the computer concerned, and may be further processed and output or displayed to show the concentrations of the components in the sample/samples.

To process the spectral data to determine the Raman spectrum/spectra present and therefore, the chemical component(s)/properties of the sample, the spectral data is first processed to remove noise, in particular fixed pattern noise.

A switched power supply 1 1 is provided for powering the electronic devices of the spectrometer. A start of the readout from the CCD 24 is synchronised with the switching of the powder supply 1 1. This may simplify the analysis of the signals for fixed pattern noise, as described below.

Referring to Figure 2, the method comprising receiving 101 spectral data comprising a plurality of spectra sequentially recorded by the spectrometer. Principal component analysis (PCA) 102 is carried out on a plurality of the spectra resulting in a set of principal components. As required by principal component analysis, the principal components are ordered with the first principal component describing the largest variance in the spectral data and each succeeding component in turn describing the highest variance in the spectral data under the constraint that it is orthogonal to preceding principal components. The light grey lines in Figure 4a show a 6^th principal component 301 for a PCA of spectral data. The principal components are transformed 103 into the frequency domain, for example using a fast Fourier transform (FFT). Figure 4b shows the 6^th principal component 302 shown in Figure 4a in the frequency domain. Fixed pattern noise is identified 104 in the principal components in the frequency domain by identifying frequency peaks characteristic of fixed pattern noise.

Devices of the spectrometer, such as the switched power supply 11 and CCD detector 24 operate as known frequencies. Such known operating frequencies are stored in memory 29. The algorithm searches for peaks in the frequency domain at or around these known operating frequencies. If a peak is found then this is attributed to noise and removed 105 from the principal component. The synchronisation of the readout from the CCD 24 and the switching of the power supply 11 may reduce the number of principal components comprising fixed pattern noise due the switched power supply 11 (as the fixed pattern noise form the switch power supply 11 has the same phase as the readout from the CCD).

Furthermore, the algorithm searches for sharp peaks in the frequency domain that are unlikely to result from spectral data. Spectra data tends to result in a more smoothly varying signal in the frequency domain with much of the signal concentrated at the lower frequencies (although some contribution will exist at the higher frequencies due to high spectral peaks in the time domain). Such peaks are identified by a suitable peak detection algorithm, as is known in the art. Peaks due to noise can then be identified based upon defined signatures, such as defined thresholds for peak maximum and/or peak width (e.g. FWHM). The thresholds may be based upon an analysis of the data in the frequency domain, for example a threshold for peak maximum may be set based upon a multiple of a RMS average of the data. In Figure 4b, the illustrated principal component comprises two peaks 302a, 302b having the required signature to be attributable to noise.

In one embodiment, the method comprises generating 104a an output identifying characteristics, such as frequency, of the identified noise signal. A user may then use the knowledge of the noise signal characteristics to identify and eliminate sources of noise that are obscuring the spectral signal.

In step 105 a modified set of principal components are generated by modifying the loadings of the principal components to remove the identified noise signal. In this embodiment, the principal components are modified in the frequency domain. A "peak flattening" method may be used to remove the peaks from the principal component and "fill" in the gap left by peak removal. In Figure 4b, the data points of the peaks are replaced at a given point at which the peak is deemed to begin with a curve or line. The modified 6^th principal component in the frequency domain is shown by the dark line 303.

The method used for "peak flattening" may be varied depending on the width and position of the peak to be removed. The peak flattening technique may be chosen in order to optimise the rebuild of the spectral data from the modified components. The modified principal component is then transformed 106 back into the time domain using a reverse FFT. The modified 6^th principal component in the time domain is shown by the dark line 304. As can be seen in the example shown in Figure 4a, removal of the fixed pattern noise reveals spectrally significant data hidden within the noise.

Modified spectral data is then generated 107 by rebuilding spectral data using the principal components including the modified principal components from which fixed pattern noise has been removed.

The method is carried out as an iterative process, with the modified spectral data being fed back into step 102 and used as the initial data (i.e. "the spectral data") for the next iteration.

The iterative process is continued until termination criteria is met. The modified spectral data of the final iteration is output as the spectral data to be used for further analysis, such as to be used for identifying properties of the sample.

In another embodiment, the process is not an iterative process, with steps 102 to 107 being carried out on the spectral data only once. However, a further PC A of the rebuilt spectral data may be carried out and principal components below a defined nth order removed in order to remove non-fixed pattern noise from the measured data.

In a further embodiment, rather than specifically identifying peaks attributable to noise, a peak flattening routine is applied to each principal component in the entire data set to remove peaks having fixed pattern noise signatures.

Referring to Figure 3, the modified principal components are generated using an iterative process. After determining the principal components (in step 102) and transforming the principal components into the frequency domain (step 103), for each principal component Wu(v) in the frequency domain, a modified principal component W (v) is generated.

In step 202, reference data is initially set equal to the principal component Wu(v). To this reference data, an analytical curve is fitted 203, such as an nth order polynomial or a set of cubic spline curves. An order of the polynomial to be used may be preset, for example the process may be preset to be a fifth order polynomial. In the case of fitting a set of cubic spline curves, a number of anchor points is defined by the user. In step 204, a deviation of each datum point of the reference data from the analytical curve is determined and a root mean square (RMS) value is calculated for the deviations. The reference data is then modified, in this embodiment by replacing 205 data points of the reference data that are more than an allowable deviation (DEV) above the analytical curve with substitute data points that are equal to a value of the analytical curve at that frequency plus DEV. In this embodiment, the allowable deviation DEV is MxRMS, wherein M is a positive real number. M may be set by the user.

In step 206, it is determined whether a termination criterion is met, in this embodiment, whether any points have been replaced in step 205. If points were removed from the reference data, the method proceeds to the next iteration, wherein an analytical curve is fitted to the modified reference data. The iterative process continues until no points are replaced in step 205. In step 208, the reference data of the final iteration is output as the modified principal component W'k(v).

The modified spectral data may be analysed to determine properties of the sample(s) using any suitable technique, such as the methods described in WO

2007/144664 or WO 2012/156667. The invention may enable faster speeds for mapping a surface as it may be possible to identify spectral data at higher ratios of noise to spectral signal (the amplitude of the spectral data reducing as the mapping speed increases).

To carry out the above described method, the computer 25 may comprise processing modules 401 to 406, as defined in hardware or software. The modules 401 to 406 perform logically discrete functions and can be called by a lower level module/unit, such as control module 407 and pass values back to the lower level module/unit. In this embodiment, the modules 401 to 406 comprise a receiving module 401 for receiving the spectral data, an analysis module 402 for carrying of PCA on a data set to determine principal components of the data set, an identification module 403 for identifying noise signals in a data set, a modification module 404 for generating a modified data, such as a modified principal component, by removing at least a portion of a noise signal present in the data set, a modified measured data module 405 for generating modified spectral data by rebuilding spectral data from ones of the components including the at least one modified component and an output module 406 for outputting the modified spectral data and/or a characteristic of the identified noise signal.

Referring to Figure 5, the apparatus may be used in a manufacturing process to identify properties of manufactured articles 504, 507. The apparatus comprises means, in this embodiment a conveyor, for transporting material 502 to a machine 503 for transforming the material 502 into a manufactured article 504. For example, the article 504 may be a medicament, such as a tablet or a pill, a wafer of a 2D material or nanotechnology, a photovoltaic cell, semiconductor wafer and/or catalysis material. The manufactured article 504 is then transported to a spectrometer 506 for obtaining at least one Raman spectrum of the article 504. In one embodiment, the spectrometer may map a surface of the article 504. A plurality of spectra obtained from the article(s) 504 are sent to processing unit 525. Processing unit 525 is arranged to carry out the method as described above to remove fixed pattern noise in the spectral data. The fixed pattern noise may be internally generated by the spectrometer 506 or by other parts of the manufacturing apparatus. The processing unit 525 may comprise modules, as described with reference to Figure 5. The processing unit analyses the modified spectral data to identify properties of the articles 507. The identified properties may be used to determine whether articles 507 meet specified requirements, for example a specified level of homogeneity, semiconductor doping, etc and articles that are deemed not to meet the specified requirements may be discarded and/or further processed to correct the deficiency. Furthermore, operation of the machine 503 may be altered based upon the determined properties in order to alter the manufacturing process. For example, the operation may be altered to ensure that desired properties are achieved for future articles.

Furthermore, the processing unit 525 may output characteristics, such as frequencies, of fixed pattern noise identified in the spectral data. A user may analyse the output and alter the machine operation based upon the identified noise signal characteristics.

It will be understood that modifications and alterations can be made to the above described embodiments without departing from the invention as defined by the claims. For example, modification of the principal component(s) may be carried out in the time domain. In some embodiments, the order of the PCA and Fourier transform may be reversed such that PCA is carried out on the spectral data in the frequency domain, with PCA and Fourier transforms being linear transformations of the data.

Claims

1. A method of removing noise from spectral data, the method comprising receiving spectral data comprising a plurality of spectra captured sequentially using a spectroscopy system, carrying out a method of blind signal separation to separate the spectral data into a plurality of components, generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and generating modified spectral data from ones of the components including the at least one modified component.

2. A method according to claim 1, wherein the method of blind signal separation comprises principal component analysis.

3. A method according to claim 1 or claim 2, comprising identifying the noise signal based upon a known noise signature.

4. A method according to claim 3, wherein the noise signature is based upon a known noise frequency.

5. A method according to claim 4, wherein the known noise frequency is an operating frequency of an electronic component of the spectroscopy system.

6. A method according to any one of the proceeding claims, wherein generating the modified component comprises mathematically transforming the component to remove the noise signal.

7. A method according to claim 6, wherein modifying the component comprises generating the components in a frequency-domain using a time to frequency transform and modifying the component in the frequency-domain to remove at least a portion of the noise signal.

8. A method according to claim 7, wherein modifying the components comprises removing peaks attributable to noise from the components in the frequency-domain.

9. A method according to claim 8, wherein the peak is replaced with substitute data interpolated from data of the component in the frequency domain.

10. A method according to claim 8 or claim 9, comprising fitting an analytical curve to the data points of the component in the frequency domain and removing data points of the component in the frequency-domain that are more than an allowable deviation above the fitted analytical curve.

1 1. A method according to claim 10, wherein the allowable deviation is a multiple of an average deviation of the data points from the fitted analytical curve.

12. A method according to claim 10 or claim 1 1, comprising replacing the removed data points with substitute data points.

13. A method according to claim 12, wherein the substitute data points are equal to the allowable deviation above the fitted analytical curve.

14. A method according to any one of claims 10 to 13, comprising carrying out an iterative process, in which each iteration comprises fitting an analytical curve to reference data, determining the allowable deviation of the reference data from the analytical curve and removing data points of the reference data or the data of the component in the frequency domain that are more than the allowable deviation above the analytical curve to provide the reference data for the next iteration until termination criterion is met, wherein the reference data is initially the data of the component in the frequency domain.

15. A method according to claim 14, wherein the reference data of the final iteration is used as the modified component.

16. A method according to claim 14 or claim 15, wherein the iterative process is carried out on each component.

17. A method according to any one of claims 6 to 16, wherein generating the modified component comprises modifying the component in the time-domain.

18. A method comprising an iterative process in which the method according to any one of the preceding claims is carried out in each iteration, wherein the spectral data for each iteration, other than a first iteration, is the modified spectral data generated in a previous iteration.

19. A method according to any one of the preceding claims, wherein the plurality of spectra are obtained from multiple points on a sample.

20. A method of identifying noise in spectral data, the method comprising receiving spectral data comprising a plurality of spectra captured sequentially using a spectroscopy system, carrying out a method of blind signal separation to separate the spectral data into a plurality of components, identifying a noise signal in one or more of the components and generating an output identifying a characteristic of the noise signal.

21. A method according to claim 20, wherein the characteristic is a frequency of the noise signal.

22. A method according to claim 20 or claim 21, comprising generating further spectral data using the spectroscopy system after the spectroscopy system and/or localized environment to the spectroscopy system has been modified to remove and/or modify a source of the noise identified based upon the characteristic.

23. A method according to any one of claims 20 to 22, comprising generating further spectral data using the spectroscopy apparatus after the spectroscopy apparatus has been modified to change a spectral resolution based upon the identified noise signal.

24. A method according to any one of claims 20 to 23, wherein the method comprises identifying the noise signal from the components in a frequency- domain.

25. A method according to claim 24, wherein the noise is identified based upon peaks in the frequency domain having a defined noise signature.

26. A data carrier having stored thereon instructions, which, when executed by a processor, cause the processor to carry out the methods of any one of claims 1 to

25.

27. Apparatus comprising a processor, wherein the processor is arranged to carry out the methods of any one of claims 1 to 25.

28. Spectroscopy apparatus comprising a light source for illuminating one or more samples, collection optics for collecting light transmitted from the one or more samples when illuminated by the light source, a detector for generating signals based upon the light collected by the collection optics, wherein readout of signals from the detector is synchronised with a frequency of a device which generates noise in the signal.

29. Spectroscopy apparatus according to claim 28, wherein the device is a switched power supply of the spectroscopy apparatus.

30. Apparatus for removing noise from spectral data comprising means for receiving spectral data comprising a plurality of spectra captured sequentially using a spectroscopy system, means for carrying out a method of blind signal separation to separate the spectral data into a plurality of components, means for generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and means for generating modified spectral data from ones of the components including the at least one modified component.

31. Apparatus for removing noise from spectral data comprising a receiving module for receiving spectral data comprising a plurality of spectra captured sequentially using a spectroscopy system, an analytical module for carrying out a method of blind signal separation to separate the spectral data into a plurality of components, a component modification module for generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and a modified spectral data module for generating modified spectral data from ones of the components including the at least one modified component.

32. Apparatus for identifying noise in spectral data comprising means for receiving spectral data comprising a plurality of spectra captured sequentially using a spectroscopy system, means for carrying out a method of blind signal separation to separate the spectral data into a plurality of components and identifying a noise signal in one or more of the components and means for generating an output identifying a characteristic of the identified noise signal.

33. Apparatus for identifying noise in spectral data comprising a receiving module for receiving spectral data comprising a plurality of spectra captured sequentially using a spectroscopy system, an analytical module for carrying out a method of blind signal separation to separate the spectral data into a plurality of components and for identifying a noise signal in one or more of the components and an output module for generating an output identifying a characteristic of the identified noise signal.

34. A method of removing noise from measured data, the method comprising receiving measured data comprising a plurality of sequential observations of discrete variables, carrying out a method of blind signal separation to separate the measured data into a plurality of components, generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and generating modified data from ones of the components including the at least one modified component.

35. A method of identifying noise in measured data, the method comprising receiving measured data comprising a plurality of sequential observations of discrete variables, carrying out a method of blind signal separation to separate the measured data into a plurality of components, identifying a noise signal in one or more of the components and generating an output identifying a characteristic of the noise signal.

36. A data carrier having stored thereon instructions, which, when executed by a processor, cause the processor to carry out the methods of claim 34 or claim 35.

37. Apparatus comprising a processor, wherein the processor is arranged to carry out the methods of claim 34 or 35.

38. Apparatus for removing noise from measured data comprising means for receiving measured data comprising a plurality of sequential observations of discrete variables, means for carrying out a method of blind signal separation to separate the measured data into a plurality of components, means for generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and means for generating modified data from ones of the components including the at least one modified component.

39. Apparatus for removing noise from measured data comprising a receiving module for receiving measured data comprising a plurality of sequential observations of discrete variables, an analytical module for carrying out a method of blind signal separation to separate the measured data into a plurality of components, a component modification module for generating at least one modified component by removing at least a portion of a noise signal present in at least one of the components and a modified data module for generating modified data from ones of the components including the at least one modified component.

40. Apparatus for identifying noise in measured data comprising means for receiving measured data comprising a plurality of sequential observations of discrete variables, means for carrying out a method of blind signal separation to separate the measured data into a plurality of components and identifying a noise signal in one or more of the components and means for generating an output identifying a characteristic of the identified noise signal.

41. Apparatus for identifying noise in measured data comprising a receiving module for receiving spectral data comprising a plurality of sequential observations of discrete variables, an analytical module for carrying out a method of blind signal separation to separate the measured data into a plurality of components and for identifying a noise signal in one or more of the components and an output module for generating an output identifying a characteristic of the identified noise signal.