EP3415882A1 - Imaging spectrophotometer and satellite comprising the same - Google Patents
Imaging spectrophotometer and satellite comprising the same Download PDFInfo
- Publication number
- EP3415882A1 EP3415882A1 EP17306289.4A EP17306289A EP3415882A1 EP 3415882 A1 EP3415882 A1 EP 3415882A1 EP 17306289 A EP17306289 A EP 17306289A EP 3415882 A1 EP3415882 A1 EP 3415882A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- telescope
- spectrophotometer
- focal plane
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 71
- 238000001228 spectrum Methods 0.000 claims abstract description 46
- 238000012545 processing Methods 0.000 claims description 26
- 238000009825 accumulation Methods 0.000 claims description 23
- 239000006185 dispersion Substances 0.000 claims description 21
- 238000000034 method Methods 0.000 claims description 13
- 230000003595 spectral effect Effects 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 5
- 238000011144 upstream manufacturing Methods 0.000 claims description 3
- 241000735235 Ligustrum vulgare Species 0.000 claims description 2
- 101100269850 Caenorhabditis elegans mask-1 gene Proteins 0.000 description 7
- 238000005070 sampling Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 101100180304 Arabidopsis thaliana ISS1 gene Proteins 0.000 description 4
- 101100519257 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) PDR17 gene Proteins 0.000 description 4
- 101100042407 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) SFB2 gene Proteins 0.000 description 4
- 101100356268 Schizosaccharomyces pombe (strain 972 / ATCC 24843) red1 gene Proteins 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000012780 transparent material Substances 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/04—Slit arrangements slit adjustment
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
- G01J3/0229—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/027—Control of working procedures of a spectrometer; Failure detection; Bandwidth calculation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/12—Generating the spectrum; Monochromators
- G01J3/14—Generating the spectrum; Monochromators using refracting elements, e.g. prisms
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/12—Generating the spectrum; Monochromators
- G01J3/14—Generating the spectrum; Monochromators using refracting elements, e.g. prisms
- G01J3/16—Generating the spectrum; Monochromators using refracting elements, e.g. prisms with autocollimation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2803—Investigating the spectrum using photoelectric array detector
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
Definitions
- the invention relates to an imaging spectrophotometer and a satellite which comprises such imaging spectrophotometer.
- Imaging spectrophotometers are observation apparatuses which output so-called multispectral or hyper-spectral images of a scene.
- One multispectral or hyper-spectral image is comprised of a plurality of spectral images which all represent one same scene but using respective limited wavelength ranges which are distinct.
- Multispectral or “hyper-spectral” refers to the number of the spectral images, and also to the wavelength pitch between two successive spectral images. In the following, the word “multispectral” will be used to encompass both common uses of “multispectral” and “hyper-spectral” words.
- imaging spectrophotometers are implemented on board satellites which have observation missions, for example Earth observation missions.
- the spectrophotometer comprises at least one array-type image detector which is arranged at the output focal plane, and is oriented so that pixel columns of this image detector are parallel to the dispersion direction.
- the spectrophotometer is oriented with respect to the mask so that the dispersion direction is perpendicular to a longitudinal direction of an image of the slit in the output focal plane.
- scene elements which are imaged in the slit by the telescope are re-imaged by the spectrophotometer on the image detector within the slit image.
- a spectrum of the light which originates from each of the scene elements is spread perpendicular to the longitudinal direction of the slit image.
- an image-spectrum segment which is associated with the slit is formed on the array-type image detector.
- radiometric performances which are necessary to obtain high enough values for signal-to-noise ratios.
- signal-to-noise ratios relate to the signal values which are read out from the pixels of the image detector.
- the radiometric performances can be improved by increasing the size of the telescope, so as to increase the amount of light collected. But any increase in the telescope size is especially unfavourable for instruments intended to be on board satellites or spacecraft.
- the radiometric performances can also be improved by increasing the accumulation time which is used for capturing one multispectral image, but long accumulation times are compatible with pushbroom operation from a satellite only by implementing a slowdown of the scanning of the scene. Then, two zones of interest in the scene which are image-captured successively require that an intermediate switch in the orientation of the apparatus line-of-sight is performed, from the end of the slowdown movement which has been performed for the first zone of interest to the start of the slowdown movement to be performed for the next zone of interest. And the average time necessary for such switch in the orientation of the apparatus line-of-sight is longer when the slowdown is maintained over an image capture duration which is longer, and this leads to an overall image capture capacity which is lower.
- one object of the present invention consists in providing an imaging spectrophotometer which alleviates the preceding issues of radiometric performances and image capture capacity.
- a first aspect of the present invention proposes an imaging spectrophotometer as recited before, but in which the mask comprises several slits which are parallel and spaced perpendicular to these slits.
- the mask comprises several slits which are parallel and spaced perpendicular to these slits.
- one respective image-spectrum segment is associated with each slit separately from each other slit.
- a spectral detection window of the spectrophotometer together with an inter-slit spacing of the mask are such that any two image-spectrum segments which are associated with two neighbouring ones of the slits do not overlap in the output focal plane.
- the imaging spectrophotometer is modified according to the invention for multiplying scene strips which are imaged and spectrally spread through the spectrophotometer.
- the imaging spectrophotometer is capable to capture simultaneously multispectral images of several scene strips, through all the slits. These scene strips are at one time within the scene at respective locations which optically correspond to the slits through the telescope.
- the mask may comprise at least three slits, preferably less than nine slits, for example four slits, which are parallel, identical and evenly spaced.
- the spectrophotometer may comprise a combination of two prims for producing the dispersion, in which both prisms are selected so that an effective non-linear wavelength-dispersion component, as resulting from the two-prism combination, is reduced when compared to a separate non-linear wavelength-dispersion component of each of the two prisms.
- the imaging spectrophotometer can have a constant or almost constant wavelength-dispersion rate along the pixel columns of the image detector, for all the image-spectrum segments.
- the imaging spectrophotometer may further comprise:
- imaging spectrophotometer allows combining contents from several image-spectrum segments which are captured at various times but correspond to the same scene strip.
- Such operation increases the signal-to-noise ratio for each pixel of the multispectral image, by a factor of N 1/2 , where N is the number of image-spectrum segment contents which are combined.
- the combining unit may be also on board the satellite or spacecraft. But preferably, the combining unit may be arranged in a remote data processing station, and the contents of at least two image-spectrum segments which are to be combined are transmitted from the satellite or spacecraft to the combining unit.
- the data processing station may be a ground station on the Earth.
- the imaging spectrophotometer may further comprise:
- the imaging spectrophotometer allows capturing a full-frame multispectral image while executing a scan of the zone of interest in the scene which may be limited in spatial length to the full-frame length divided by the slit number.
- a slowdown rate of the scanning which may be necessary for operation on board a satellite can also be reduced as a consequence, and allows the image capture capacity to be higher.
- the image reconstruction unit may be also on board the satellite or spacecraft. But preferably, the image reconstruction unit may be arranged in a remote data processing station, and the line contents which are to be gathered to form a full-frame multispectral image are transmitted from the satellite or spacecraft to the image reconstruction unit.
- the data processing station may be again a ground station on the Earth.
- this latter may further comprise:
- Such improvement consists in adding a further imaging channel, which is devoid of spectral dispersion.
- additional imaging channel may in particular detect light over the entire wavelength range which is involved by the spectrophotometer.
- the beam divider used for such improvement may be designed so as to direct respective first sub-beams of the light collected by the telescope to the slits of the mask, and respective second sub-beams of said light collected by the telescope to spaced-apart pixel lines or spaced-apart pixel line groups of the additional image detector set.
- pixel line group denotes a group of neighbouring pixel lines.
- the pixel lines or pixel line groups of the additional image detector set may be optically interleaved with the slits through the beam divider.
- the additional image detector set which is used in the added imaging channel may be comprised of at least one another array-type image detector. Then the pixel lines or pixel line groups are spaced apart within the photosensitive surface of this another array-type image detector.
- the number of the pixel lines or pixel line groups of the additional image detector set may advantageously equal the number of the slits provided in the mask plus one.
- the spectrophotometer channel and the added imaging channel can produce respective images of one common zone of interest in the scene which is optimized in size with respect to the image detectors of both channels.
- a second aspect of the invention proposes a satellite which comprises a platform, an attitude control system for varying an orientation of the platform, and an imaging spectrophotometer according to the first invention aspect, this imaging spectrophotometer being supported rigidly by the platform.
- the satellite further comprises steering means suitable for directing the line-of-sight of the imaging spectrophotometer towards successive scene strips in a pushbroom manner, while the image detector performs a series of successive accumulation and read-out operations.
- the steering means may be part of the satellite attitude control system, or may comprise a steering mirror system which is arranged in front of a primary mirror of the telescope, or a combination of an operation of the satellite attitude control system with a steering mirror system.
- the steering means may be suitable to produce a slowdown of a scanning of a zone of interest in the scene by the line-of-sight of the imaging spectrophotometer while the image detector performs the series of successive accumulation and read-out operations for capturing a full-frame multispectral image.
- the steering means may be suitable to produce the slowdown of the scanning of the zone of interest in the scene by the line-of-sight of the imaging spectrophotometer limitedly within a scan duration in which the image of the scene which is formed by the telescope moves in the telescope focal plane over a distance that is comprised between the sum of the slit width and the inter-slit spacing, and another sum of the slit width and twice the inter-slit spacing.
- the attitude control system or steering mirror system is also suitable to switch the line-of-sight of the imaging spectrophotometer to another zone of interest in the scene after the scan duration has ended.
- a third aspect of the invention proposes a process for capturing a multispectral image, which comprises the following steps:
- the data processing unit may be on board the satellite, but it may be preferably located on the Earth. In this latter case, the read-out data are to be transmitted from the satellite to a ground receiving station and then forwarded to the data processing unit.
- a fourth aspect of the invention proposes a process for obtaining a multispectral image, which process comprises:
- the data processing unit may be a combining unit or an image reconstruction unit as recited above in connection with the first and second embodiments of the first invention aspect.
- Figure 1 shows a satellite 200 in orbit over the Earth E, with the arrow V denoting the speed vector of the satellite.
- the satellite 200 is provided on board with an imaging spectrophotometer 100 according to the invention, a satellite platform 201, and a satellite attitude control system 202.
- the imaging spectrophotometer 100 is fixed rigidly to the platform 201, and the attitude control system 202 allows controlling an orientation of the platform 201.
- the imaging spectrophotometer 100 is designed for capturing images of strips of the Earth surface which are perpendicular to the satellite speed vector V. Then, repeating several image capturing operations while the satellite 200 moves allows imaging a two-dimensional area in the Earth surface, called zone of interest.
- ZI1 denotes a first zone of interest, and ZI2 another one.
- Each zone of interest can be image-captured in this manner, known as pushbroom process by the Man skilled in the art.
- the longitudinal direction of the strips of the Earth surface which are successively image-captured is denoted x.
- the scanning direction of each zone of interest denoted y, is parallel to the speed vector V of the satellite 200, but other operations are also possible where the scanning direction is no longer parallel to the speed vector V.
- the orientation of the line-of-sight L of the imaging spectrophotometer 100 towards any point of a median line of a zone of interest to be pushbroom-scanned can be performed by rotating the satellite 200 appropriately, using the attitude control system 202. Alternatively, this can also be performed by controlling a steering mirror which is arranged on board the satellite 200 in front of the optical entrance of the imaging spectrophotometer 100. Such steering mirror is labelled 202' in Figure 1 , but it may not be implemented if the satellite attitude control system 202 is agile enough. Also possibly, satellite rotation and steering mirror may be combined for varying the orientation of the line-of-sight L.
- the pushbroom-scan of each zone of interest in the Earth surface may be performed by implementing a slowdown of the shift of the line-of-sight L with respect to the moving of the satellite 200 along its orbit.
- Such slowdown can be made necessary by the maximum frame readout frequency of the image detector which is used in the imaging spectrophotometer 100, or by insufficient light intensity which makes necessary to increase the accumulation duration for each strip within the zone of interest being currently scanned.
- reference numbers 10 and 20 denote a telescope and a spectrophotometer, respectively.
- the telescope 10 may be of any type known in the art. For example, it may be of three-mirror anastigmat type, with primary mirror 11, secondary mirror 12 and tertiary mirror 13.
- the light beam F originates from a scene such as the zone of interest ZI1 in the Earth surface when the imaging spectrophotometer 100 is implemented on board the satellite 200.
- Reference letter P indicates the entrance pupil of the telescope 10.
- the light beam F is reflected successively by the primary mirror 11, then the secondary mirror 12, and then the tertiary mirror 13, so as to be focussed in a focal plane FP 10 of the telescope 10.
- the spectrophotometer 20 may also be of any type known in the art. For example, it may have a Littrow configuration, with first mirror 21, second mirror 22, third mirror 23, and a dispersing element 24.
- the spectrophotometer 20 is designed to image a content of the focal plane FP 10 of the telescope 10 within an output focal plane FP 20 .
- the light which is transmitted through the telescope focal plane FP 10 is reflected first by the third mirror 23, then the second mirror 22, then the first mirror 21, then is reflection-dispersed by the dispersing element 24, then reflected again by the first mirror 21, then the second mirror 22, then the third mirror 23, so as to be focussed in the output focal plane FP 20 .
- the imaging spectrophotometer 100 produces in the output focal plane FP 20 an image of the scene which is combined with the pattern of a mask that may be located within the telescope focal plane FP 10 .
- the scene image in the output focal plane FP 20 is combined with dispersion produced by the dispersing element 24.
- the dispersing element 24 may be based on at least one prism of a refracting transparent material, which is selected with respect to the operation wavelength range desired for the imaging spectrophotometer 100.
- this wavelength range may extend from 400 nm (nanometer) to 2500 nm
- the dispersing element 24 may be a combination of two prisms which are arranged serially in the Littrow configuration of the spectrophotometer 20.
- One of these prisms, labelled 24' may be out of a highly dispersing glass, and the other one, labelled 24", may be out of another transparent material which is selected for compensating a non-linear component of the wavelength dispersion which is produced by the prism 24'.
- the total dispersion rate which is obtained in this way when expressed as a derivation of an angular deviation with respect to the wavelength, may be both important and almost constant over the whole wavelength range or a large part thereof.
- Reference number 2 denotes an image detector which is located at the output focal plane FP 20 .
- the image detector 2 is of array-type, with pixels which are distributed in columns and rows in a well-known manner.
- the image detector 2 is oriented so that the pixel columns are parallel to a dispersion direction which is produced by the spectrophotometer 20 in the output focal plane FP 20 .
- Reference number 3 denotes an optional beam divider which may be located between the tertiary mirror 13 of the telescope 10 and the telescope focal plane FP 10 along the light path.
- the beam divider 3 may be a bi-prism designed for transmitting a first part of the light which propagates from the telescope 10, and reflecting another part of this light.
- the beam divider 3 duplicates the telescope focal plane FP 10 into an additional telescope focal plane FP 10 '. Then, arranging an additional image detector 2' at this additional telescope focal plane FP 10 ' allows recovering an image of the scene from light which has not been dispersed by the spectrophotometer 20.
- Reference numbers 4 and 5 denote two optional folding mirrors.
- the folding mirror 4 is located between the telescope focal plane FP 10 and the third mirror 23 along the light path at the entrance end of the spectrophotometer 20, and the folding mirror 5 is located between the third mirror 23 and the output focal plane FP 20 at the output end of the spectrophotometer 20.
- the folding mirrors 4 and 5 may be useful for obtaining a compact design for the whole imaging spectrophotometer 100.
- a mask 1 is light-blocking except within slits which transmit light.
- the mask 1 may be provided with four slits labelled S1 to S4.
- the slits S1-S4 are parallel and spaced apart from each other along a direction which is perpendicular to these slits.
- l s , w s and i s denote respectively the slit length, the slit width and the inter-slit spacing, which are common to all slits.
- the mask 1 is to be located in the telescope focal plane FP 10 ( Figure 2a ), with the slits oriented so as to appear perpendicular to the dispersion direction which is produced by the spectrophotometer 20 in the output focal plane FP 20 .
- the slits are oriented in the telescope focal plane FP 10 so that their respective monochromatic images through the spectrophotometer 20 each extend longitudinally parallel to the pixel rows of the image detector 2 within the output focal plane FP 20 .
- each one of the slits S1-S4 of the mask 1 allows a scene strip which is imaged within this slit in the telescope focal plane FP 10 to be re-imaged in the output focal plane FP 20 parallel to the pixel row direction of the image detector 2, and the light which corresponds to each location in this latter image is spread spectrally parallel to the pixel column direction.
- Such detector portion has been called image-spectrum segment in the general part of this description, and it is an issue that two image-spectrum segments which correspond two neighboring ones of the slits S1-S4 do not overlap along the pixel column direction.
- Such overlapping suppression can be obtained by selecting appropriately the inter-slit spacing is with respect to the extension of the image-spectrum segments along the pixel column direction, or by limiting the length of the spectral range of the light which reaches the image detector 2.
- the Man skilled in optics will be able to select appropriately these parameters of the imaging spectrophotometer 100, and the following values are only provided as a non-limiting example:
- the invention may be implemented using the NGP detector produced by Sofradir®, NGP standing for «Next Generation Panchromatic®», for the array-type image detector 2.
- Figure 4 shows the allocation of the photosensitive area of the image detector 2 which results from such implementation.
- C and R denote the column direction and the row direction of the image detector 2, respectively.
- ISSn denotes the image-spectrum segment which corresponds to the slit n, n being an integer index from 1 to 4 in the implementation example considered.
- the longitudinal direction of the scene strip which is imaged denoted x
- the dispersion direction denoted ⁇
- the four image-spectrum segments ISS1-ISS4 relate to four scene strips which are spaced apart in the zone of interest ZI1 perpendicular to the longitudinal direction of the strips.
- the first pixel row within each image-spectrum segment ISSn corresponds to the wavelength sampling interval 400 nm - 410 nm
- the last pixel row within each image-spectrum segment ISSn corresponds to wavelength sampling interval 2490 nm - 2500 nm.
- Second embodiments of the invention now described may be appropriate for increasing a value of a signal-to-noise ratio which exists upon multispectrally image-capturing one zone of interest, such as ZI1 for example.
- Photonic noise involved upon signal accumulation in one pixel of the image detector 2 and signal readout is proportional to I 1/2 , where I is the image intensity read out for this pixel. Therefore, the pixel signal-to-noise ratio increases by a factor N 1/2 , where N is a number of iterations that are performed for the capture of one and same scene portion which is imaged into one pixel area.
- the plurality of image-spectrum segments which are produced by the invention within the photosensitive area of the image detector 2 allows that one and same scene strip is imaged and spectrally spread multiple times during the pushbroom-scanning of one complete zone of interest by the line-of-sight L of the imaging spectrophotometer 100.
- one and same scene strip is imaged at time t 1 in the image-spectrum segment ISS1 through the slit S1, then at a later time t 2 in the image-spectrum segment ISS2 through the slit S2, and again at a later time t 3 in the image-spectrum segment ISS3 through the slit S3, and one more time at t 4 in the image-spectrum segment ISS4 through the slit S4.
- Figure 5 illustrates the instant content of each image-spectrum segment at times t 1 , t 2 , t 3 or t 4 , respectively, in the output focal plane FP 20 , due to the slits in the telescope focal plane and the effect of the spectrophotometer 20.
- the content represented is a scene SC but only four slit-limited strips of such scene are imaged at one time and simultaneously wavelength dispersed along the column direction in each image-spectrum segment, and the complete image of the scene SC as represented only results from the pushbroom-scanning.
- a controller 21 and a combining unit 22 may be coupled with the image detector 2, whereby the controller 21 triggers a series of successive accumulation and readout operations while the image of the scene SC moves across the slits S1-S4, and the combining unit 22 adds the pixel signals read out from image-spectrum segments ISS1-ISS4 at the times t 1 -t 4 .
- Each of these pixel signal additions relates to pixels which are located identically within the separate image-spectrum segments ISS1-ISS4.
- the signal-to-noise increase principle is applied independently and simultaneously for each scene element which is imaged and for each wavelength sampling interval which corresponds to one pixel line within the image-spectrum segments ISS1-ISS4.
- the controller 21 is located on board the satellite 200 whereas the combining unit 22 is preferably within a data processing ground station on the Earth E.
- the combining unit 22 is coupled with the other components through appropriate transmission means, such as radio or optical communication means.
- Second embodiments of the invention may be appropriate for decreasing the slowdown requirement that may apply for capturing a multispectral image of one zone of interest.
- the left part of Figure 6 shows the telescope focal plane with the four mask slits S1-S4.
- the frame drawn in thin line and denoted ZI1' is the peripheral limit of the image of the zone of interest ZI1 in the telescope focal plane FP 10 . Then, during one and same accumulation and readout operation of the whole image detector 2, signals are recovered simultaneously from four scene strips which are imaged on the image detector 2 respectively through the four slits S1-S4.
- the multi-slit configuration of the mask 1 allows reducing the slowdown duration which is necessary for each zone of interest to be captured in a full-frame multispectral image.
- the slowdown area is reduced in length, allowing faster imagery, or increased slowdown rate.
- the jumps of the line-of-sight L which are necessary between the processing sequences of several zones of interest are also reduced in length on average.
- each one of the image-spectrum segments ISS1-ISS4 (right part of Figure 6 ) outputs progressively in time one fourth of the complete multispectral image of the zone of interest ZI1, and the complete multispectral image is to be reconstructed by joining the multispectral image segments which are outputted simultaneously and in parallel by the image-spectrum segments ISS1-ISS4 of the image detector 2.
- the image detector 2 is to be coupled with the controller 21', but also with an image reconstruction unit 23 as shown in Figure 6 .
- the controller 21' is located on board the satellite 200 whereas the image reconstruction unit 23 may be within a data processing ground station on the Earth E.
- the image reconstruction unit 23 is coupled with the other components through appropriate transmission means, such as radio or optical communication means.
- the controller 21' triggers a series of successive accumulation and readout operations while the image of the scene moves across one inter-slit space.
- the reconstruction unit 23 gathers the line contents which are outputted successively by the image detector 2 from all the image-spectrum segments ISS1-ISS4, separately for each wavelength sampling interval.
- Embodiments of the controllers 21 and 21', the combination unit 22 and the image reconstruction unit 23 are known from the Man skilled in the art, so that it is not necessary to describe them further herein.
- both image-spectrum segments ISS1 and ISS2 are operated as for the first embodiments described above, and also both image-spectrum segments ISS3 and ISS4 simultaneously and in parallel with the image-spectrum segments ISS1 and ISS2, but both segment pairs ISS1+ISS2 and ISS3+ISS4 are further operated as for the second embodiments.
- Figure 3b corresponds to Figure 3a when the additional image detector 2' is implemented.
- Figure 3b further indicates pixel lines of the additional image detector 2' which are allowed to detect light originating from the scene and transmitted by the telescope 10.
- These pixel lines of the additional image detector 2' may be selected from a two-dimensional pixel array by a detector controller in a well-known manner. These pixel lines are denoted L1-L5, and are interleaved with the slits S1-S4 when superposing both telescope focal planes and FP 10 '.
- the pixel line L5 is optional, and advantageously implemented for the second embodiments described above.
- each one of the pixel lines L1-L5 of the image detector 2' thus implemented may be replaced by several pixel lines, for example two or three, which are next to one another in the two-dimensional pixel array of the image detector 2', and which may be read out using a binning process.
- the image detector 2' may be of a type similar as that of the image detector 2.
- the image detector 2' may have a pixel pitch smaller than that of the image detector 2, for example two or three times smaller, for obtaining a better resolution.
- No spectral filtering may be implemented for the image detection which is performed using the image detector 2', so that it constitutes an imaging channel which may be called panchromatic.
- the wavelength detection range which is effective for the imaging channel comprised of the telescope 10, the beam splitter 3 and the image detector 2' may only result from spectral windows of these optical components.
- the beam splitter 3 may be designed so as to reflect and transmit in an interleaved manner, light sub-beams from the telescope 10. As shown in Figure 3c , the reflected light sub-beams may be directed to the spectrophotometer 20, one sub-beam for each one of the slits S1-S4 of the mask 1, and the transmitted light sub-beams may be directed to the image detector 2', one sub-beam for each one of the pixel lines L1-L5.
- the beam splitter 3 may be comprised of two prisms 30 and 30' which are glued together on an interface, with this interface being provided with spaced-apart reflecting strips Re. One reflecting strip Re may be dedicated to each slit of the mask 1, and two neighbouring strips Re are separated by a transmitting gap Tr dedicated of one of the pixel lines L1-L5.
Landscapes
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- General Physics & Mathematics (AREA)
- Spectrometry And Color Measurement (AREA)
Abstract
Description
- The invention relates to an imaging spectrophotometer and a satellite which comprises such imaging spectrophotometer.
- Imaging spectrophotometers are observation apparatuses which output so-called multispectral or hyper-spectral images of a scene. One multispectral or hyper-spectral image is comprised of a plurality of spectral images which all represent one same scene but using respective limited wavelength ranges which are distinct. "Multispectral" or "hyper-spectral" refers to the number of the spectral images, and also to the wavelength pitch between two successive spectral images. In the following, the word "multispectral" will be used to encompass both common uses of "multispectral" and "hyper-spectral" words.
- In particular, imaging spectrophotometers are implemented on board satellites which have observation missions, for example Earth observation missions.
- Such imaging spectrophotometer as known from prior art comprises:
- a telescope, which is adapted for collecting light originating from a scene and for producing an image of the scene in a focal plane of this telescope;
- a mask which is located in the telescope focal plane, and which is provided with a slit so as to transmit selectively light rays which are sent by the telescope through this slit; and
- a spectrophotometer which is arranged for imaging a content of the telescope focal plane in an output focal plane, and has a dispersion direction that is contained in the output focal plane.
- The spectrophotometer comprises at least one array-type image detector which is arranged at the output focal plane, and is oriented so that pixel columns of this image detector are parallel to the dispersion direction.
- Furthermore, the spectrophotometer is oriented with respect to the mask so that the dispersion direction is perpendicular to a longitudinal direction of an image of the slit in the output focal plane. In this way, scene elements which are imaged in the slit by the telescope are re-imaged by the spectrophotometer on the image detector within the slit image. Additionally, a spectrum of the light which originates from each of the scene elements is spread perpendicular to the longitudinal direction of the slit image. Thus, an image-spectrum segment which is associated with the slit is formed on the array-type image detector.
- But because of spatial and spectral resolution requirements, an important issue of such apparatuses relates to the radiometric performances which are necessary to obtain high enough values for signal-to-noise ratios. Such signal-to-noise ratios relate to the signal values which are read out from the pixels of the image detector. The radiometric performances can be improved by increasing the size of the telescope, so as to increase the amount of light collected. But any increase in the telescope size is especially unfavourable for instruments intended to be on board satellites or spacecraft.
- The radiometric performances can also be improved by increasing the accumulation time which is used for capturing one multispectral image, but long accumulation times are compatible with pushbroom operation from a satellite only by implementing a slowdown of the scanning of the scene. Then, two zones of interest in the scene which are image-captured successively require that an intermediate switch in the orientation of the apparatus line-of-sight is performed, from the end of the slowdown movement which has been performed for the first zone of interest to the start of the slowdown movement to be performed for the next zone of interest. And the average time necessary for such switch in the orientation of the apparatus line-of-sight is longer when the slowdown is maintained over an image capture duration which is longer, and this leads to an overall image capture capacity which is lower.
- Starting from this situation, one object of the present invention consists in providing an imaging spectrophotometer which alleviates the preceding issues of radiometric performances and image capture capacity.
- For meeting this object or another one, a first aspect of the present invention proposes an imaging spectrophotometer as recited before, but in which the mask comprises several slits which are parallel and spaced perpendicular to these slits. Thus, one respective image-spectrum segment is associated with each slit separately from each other slit. Furthermore, a spectral detection window of the spectrophotometer together with an inter-slit spacing of the mask are such that any two image-spectrum segments which are associated with two neighbouring ones of the slits do not overlap in the output focal plane.
- So, the imaging spectrophotometer is modified according to the invention for multiplying scene strips which are imaged and spectrally spread through the spectrophotometer. In this way, the imaging spectrophotometer is capable to capture simultaneously multispectral images of several scene strips, through all the slits. These scene strips are at one time within the scene at respective locations which optically correspond to the slits through the telescope.
- Generally for the invention, the mask may comprise at least three slits, preferably less than nine slits, for example four slits, which are parallel, identical and evenly spaced.
- Also generally for the invention, the spectrophotometer may comprise a combination of two prims for producing the dispersion, in which both prisms are selected so that an effective non-linear wavelength-dispersion component, as resulting from the two-prism combination, is reduced when compared to a separate non-linear wavelength-dispersion component of each of the two prisms. In this way, the imaging spectrophotometer can have a constant or almost constant wavelength-dispersion rate along the pixel columns of the image detector, for all the image-spectrum segments.
- In first embodiments of the invention, the imaging spectrophotometer may further comprise:
- a controller suitable to cause the image detector to perform a series of successive accumulation and read-out operations while the image of the scene which is formed by the telescope moves in the telescope focal plane in such a way that one and same scene strip is imaged in the output focal plane successively through several ones of the slits; and
- optionally a combining unit suitable to combine contents from at least two of the image-spectrum segments, when these contents relate to said one and same scene strip but pertain to accumulation and read-out operations which have been performed separately.
- Put another way, such imaging spectrophotometer allows combining contents from several image-spectrum segments which are captured at various times but correspond to the same scene strip. Such operation increases the signal-to-noise ratio for each pixel of the multispectral image, by a factor of N1/2, where N is the number of image-spectrum segment contents which are combined.
- When the assembly of the telescope, the mask, the spectrophotometer and the controller is arranged on board a satellite or a spacecraft, the combining unit may be also on board the satellite or spacecraft. But preferably, the combining unit may be arranged in a remote data processing station, and the contents of at least two image-spectrum segments which are to be combined are transmitted from the satellite or spacecraft to the combining unit. The data processing station may be a ground station on the Earth.
- In second embodiments of the invention, the imaging spectrophotometer may further comprise:
- another controller suitable to cause the image detector to perform a series of successive accumulation and read-out operations while the image of the scene which is formed by the telescope moves in the telescope focal plane over a distance at least equal to a sum of a slit width and the inter-slit spacing; and
- optionally an image reconstruction unit suitable to form a full-frame multispectral image by gathering line contents which are outputted successively by the image detector from all the image-spectrum segments.
- In this way, the imaging spectrophotometer allows capturing a full-frame multispectral image while executing a scan of the zone of interest in the scene which may be limited in spatial length to the full-frame length divided by the slit number. A slowdown rate of the scanning which may be necessary for operation on board a satellite can also be reduced as a consequence, and allows the image capture capacity to be higher.
- When the assembly of the telescope, the mask, the spectrophotometer and the controller is arranged on board a satellite or a spacecraft, the image reconstruction unit may be also on board the satellite or spacecraft. But preferably, the image reconstruction unit may be arranged in a remote data processing station, and the line contents which are to be gathered to form a full-frame multispectral image are transmitted from the satellite or spacecraft to the image reconstruction unit. The data processing station may be again a ground station on the Earth.
- Both first and second embodiments just recited may also be combined to form other imaging spectrophotometers which meet again the invention features.
- According to a general improvement of an imaging spectrophotometer according to the invention, this latter may further comprise:
- a beam divider arranged upstream the telescope focal plane along a light propagation path within the telescope, and designed for allowing each slit of the mask to be entirely exposed to part of the light which is collected by the telescope, and for allowing another part of said light which is collected by the telescope to focus into another telescope focal plane without further travelling through the spectrophotometer; and
- an additional image detector set arranged at said another telescope focal plane so as to capture scene images produced by the telescope from said another part of the light which does not travel through the spectrophotometer.
- Thus, such improvement consists in adding a further imaging channel, which is devoid of spectral dispersion. Such additional imaging channel may in particular detect light over the entire wavelength range which is involved by the spectrophotometer.
- The beam divider used for such improvement may be designed so as to direct respective first sub-beams of the light collected by the telescope to the slits of the mask, and respective second sub-beams of said light collected by the telescope to spaced-apart pixel lines or spaced-apart pixel line groups of the additional image detector set. Here, pixel line group denotes a group of neighbouring pixel lines. Additionally, the pixel lines or pixel line groups of the additional image detector set may be optically interleaved with the slits through the beam divider.
- The additional image detector set which is used in the added imaging channel may be comprised of at least one another array-type image detector. Then the pixel lines or pixel line groups are spaced apart within the photosensitive surface of this another array-type image detector.
- When second embodiments of the invention as described above are provided with the improvement of the added imaging channel, the number of the pixel lines or pixel line groups of the additional image detector set may advantageously equal the number of the slits provided in the mask plus one. Thus, the spectrophotometer channel and the added imaging channel can produce respective images of one common zone of interest in the scene which is optimized in size with respect to the image detectors of both channels.
- A second aspect of the invention proposes a satellite which comprises a platform, an attitude control system for varying an orientation of the platform, and an imaging spectrophotometer according to the first invention aspect, this imaging spectrophotometer being supported rigidly by the platform. The satellite further comprises steering means suitable for directing the line-of-sight of the imaging spectrophotometer towards successive scene strips in a pushbroom manner, while the image detector performs a series of successive accumulation and read-out operations. The steering means may be part of the satellite attitude control system, or may comprise a steering mirror system which is arranged in front of a primary mirror of the telescope, or a combination of an operation of the satellite attitude control system with a steering mirror system.
- Preferably, the steering means may be suitable to produce a slowdown of a scanning of a zone of interest in the scene by the line-of-sight of the imaging spectrophotometer while the image detector performs the series of successive accumulation and read-out operations for capturing a full-frame multispectral image.
- For the second embodiments of the invention, the steering means may be suitable to produce the slowdown of the scanning of the zone of interest in the scene by the line-of-sight of the imaging spectrophotometer limitedly within a scan duration in which the image of the scene which is formed by the telescope moves in the telescope focal plane over a distance that is comprised between the sum of the slit width and the inter-slit spacing, and another sum of the slit width and twice the inter-slit spacing. The attitude control system or steering mirror system is also suitable to switch the line-of-sight of the imaging spectrophotometer to another zone of interest in the scene after the scan duration has ended.
- A third aspect of the invention proposes a process for capturing a multispectral image, which comprises the following steps:
- providing a satellite according to the second invention aspect, which satellite is in orbit around the Earth;
- directing the line-of-sight of the imaging spectrophotometer towards a zone of interest on the Earth;
- moving the line-of sight so that an image of a scene which is contained in the zone of interest moves in the telescope focal plane, and simultaneously controlling the array-type image detector so as to perform a series of successive accumulation and read-out operations; and
- transmitting read-out data which have been produced by the successive accumulation and read-out operations to a data processing unit, this data processing unit being suitable for producing the multispectral image from the read-out data.
- The data processing unit may be on board the satellite, but it may be preferably located on the Earth. In this latter case, the read-out data are to be transmitted from the satellite to a ground receiving station and then forwarded to the data processing unit.
- A fourth aspect of the invention proposes a process for obtaining a multispectral image, which process comprises:
- providing a data processing unit located on the Earth, this data processing unit being suitable for producing the multispectral image from the read-out data which have been produced by a process according to the third invention aspect;
- collecting the read-out data and transmitting them to the data processing unit; and
- operating the data processing unit for producing the multispectral image from the read-out data.
- For the third and fourth invention aspects, the data processing unit may be a combining unit or an image reconstruction unit as recited above in connection with the first and second embodiments of the first invention aspect.
- These and other features of the invention will be now described with reference to the appended figures, which relate to preferred but not-limiting embodiments of the invention.
-
-
Figure 1 illustrates an implementation of an imaging spectrophotometer according to the invention on board a satellite. -
Figure 2 is general perspective view of an imaging spectrophotometer according to the invention, suitable for the implementation ofFigure 1 . -
Figure 2a is an enlarged view of a part ofFigure 2 , from another observation side. -
Figure 3a and 3b are plan views of a mask used in the imaging spectrophotometer ofFigure 1 . -
Figure 3c is an optical diagram of a beam splitter which may be used in the imaging spectrophotometer ofFigure 1 . -
Figure 4 illustrates an allocation of portions of a photosensitive surface of an image detector which is used in the imaging spectrophotometer ofFigure 1 . -
Figure 5 illustrates first embodiments of the invention. -
Figure 6 illustrates second embodiments of the invention. - For clarity sake, element sizes which appear in these figures do not correspond to actual dimensions or dimension ratios. Also, same reference numbers which are indicated in different ones of these figures denote identical elements of elements with identical function.
-
Figure 1 shows asatellite 200 in orbit over the Earth E, with the arrow V denoting the speed vector of the satellite. Thesatellite 200 is provided on board with animaging spectrophotometer 100 according to the invention, asatellite platform 201, and a satelliteattitude control system 202. Theimaging spectrophotometer 100 is fixed rigidly to theplatform 201, and theattitude control system 202 allows controlling an orientation of theplatform 201. - The
imaging spectrophotometer 100 is designed for capturing images of strips of the Earth surface which are perpendicular to the satellite speed vector V. Then, repeating several image capturing operations while thesatellite 200 moves allows imaging a two-dimensional area in the Earth surface, called zone of interest. ZI1 denotes a first zone of interest, and ZI2 another one. Each zone of interest can be image-captured in this manner, known as pushbroom process by the Man skilled in the art. The longitudinal direction of the strips of the Earth surface which are successively image-captured is denoted x. For such operation, the scanning direction of each zone of interest, denoted y, is parallel to the speed vector V of thesatellite 200, but other operations are also possible where the scanning direction is no longer parallel to the speed vector V. - As commonly known, the orientation of the line-of-sight L of the
imaging spectrophotometer 100 towards any point of a median line of a zone of interest to be pushbroom-scanned can be performed by rotating thesatellite 200 appropriately, using theattitude control system 202. Alternatively, this can also be performed by controlling a steering mirror which is arranged on board thesatellite 200 in front of the optical entrance of theimaging spectrophotometer 100. Such steering mirror is labelled 202' inFigure 1 , but it may not be implemented if the satelliteattitude control system 202 is agile enough. Also possibly, satellite rotation and steering mirror may be combined for varying the orientation of the line-of-sight L. - Again as commonly known, the pushbroom-scan of each zone of interest in the Earth surface may be performed by implementing a slowdown of the shift of the line-of-sight L with respect to the moving of the
satellite 200 along its orbit. Such slowdown can be made necessary by the maximum frame readout frequency of the image detector which is used in theimaging spectrophotometer 100, or by insufficient light intensity which makes necessary to increase the accumulation duration for each strip within the zone of interest being currently scanned. - Also, moving the line-of-sight L from one zone of interest to a next one to be scanned, for example from ZI1 to ZI2, is time consuming. Such durations between successive pushbroom scans reduce the overall number of zones of interest which can be processed during one travel of the
satellite 200 over the Earth surface portion which contains these zones of interest. - In
Figures 2 and2a ,reference numbers - The
telescope 10 may be of any type known in the art. For example, it may be of three-mirror anastigmat type, withprimary mirror 11,secondary mirror 12 andtertiary mirror 13. The light beam F originates from a scene such as the zone of interest ZI1 in the Earth surface when theimaging spectrophotometer 100 is implemented on board thesatellite 200. Reference letter P indicates the entrance pupil of thetelescope 10. The light beam F is reflected successively by theprimary mirror 11, then thesecondary mirror 12, and then thetertiary mirror 13, so as to be focussed in a focal plane FP10 of thetelescope 10. - The
spectrophotometer 20 may also be of any type known in the art. For example, it may have a Littrow configuration, withfirst mirror 21,second mirror 22,third mirror 23, and a dispersingelement 24. Thespectrophotometer 20 is designed to image a content of the focal plane FP10 of thetelescope 10 within an output focal plane FP20. The light which is transmitted through the telescope focal plane FP10 is reflected first by thethird mirror 23, then thesecond mirror 22, then thefirst mirror 21, then is reflection-dispersed by the dispersingelement 24, then reflected again by thefirst mirror 21, then thesecond mirror 22, then thethird mirror 23, so as to be focussed in the output focal plane FP20. Thus, theimaging spectrophotometer 100 produces in the output focal plane FP20 an image of the scene which is combined with the pattern of a mask that may be located within the telescope focal plane FP10. In addition, the scene image in the output focal plane FP20 is combined with dispersion produced by the dispersingelement 24. - Advantageously, the dispersing
element 24 may be based on at least one prism of a refracting transparent material, which is selected with respect to the operation wavelength range desired for theimaging spectrophotometer 100. For example, this wavelength range may extend from 400 nm (nanometer) to 2500 nm, and the dispersingelement 24 may be a combination of two prisms which are arranged serially in the Littrow configuration of thespectrophotometer 20. One of these prisms, labelled 24', may be out of a highly dispersing glass, and the other one, labelled 24", may be out of another transparent material which is selected for compensating a non-linear component of the wavelength dispersion which is produced by the prism 24'. The total dispersion rate which is obtained in this way, when expressed as a derivation of an angular deviation with respect to the wavelength, may be both important and almost constant over the whole wavelength range or a large part thereof. -
Reference number 2 denotes an image detector which is located at the output focal plane FP20. For implementing the invention, theimage detector 2 is of array-type, with pixels which are distributed in columns and rows in a well-known manner. Theimage detector 2 is oriented so that the pixel columns are parallel to a dispersion direction which is produced by thespectrophotometer 20 in the output focal plane FP20. -
Reference number 3 denotes an optional beam divider which may be located between thetertiary mirror 13 of thetelescope 10 and the telescope focal plane FP10 along the light path. For example, thebeam divider 3 may be a bi-prism designed for transmitting a first part of the light which propagates from thetelescope 10, and reflecting another part of this light. Hence, thebeam divider 3 duplicates the telescope focal plane FP10 into an additional telescope focal plane FP10'. Then, arranging an additional image detector 2' at this additional telescope focal plane FP10' allows recovering an image of the scene from light which has not been dispersed by thespectrophotometer 20. -
Reference numbers folding mirror 4 is located between the telescope focal plane FP10 and thethird mirror 23 along the light path at the entrance end of thespectrophotometer 20, and thefolding mirror 5 is located between thethird mirror 23 and the output focal plane FP20 at the output end of thespectrophotometer 20. The folding mirrors 4 and 5 may be useful for obtaining a compact design for thewhole imaging spectrophotometer 100. - According to
Figure 3a , amask 1 is light-blocking except within slits which transmit light. For example, themask 1 may be provided with four slits labelled S1 to S4. The slits S1-S4 are parallel and spaced apart from each other along a direction which is perpendicular to these slits. ls, ws and is denote respectively the slit length, the slit width and the inter-slit spacing, which are common to all slits. Themask 1 is to be located in the telescope focal plane FP10 (Figure 2a ), with the slits oriented so as to appear perpendicular to the dispersion direction which is produced by thespectrophotometer 20 in the output focal plane FP20. Put another way, the slits are oriented in the telescope focal plane FP10 so that their respective monochromatic images through thespectrophotometer 20 each extend longitudinally parallel to the pixel rows of theimage detector 2 within the output focal plane FP20. - Therefore, each one of the slits S1-S4 of the
mask 1 allows a scene strip which is imaged within this slit in the telescope focal plane FP10 to be re-imaged in the output focal plane FP20 parallel to the pixel row direction of theimage detector 2, and the light which corresponds to each location in this latter image is spread spectrally parallel to the pixel column direction. This results in a portion of theimage detector 2 being dedicated to each slit. Such detector portion has been called image-spectrum segment in the general part of this description, and it is an issue that two image-spectrum segments which correspond two neighboring ones of the slits S1-S4 do not overlap along the pixel column direction. Such overlapping suppression can be obtained by selecting appropriately the inter-slit spacing is with respect to the extension of the image-spectrum segments along the pixel column direction, or by limiting the length of the spectral range of the light which reaches theimage detector 2. The Man skilled in optics will be able to select appropriately these parameters of theimaging spectrophotometer 100, and the following values are only provided as a non-limiting example: - wavelength detection range of the image detector 2: 400 nm - 2500 nm focal length of the telescope 10: 937.5 mm (millimeter)
- slit width ws: about 15 µm (micrometer)
- slit length ls: about 15 mm
- inter-slit spacing is: 3.84 mm
- focal length of the spectrophotometer 20: 334.8 mm
- size of the image detector 2: 1024 pixels x 1024 pixels
- pixel pitch of the
image detector 2 along the column direction and the row direction: 15 µm - wavelength sampling pitch along the column direction of the image detector 2: 10 nm
- height of each image-spectrum segment along the column direction of the image detector 2: 210 pixels
- spacing between two neighboring image-spectrum segments along the column direction of the image detector 2: about 46 pixels
- In particular, the invention may be implemented using the NGP detector produced by Sofradir®, NGP standing for «Next Generation Panchromatic®», for the array-
type image detector 2. -
Figure 4 shows the allocation of the photosensitive area of theimage detector 2 which results from such implementation. C and R denote the column direction and the row direction of theimage detector 2, respectively. ISSn denotes the image-spectrum segment which corresponds to the slit n, n being an integer index from 1 to 4 in the implementation example considered. Within each image-spectrum segment ISSn, the longitudinal direction of the scene strip which is imaged, denoted x, is parallel to the row direction R, and the dispersion direction, denoted λ, is parallel to the column direction C. The four image-spectrum segments ISS1-ISS4 relate to four scene strips which are spaced apart in the zone of interest ZI1 perpendicular to the longitudinal direction of the strips. Furthermore, the first pixel row within each image-spectrum segment ISSn corresponds to the wavelength sampling interval 400 nm - 410 nm, and the last pixel row within each image-spectrum segment ISSn corresponds to wavelength sampling interval 2490 nm - 2500 nm. - First embodiments of the invention now described may be appropriate for increasing a value of a signal-to-noise ratio which exists upon multispectrally image-capturing one zone of interest, such as ZI1 for example. Photonic noise involved upon signal accumulation in one pixel of the
image detector 2 and signal readout is proportional to I1/2, where I is the image intensity read out for this pixel. Therefore, the pixel signal-to-noise ratio increases by a factor N1/2, where N is a number of iterations that are performed for the capture of one and same scene portion which is imaged into one pixel area. Then, the plurality of image-spectrum segments which are produced by the invention within the photosensitive area of theimage detector 2 allows that one and same scene strip is imaged and spectrally spread multiple times during the pushbroom-scanning of one complete zone of interest by the line-of-sight L of theimaging spectrophotometer 100. For example, as illustrated inFigure 5 , one and same scene strip is imaged at time t1 in the image-spectrum segment ISS1 through the slit S1, then at a later time t2 in the image-spectrum segment ISS2 through the slit S2, and again at a later time t3 in the image-spectrum segment ISS3 through the slit S3, and one more time at t4 in the image-spectrum segment ISS4 through the slit S4. Thus, adding or combining the signals which are read out from the pixels of the image-spectrum segments ISS1-ISS4 leads to a resulting image of the scene strip which corresponds to the four accumulation and read-out sequences, and which has a signal-to-noise ratio multiplied by afactor 41/2=2.Figure 5 illustrates the instant content of each image-spectrum segment at times t1, t2, t3 or t4, respectively, in the output focal plane FP20, due to the slits in the telescope focal plane and the effect of thespectrophotometer 20. The content represented is a scene SC but only four slit-limited strips of such scene are imaged at one time and simultaneously wavelength dispersed along the column direction in each image-spectrum segment, and the complete image of the scene SC as represented only results from the pushbroom-scanning. For such operation, acontroller 21 and a combiningunit 22 may be coupled with theimage detector 2, whereby thecontroller 21 triggers a series of successive accumulation and readout operations while the image of the scene SC moves across the slits S1-S4, and the combiningunit 22 adds the pixel signals read out from image-spectrum segments ISS1-ISS4 at the times t1-t4. Each of these pixel signal additions relates to pixels which are located identically within the separate image-spectrum segments ISS1-ISS4. Hence, the signal-to-noise increase principle is applied independently and simultaneously for each scene element which is imaged and for each wavelength sampling interval which corresponds to one pixel line within the image-spectrum segments ISS1-ISS4. Thecontroller 21 is located on board thesatellite 200 whereas the combiningunit 22 is preferably within a data processing ground station on the Earth E. For such resource distribution, the combiningunit 22 is coupled with the other components through appropriate transmission means, such as radio or optical communication means. - Second embodiments of the invention may be appropriate for decreasing the slowdown requirement that may apply for capturing a multispectral image of one zone of interest. The left part of
Figure 6 shows the telescope focal plane with the four mask slits S1-S4. The frame drawn in thin line and denoted ZI1' is the peripheral limit of the image of the zone of interest ZI1 in the telescope focal plane FP10. Then, during one and same accumulation and readout operation of thewhole image detector 2, signals are recovered simultaneously from four scene strips which are imaged on theimage detector 2 respectively through the four slits S1-S4. Then, it is sufficient to scan the scene at least along a distance which corresponds to the slit offset, as indicated by the arrow A, for obtaining multispectral information from the whole area of the zone of interest ZI1. Thus, when theimage detector 2 operates at the maximum full-frame readout frequency, the multi-slit configuration of themask 1 allows reducing the slowdown duration which is necessary for each zone of interest to be captured in a full-frame multispectral image. Thus, the slowdown area is reduced in length, allowing faster imagery, or increased slowdown rate. In addition, the jumps of the line-of-sight L which are necessary between the processing sequences of several zones of interest are also reduced in length on average. This allows in turn increasing the numbers of zones of interest which can be multispectrally image-captured during one and same travel of thesatellite 200 over these zones of interest. But in such operation, each one of the image-spectrum segments ISS1-ISS4 (right part ofFigure 6 ) outputs progressively in time one fourth of the complete multispectral image of the zone of interest ZI1, and the complete multispectral image is to be reconstructed by joining the multispectral image segments which are outputted simultaneously and in parallel by the image-spectrum segments ISS1-ISS4 of theimage detector 2. To this purpose, theimage detector 2 is to be coupled with the controller 21', but also with animage reconstruction unit 23 as shown inFigure 6 . The controller 21' is located on board thesatellite 200 whereas theimage reconstruction unit 23 may be within a data processing ground station on the Earth E. For such resource distribution, theimage reconstruction unit 23 is coupled with the other components through appropriate transmission means, such as radio or optical communication means. The controller 21' triggers a series of successive accumulation and readout operations while the image of the scene moves across one inter-slit space. Thereconstruction unit 23 gathers the line contents which are outputted successively by theimage detector 2 from all the image-spectrum segments ISS1-ISS4, separately for each wavelength sampling interval. - Embodiments of the
controllers 21 and 21', thecombination unit 22 and theimage reconstruction unit 23 are known from the Man skilled in the art, so that it is not necessary to describe them further herein. - It is also possible to obtain combined embodiments of the invention wherein for example, both image-spectrum segments ISS1 and ISS2 are operated as for the first embodiments described above, and also both image-spectrum segments ISS3 and ISS4 simultaneously and in parallel with the image-spectrum segments ISS1 and ISS2, but both segment pairs ISS1+ISS2 and ISS3+ISS4 are further operated as for the second embodiments.
-
Figure 3b corresponds toFigure 3a when the additional image detector 2' is implemented.Figure 3b further indicates pixel lines of the additional image detector 2' which are allowed to detect light originating from the scene and transmitted by thetelescope 10. These pixel lines of the additional image detector 2' may be selected from a two-dimensional pixel array by a detector controller in a well-known manner. These pixel lines are denoted L1-L5, and are interleaved with the slits S1-S4 when superposing both telescope focal planes and FP10'. The pixel line L5 is optional, and advantageously implemented for the second embodiments described above. Possibly, each one of the pixel lines L1-L5 of the image detector 2' thus implemented may be replaced by several pixel lines, for example two or three, which are next to one another in the two-dimensional pixel array of the image detector 2', and which may be read out using a binning process. - Possibly, the image detector 2' may be of a type similar as that of the
image detector 2. Alternatively, the image detector 2' may have a pixel pitch smaller than that of theimage detector 2, for example two or three times smaller, for obtaining a better resolution. - No spectral filtering may be implemented for the image detection which is performed using the image detector 2', so that it constitutes an imaging channel which may be called panchromatic. Put another way, the wavelength detection range which is effective for the imaging channel comprised of the
telescope 10, thebeam splitter 3 and the image detector 2' may only result from spectral windows of these optical components. - For such combination of the
image detectors 2 and 2' within theimaging spectrophotometer 100, thebeam splitter 3 may be designed so as to reflect and transmit in an interleaved manner, light sub-beams from thetelescope 10. As shown inFigure 3c , the reflected light sub-beams may be directed to thespectrophotometer 20, one sub-beam for each one of the slits S1-S4 of themask 1, and the transmitted light sub-beams may be directed to the image detector 2', one sub-beam for each one of the pixel lines L1-L5. For example, thebeam splitter 3 may be comprised of twoprisms 30 and 30' which are glued together on an interface, with this interface being provided with spaced-apart reflecting strips Re. One reflecting strip Re may be dedicated to each slit of themask 1, and two neighbouring strips Re are separated by a transmitting gap Tr dedicated of one of the pixel lines L1-L5. - It should be understood that the invention may be reproduced by changing secondary aspects thereof with respect to the detailed description just provided, while maintaining at least some of its advantages. It particular, the values cited, including the pixel pitch and numbers, wavelength sampling pitch, and also the slit design parameters and number, may be changed depending on the application of the invention. In particular, the following modifications may be implemented with respect to the embodiments described in detail, without involving inventive step:
- the wavelength-dispersion which is produced by the spectrophotometer may be generated by a single prism or by a grating, or by a combination of a prism and a grating, depending on the operation wavelength range;
- the folding mirrors 4 and 5 are optional, and the invention may be embodied without folding mirrors or using folding configurations which are different from that represented in
Figures 2 and2a ; - with respect to the embodiment illustrated in
Figures 2 ,2a and3c , the spectrophotometer and the additional image detector may be reversed between both light outputs of the beam divider; and - when a beam divider is arranged upstream the telescope focal plane, the additional image detector may be replaced with another spectrophotometer designed for operating in a wavelength range different or separate from that of the first spectrophotometer.
Claims (17)
- An imaging spectrophotometer (100) comprising:- a telescope (10), adapted for collecting light originating from a scene and for producing an image of the scene in a focal plane (FP10) of said telescope;- a mask (1) located in the telescope focal plane (FP10), said mask being provided with a slit so as to transmit selectively light rays which are sent by the telescope (10) through said slit; and- a spectrophotometer (20) arranged for imaging a content of the telescope focal plane (FP10) in an output focal plane (FP20), and having a dispersion direction which is contained in said output focal plane;the spectrophotometer (20) comprising at least one array-type image detector (2) which is arranged at the output focal plane (FP20), and oriented so that pixel columns of said image detector are parallel to the dispersion direction,
wherein the spectrophotometer (20) is oriented with respect to the mask (1) so that the dispersion direction is perpendicular to a longitudinal direction of an image of the slit in the output focal plane (FP20), whereby scene elements which are imaged in the slit by the telescope (10) are re-imaged by the spectrophotometer on the image detector (2) within the slit image, and a spectrum of the light originating from each of said scene elements is spread perpendicular to the longitudinal direction of the slit image, thus forming on the array-type image detector an image-spectrum segment which is associated with the slit,
characterized in that the mask (1) comprises several slits (S1-S4) which are parallel and spaced perpendicular to said slits, so that one respective image-spectrum segment (ISS1-ISS4) is associated with each slit separately from each other slit, and in that a spectral detection window of the spectrophotometer (20) together with an inter-slit spacing (is) of the mask are such that any two image-spectrum segments which are associated with two neighbouring ones of the slits do not overlap in the output focal plane (FP20). - The imaging spectrophotometer (100) of claim 1, wherein the mask (1) comprises at least three slits, preferably four slits (S1-S4), preferably less than nine slits, which are parallel, identical and evenly spaced.
- The imaging spectrophotometer (100) of claim 1 or 2, wherein the spectrophotometer (20) comprises a combination of two prims (24', 24") for producing dispersion, both prisms being selected so that an effective non-linear wavelength-dispersion component, as resulting from said two-prism combination, is reduced when compared to a separate non-linear wavelength-dispersion component of each of said two prisms.
- The imaging spectrophotometer (100) of any one of claims 1 to 3, further comprising:- a controller (21) suitable to cause the image detector (2) to perform a series of successive accumulation and read-out operations while the image of the scene which is formed by the telescope (10) moves in the telescope focal plane (FP10) in such a way that one and same scene strip is imaged in the output focal plane (FP20) successively through several ones of the slits (S1-S4); and- optionally a combining unit (22) suitable to combine contents from at least two of the image-spectrum segments (ISS1-ISS4), when said contents relate to said one and same scene strip but pertain to accumulation and read-out operations which have been performed separately.
- The imaging spectrophotometer of any one of claims 1 to 4, further comprising:- another controller (21') suitable to cause the image detector (2) to perform a series of successive accumulation and read-out operations while the image of the scene which is formed by the telescope (10) moves in the telescope focal plane (FP10) over a distance at least equal to a sum of a slit width (ws) and the inter-slit spacing (is); and- optionally an image reconstruction unit (23) suitable to form a full-frame multispectral image by gathering line contents which are outputted successively by the image detector (2) from all the image-spectrum segments (ISS1-ISS4).
- The imaging spectrophotometer (100) of any one of the preceding claims, further comprising:- a beam divider (3) arranged upstream the telescope focal plane (FP10) along a light propagation path within the telescope (10), and designed for allowing each slit (S1-S4) of the mask (1) to be entirely exposed to part of the light which is collected by the telescope, and for allowing another part of said light which is collected by the telescope to focus into another telescope focal plane (FP10') without further travelling through the spectrophotometer (20); and- an additional image detector set arranged at said another telescope focal plane (FP10') so as to capture scene images produced by the telescope (10) from said another part of the light which does not travel through the spectrophotometer (20).
- The imaging spectrophotometer (100) of claim 6, wherein the beam divider (3) is designed so as to direct respective first sub-beams of the light collected by the telescope (10) to the slits (S1-S4) of the mask (1), and respective second sub-beams of said light collected by the telescope to spaced-apart pixel lines (L1-L5) or spaced-apart pixel line groups of the additional image detector set, so that said pixel lines or pixel line groups of the additional image detector set are optically interleaved with the slits through said beam divider.
- The imaging spectrophotometer (100) of claim 6 or 7, wherein the additional image detector set is comprised of at least one another array-type image detector (2'), said pixel lines (L1-L5) or pixel line groups being spaced apart within a photosensitive surface of said another array-type image detector.
- The imaging spectrophotometer (100) of claims 5 and 7, wherein a number of the pixel lines (L1-L5) or pixel line groups of the additional image detector set equals a number of the slits (S1-S4) provided in the mask (1) plus one.
- Satellite (200) comprising a platform (201), an attitude control system (202) for varying an orientation of the platform, and an imaging spectrophotometer (100) according to any one of the preceding claims, said imaging spectrophotometer being supported rigidly by the platform, wherein the satellite further comprises steering means suitable for directing a line-of-sight of the imaging spectrophotometer (100) towards successive scene strips in a pushbroom manner, while the image detector (2) performs a series of successive accumulation and read-out operations.
- Satellite (200) according to claim 10, wherein the steering means are suitable to produce a slowdown of a scanning of a zone of interest in the scene by the line-of-sight of the imaging spectrophotometer (100) while the image detector (2) performs the series of successive accumulation and read-out operations for capturing a full-frame multispectral image.
- Satellite (200) according to claim 11, wherein the imaging spectrophotometer (100) meets claim 5, and the steering means are suitable to produce the slowdown of the scanning of the zone of interest in the scene by the line-of-sight of the imaging spectrophotometer limitedly within a scan duration in which the image of the scene which is formed by the telescope (10) moves in the telescope focal plane (FP10) over a distance comprised between the sum of the a width (ws) and the inter-slit spacing (is), and another sum of the slit width and twice the inter-slit spacing, and the attitude control system or steering mirror system is suitable to switch said line-of-sight of the imaging spectrophotometer to another zone of interest in the scene after said scan duration has ended.
- Process for capturing a multispectral image, comprising the following steps:- providing a satellite (200) according to any one of claims 10 to 12, said satellite being in orbit around the Earth;- directing the line-of-sight of the imaging spectrophotometer (100) towards a zone of interest on the Earth;- moving the line-of sight so that an image of a scene which is contained in the zone of interest moves in the telescope focal plane (FP10), and simultaneously controlling the array-type image detector (2) so as to perform a series of successive accumulation and read-out operations; and- transmitting read-out data which have been produced by the successive accumulation and read-out operations to a data processing unit, said data processing unit being suitable for producing the multispectral image from the read-out data.
- Process according to claim 13, wherein the data processing unit is located on the Earth, and the read-out data are transmitted from the satellite (200) to a ground receiving station and then forwarded to the data processing unit.
- Process for obtaining a multispectral image, said process comprising:- providing a data processing unit located on the Earth, said data processing unit being suitable for producing the multispectral image from the read-out data which have been produced by a process for capturing a multispectral image according to claim 13;- collecting the read-out data and transmitting said read-out data to the data processing unit; and- operating the data processing unit for producing the multispectral image from the read-out data.
- Process according to any one of claims 13 to 15, wherein the data processing unit is a combining unit (22) suitable to combine contents from at least two of the image-spectrum segments (ISS1-ISS4), when said contents relate to one and same scene strip but pertain to accumulation and read-out operations which have been performed separately.
- Process according to any one of claims 13 to 15, wherein the data processing unit is an image reconstruction unit (23) suitable to form a full-frame multispectral image by gathering line contents which are outputted successively by the image detector (2) from all the image-spectrum segments (ISS1-ISS4).
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1755445 | 2017-06-15 | ||
FR1755447 | 2017-06-15 |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3415882A1 true EP3415882A1 (en) | 2018-12-19 |
EP3415882B1 EP3415882B1 (en) | 2022-03-02 |
Family
ID=60262859
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17306289.4A Active EP3415882B1 (en) | 2017-06-15 | 2017-09-28 | Imaging spectrophotometer and satellite comprising the same |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP3415882B1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112525344A (en) * | 2020-11-11 | 2021-03-19 | 中国科学院空天信息创新研究院 | Installation and adjustment device of dispersion type imaging spectrometer |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0642005A1 (en) * | 1993-09-07 | 1995-03-08 | Thomson-Csf | Optical imaging device for the spectral analysis of a scene |
WO1999063311A1 (en) * | 1998-06-04 | 1999-12-09 | Raytheon Company | Multi-slit imaging spectrometer |
WO2002052233A2 (en) * | 2000-12-26 | 2002-07-04 | Honeywell International Inc. | Lightweight infrared camera |
US20100328659A1 (en) * | 2001-12-21 | 2010-12-30 | Andrew Bodkin | Hyperspectral Imaging Systems |
-
2017
- 2017-09-28 EP EP17306289.4A patent/EP3415882B1/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0642005A1 (en) * | 1993-09-07 | 1995-03-08 | Thomson-Csf | Optical imaging device for the spectral analysis of a scene |
WO1999063311A1 (en) * | 1998-06-04 | 1999-12-09 | Raytheon Company | Multi-slit imaging spectrometer |
WO2002052233A2 (en) * | 2000-12-26 | 2002-07-04 | Honeywell International Inc. | Lightweight infrared camera |
US20100328659A1 (en) * | 2001-12-21 | 2010-12-30 | Andrew Bodkin | Hyperspectral Imaging Systems |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112525344A (en) * | 2020-11-11 | 2021-03-19 | 中国科学院空天信息创新研究院 | Installation and adjustment device of dispersion type imaging spectrometer |
Also Published As
Publication number | Publication date |
---|---|
EP3415882B1 (en) | 2022-03-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2640311C (en) | Optically multiplexed imaging systems and methods of operation | |
US20140085629A1 (en) | Active Hyperspectral Imaging Systems | |
US6122051A (en) | Multi-slit spectrometer | |
US6831688B2 (en) | Multispectral or hyperspectral imaging system and method for tactical reconnaissance | |
US8351031B2 (en) | Single-shot spectral imager | |
US4264921A (en) | Apparatus for color or panchromatic imaging | |
US6211906B1 (en) | Computerized component variable interference filter imaging spectrometer system method and apparatus | |
US9055240B2 (en) | Polychromatic imaging method | |
Mein | The MSDP of THEMIS: Capabilities, first results and prospects | |
US20100253941A1 (en) | Coded Aperture Snapshot Spectral Imager and Method Therefor | |
CN107655569B (en) | Hyperspectral camera, hyperspectral imaging device and control method | |
CN107462329B (en) | Multispectral camera, multispectral imaging device and control method | |
CN206281570U (en) | Hyperspectral imager based on step optical filter | |
EP3026407B1 (en) | Fourier transform infrared spectrometer with enhanced readout speed | |
EP3415882B1 (en) | Imaging spectrophotometer and satellite comprising the same | |
JP4502787B2 (en) | Optical spectrum sensor | |
US20080024871A1 (en) | Optically multiplexed imaging systems and methods of operation | |
EP3190394A2 (en) | System and method for spectral imaging | |
US4724326A (en) | Off-chip time-delayed integration area array Fraunhofer line discriminator | |
WO2020100139A1 (en) | An optical module and method for extending field of view in hyperspectral imaging systems | |
US4234789A (en) | Observation optical apparatus | |
CA3102692A1 (en) | On-board optical observation instrument of variable spatial and spectral resolution | |
CN106052868A (en) | Staring multispectral imaging method | |
Couce et al. | A windowing/pushbroom hyperspectral imager | |
Rocadenbosch Burillo | Chap. 7. Imaging Systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20190725 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20211210 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 1472623 Country of ref document: AT Kind code of ref document: T Effective date: 20220315 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602017053977 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220602 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220602 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1472623 Country of ref document: AT Kind code of ref document: T Effective date: 20220302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220603 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220704 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220702 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602017053977 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
26N | No opposition filed |
Effective date: 20221205 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20220930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220930 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220928 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220930 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20230927 Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20170928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240918 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20240920 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240925 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20240918 Year of fee payment: 8 |