WO2022027336A1 - 一种滤光器、滤光方法及多光谱成像系统 - Google Patents
一种滤光器、滤光方法及多光谱成像系统 Download PDFInfo
- Publication number
- WO2022027336A1 WO2022027336A1 PCT/CN2020/107160 CN2020107160W WO2022027336A1 WO 2022027336 A1 WO2022027336 A1 WO 2022027336A1 CN 2020107160 W CN2020107160 W CN 2020107160W WO 2022027336 A1 WO2022027336 A1 WO 2022027336A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- light signal
- pattern
- unit
- optical
- filter
- Prior art date
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 109
- 238000001914 filtration Methods 0.000 title claims abstract description 56
- 238000000701 chemical imaging Methods 0.000 title claims abstract description 28
- 238000000034 method Methods 0.000 title claims abstract description 22
- 239000006185 dispersion Substances 0.000 claims abstract description 76
- 238000001514 detection method Methods 0.000 claims abstract description 12
- 238000011068 loading method Methods 0.000 claims description 37
- 230000003595 spectral effect Effects 0.000 claims description 22
- 239000013307 optical fiber Substances 0.000 claims description 15
- 230000005540 biological transmission Effects 0.000 claims description 8
- 238000001228 spectrum Methods 0.000 description 24
- 238000010586 diagram Methods 0.000 description 22
- 238000003384 imaging method Methods 0.000 description 16
- 238000002474 experimental method Methods 0.000 description 9
- 239000000835 fiber Substances 0.000 description 9
- 230000000694 effects Effects 0.000 description 8
- 230000005284 excitation Effects 0.000 description 8
- 230000008859 change Effects 0.000 description 6
- 239000002131 composite material Substances 0.000 description 5
- 238000010226 confocal imaging Methods 0.000 description 3
- 238000004624 confocal microscopy Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000717 retained effect Effects 0.000 description 3
- 230000007480 spreading Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001154 acute effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002073 fluorescence micrograph Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000000386 microscopy Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000002834 transmittance Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/10—Beam splitting or combining systems
Definitions
- the present invention relates to the field of optical technology, in particular to an optical filter, a corresponding optical filtering method, and a multispectral imaging system using the optical filter.
- Confocal microscopy is a microscopic imaging technique that uses a laser beam to scan the excitation sample point by point, and uses a pinhole to spatially filter the fluorescence signal to remove stray light from the non-focal plane. It is also a common multispectral imaging system.
- the meaning of the term "confocal" is that the aperture at the light source, the focal point in the sample and the aperture in front of the detector are conjugated to each other. Compared with widefield microscopy, confocal microscopy has good optical resolution and signal-to-noise ratio. Generally speaking, the central wavelength of fluorescence is different from the wavelength of the laser.
- multicolor or multispectral imaging is not a unique requirement of confocal microscopes.
- many imaging systems such as other types of fluorescence microscopes and multispectral imagers require time-sharing or simultaneous processing of multiple different wavelength bands. light signal.
- the conventional method for realizing multispectral imaging of such equipment including confocal microscopes is to install filter wheels, filter sets, etc., that is, filters need to be used to obtain fluorescence signals in corresponding wavelength bands, but optical filters
- the filter band of the filter is fixed.
- multiple filters or a multi-bandwidth filter must be installed to match different fluorescence. This method is not only expensive, slow, but also spectrally resolved. The frequency is low, the arbitrary multi-band tuning function is poor, and the imaging bandwidth also depends entirely on the filter manufacturer.
- the commercial confocal systems that can realize the adjustable filter function include Leica's TCS SP5, Nikon (Nikon). ) of C1si et al. Referring to Figure 1 of the description, it is a representative prior art.
- the system mainly includes excitation light module A, confocal imaging module B and multi-channel spectral imaging module C.
- the excitation light module A includes lasers with different wavelengths, dichroic mirrors 1-5, mirrors 6 and acousto-optic tunable filters 7;
- confocal imaging module B includes a dichroic mirror 8, an X-Y galvanometer mirror 9 and 10, scanning lens 11, tube lens 12, objective lens 13, sample 14, focusing lens 15 and aperture 16;
- multi-channel spectral imaging module C consists of collimating lens 17, beam splitting prism 18, first, second and third Focusing lenses 19 , 22 and 25 , first, second and third variable slits 20 , 23 and 26 and first, second and third detectors 21 , 24 and 27 are composed.
- the working principle of the tunable filter-confocal system is as follows: in the excitation light module A, lasers with different wavelengths are integrated into an optical path after passing through the corresponding dichroic mirrors, and then injected into the acousto-optic tunable filter. . Adjusting the corresponding parameters of the acousto-optic tunable filter can change the wavelength and intensity of the outgoing light.
- the laser light emitted from A is first reflected by the dichroic mirror 8 into the two X-Y galvanometers, and then passes through the scanning lens 11, the tube lens 12 and the objective lens 13 in sequence, and the beam is focused on the sample 14. superior.
- the two X-Y galvanometers can change the position of the excitation light on the sample, an image of an area on the sample can be obtained by scanning the focused beam with that area.
- the fluorescence returns along the same optical path and is filtered by the dichroic mirror 8 to remove the excitation light, and finally focused by the lens 15 onto the aperture 16 .
- the fluorescence filtered by the aperture directly enters the multi-channel spectral imaging module C. First, it is collimated by the collimating lens 17 and then enters the beam splitting prism 18.
- the beam splitting prism disperses the fluorescence to expand its spectrum in space, and is then focused by the first focusing lens 19 into a fringe containing spectral information.
- a first variable slit 20 is installed at the stripe, and the position and spacing of the slits are variable. Only after the relationship between the position and wavelength of the slits is determined, the system can realize tunable filtering. The filtered fluorescence can also be reflected by the mirror on the first variable slit 20 to the remaining two slits 23 and 26 , and filtered again to enter the second and third detectors 24 and 27 .
- the movement of the slit is driven by a motor, so the speed is limited.
- the movement of the mechanical device will inevitably bring about accuracy problems, which will lead to a certain error between the actual value and the calibration value.
- the number of slits is limited, so usually only three-channel filtering can be achieved at most. Therefore, the above-mentioned prior art cannot actually achieve arbitrary tunable filtering (arbitrary center wavelength, arbitrary bandwidth), which also affects the multispectral imaging capability of many imaging systems including confocal microscopes.
- the present invention proposes an optical filter with an arbitrarily tunable working band, a filtering method and a corresponding multi-spectral imaging system.
- the unit collimates the dispersed optical signal, and combines the control of the digital micromirror device DMD to realize the tuning of arbitrary central wavelengths and (multiple) bandwidths in the filtering technology. Therefore, compared with the prior art, the optical filter and the optical filtering method of the present invention can significantly improve the imaging capability of various multispectral imaging systems such as confocal microscopes.
- the optical filter of the present invention includes a dispersion unit for dispersing an incident light signal containing multiple wavelength bands; and a collimation unit for collimating the incident light signal dispersed by the dispersion unit; a digital micromirror device , which receives the incident light signal collimated by the collimating unit; the digital micromirror device reflects part of the light signal in the incident light signal as a reflected light signal back to the collimating unit under the control of the control unit; the mirror , the reflected light signal is incident on the reflector after passing through the collimation unit and the dispersion unit, and is reflected by the reflector into the detection light path; wherein, the incident light signal collimated by the collimation unit
- the optical axis of the DMD does not coincide with the optical axis of the reflected light signal reflected by the digital micromirror device.
- control unit includes a pattern generating unit and a pattern loading unit; the pattern generating unit can generate a loading pattern, and the loading pattern is composed of one or more basic patterns; the pattern loading unit loads the loading pattern Loaded into the digital micromirror device, so that part of the light signal in the incident light signal is reflected back to the collimating unit as a reflected light signal.
- the pattern generating unit can generate a plurality of the basic patterns, the basic patterns are determined by the angle between the hypotenuse in the pattern and the edge on the target surface of the digital micromirror device, and the pattern on the digital micromirror device. The length of the upper edge and/or the lower edge of the target surface of the micromirror device is determined; the plurality of basic patterns have the same angle and have different lengths.
- the dispersion unit is a first dispersion prism
- the collimation unit is a second dispersion prism; wherein, the light exit surface of the first dispersion prism is parallel to the light incident surface of the second dispersion prism.
- the structure of the first dispersion prism is the same as that of the second dispersion prism. Therefore, the functions of the dispersion unit and the collimation unit are realized by the double prism structure.
- the present invention also provides a light filtering method, which includes the following steps: dispersing an incident light signal containing multiple wavelength bands; collimating the dispersive incident light signal; using a digital micromirror device to receive the collimated light signal direct incident light signal; control the digital micromirror device to reflect part of the light signal in the incident light signal as a reflected light signal; use a mirror to reflect the reflected light signal into the detection light path; wherein, the The optical axis of the collimated incident optical signal does not coincide with the optical axis of the reflected optical signal reflected by the digital micromirror device.
- the operation of controlling the digital micromirror device to reflect part of the incident optical signal as a reflected optical signal is specifically implemented by loading a loading pattern into the digital micromirror device; wherein the The loading pattern is generated by the following steps:
- the basic pattern is determined by the angle between the hypotenuse in the pattern and the edge on the target surface of the digital micromirror device, and the length of the edge and/or the lower edge of the pattern on the target surface of the digital micromirror device. Determine; the plurality of basic patterns in the step S1 have the same angle and have different lengths.
- a first dispersion prism is used to disperse the incident light signal including multiple wavelength bands
- a second dispersion prism is used to collimate the dispersed incident light signal.
- the structures of the first dispersive prism and the second dispersive prism may be the same.
- the functions of dispersion and collimation are achieved by using a double prism structure.
- the present invention also provides a multispectral imaging system, which includes the above-mentioned optical filter, and an optical signal generating unit.
- the light signal generating unit may be, for example, a confocal microscope.
- the optical signal is transmitted between the confocal microscope and the optical filter by means of optical fiber transmission or spatial transmission.
- the optical filter, the optical filtering method and the corresponding multispectral imaging system of the present invention can easily adjust the filtering performance of the optical filter by manipulating the pattern loaded by the digital micromirror device DMD, which can not only realize the filtering of a single waveband but also Can achieve arbitrary multi-band multi-channel filtering. Since the tuning speed of the digital micromirror device DMD is extremely fast, which can reach 10 kHz or even several tens of kHz, the speed of the optical filter of the present invention will not affect the original imaging system such as confocal, so the above can be achieved. Imaging Systems Ultra-fast multispectral imaging.
- the digital micromirror device DMD in the system is fixed during the imaging process and does not require any mechanical movement, so there is no problem that the actual wavelength is different from the calibrated wavelength due to insufficient mechanical movement precision in the prior art. Therefore, the technical solution of the present invention can greatly improve the multispectral imaging performance of the multispectral imaging system.
- FIG. 1 is a schematic structural diagram of a confocal microscope with adjustable filter function in the prior art
- FIG. 2 is a schematic structural diagram of a confocal microscope system of the present invention
- Fig. 3 is the structure schematic diagram of the optical filter of the present invention.
- Fig. 4 is the principle schematic diagram of the first and second dispersive prism collimation dispersive beam of the present invention.
- 5a is a schematic diagram of the separation of fluorescent light beams of an optical filter of the present invention.
- 5b is a schematic diagram of the DMD control unit of the present invention.
- Fig. 7 is the schematic diagram of dispersion fringe of the present invention.
- Fig. 8 is the DMD use state diagram of the present invention.
- FIG. 10 is a schematic diagram of a series of basic patterns of the present invention.
- FIG. 11 is a schematic diagram of another series of basic patterns of the present invention.
- FIG. 13 is the experimental result one of filtering the spectrum of the LED light source according to the present invention.
- Fig. 14 is the experiment result 2 of filtering the spectrum of the LED light source according to the present invention.
- FIG. 15 is the third experiment result of filtering the spectrum of the LED light source according to the present invention.
- FIG. 16 is the experiment result four of filtering the spectrum of the LED light source according to the present invention.
- Fig. 17 is the experiment result five of filtering the spectrum of the LED light source according to the present invention.
- Fig. 18 is the experiment result six of filtering the spectrum of the LED light source according to the present invention.
- Fig. 19 is the resolution test chart of filtering the spectrum of the LED light source according to the present invention.
- Figure 21 is a schematic diagram of another series of basic patterns in different directions of the present invention.
- FIG. 22 is a schematic structural diagram of another embodiment of the confocal microscope system of the present invention.
- Prior art drawings 1-5- dichroic mirror, 6-reflector, 7- acousto-optic tunable filter, 8- dichroic mirror, 9- XY galvanometer, 10- XY galvanometer Mirror, 11-scanning lens, 12-tube lens, 13-objective lens, 14-sample, 15-focusing lens, 16-small hole, 17-collimating lens, 18-beam splitting prism, 19-first focusing lens, 20- First variable slit, 21-first detector, 22-second focusing lens, 23-second variable slit, 24-second detector, 25-third focusing lens, 26-third variable Slit, 27 - Third detector.
- Example drawings 201-laser, 202-collimating lens, 203-dichroic mirror, 204-galvanometer, 205-scanning lens, 206-tube lens, 207-objective lens, 208-sample, 209-focusing lens, 210-small hole, 211-fiber, 212-filter, 301-filter collimating lens, 302-reflector, 303-first dispersion prism, 304-second dispersion prism, 305-digital micromirror device, 306 - filter slit, 307 - filter focusing lens, 308 - detector.
- a confocal microscope is taken as an example to illustrate the optical filter, the optical filtering method and the corresponding imaging system of the present invention.
- the confocal microscope is a conventional multispectral imaging system, which generally includes a laser, a pinhole, a galvanometer, an objective lens, a dichroic mirror and a collimating lens.
- a laser a laser
- a pinhole a galvanometer
- an objective lens a dichroic mirror
- collimating lens a collimating lens
- the light beam from the laser is expanded to reach the dichroic mirror, the galvanometer, the scanning lens, the tube lens and the objective lens respectively.
- the fluorescence generated by excitation returns to the original way through the objective lens, tube lens, scanning lens, and galvanometer, and then is filtered by the dichroic mirror to remove the laser light and some stray light, and then is filtered by the pinhole and then coupled into the fiber.
- the fluorescent signal is transmitted in an optical fiber, and the other end of the optical fiber is a filter.
- the optical filter of the present invention includes, for example, an optical fiber entrance port, a filter collimating lens, a dispersive unit (for example, composed of a first dispersive prism), a collimation unit (for example, composed of a second dispersive prism), a digital micromirror device ( DMD), mirrors, slits, focusing lenses and detectors.
- the filter collimating lens such as a fiber collimating lens, collimates the fluorescence signal at the fiber port and sends it to the dispersion unit.
- the fluorescence signals of different wavelengths are expanded in parallel in space, and the dispersion expanded fluorescence signal passes through the collimation unit and is irradiated on the DMD.
- the fluorescent signal of a specific wavelength can be reflected off the original optical path and received by the detector.
- a confocal microscope is used as an example to describe what kind of light signal the light signal generating unit can generate.
- FIG 2 it is a schematic structural diagram of the confocal microscope system of the present invention.
- the laser 201 , the collimating lens 202 , the dichroic mirror 203 , the galvanometer 204 , the scanning lens 205 , the tube lens 206 , the objective lens 207 , the focusing lens 209 and the small hole 210 form a laser scanning confocal microscope.
- the confocal microscope is connected to the filter 212 through the optical fiber 211 .
- the optical path structure of the laser scanning confocal microscope is as follows: the laser light is emitted from the laser 201 and collimated into a laser beam with a suitable spot size after passing through the collimating lens 202. After selecting the corresponding dichroic mirror 203, the light beam can be transmitted through 203. The collimated light beam transmitted from the dichroic mirror 203 passes through the galvanometer mirror 204 , the scanning lens 205 , the tube lens 206 and the objective lens 207 in sequence, and finally the light beam is focused on the sample 208 . The fluorescence generated by exciting the sample 208 returns to the original path and is reflected by the dichroic mirror 203 to the focusing lens 209. The focusing lens 209 focuses the light beam on the small hole 210. Stray light.
- the above-mentioned stray light filtered fluorescence is transmitted to the optical filter 212 through the optical fiber 211 .
- the structure of the optical filter 212 of the present invention is shown in FIG. 3 of the specification.
- the filter 212 includes a filter collimating lens 301 , a mirror 302 , a dispersive unit 303 , a collimating unit 304 , a digital micromirror device (DMD) 305 , a filter slit 306 , a focusing lens 307 and a detector 308 .
- DMD digital micromirror device
- the function of the dispersion unit 303 is to expand the optical signals of different wavelength bands in space
- the function of the collimation unit 304 is to collimate the dispersed optical signal to ensure the realization of the subsequent filtering process.
- the dispersion unit 303 is, for example, a first dispersion prism
- the collimation unit 304 is, for example, a second dispersion prism, wherein the first dispersion prism and the second dispersion prism may be the same dispersion prism.
- those skilled in the art can know that as long as the modification can realize the above functions, it belongs to the scope of the present invention.
- the basic optical path structure of the optical filter 212 is as follows: the fluorescent signal filtered by the small hole 210 is coupled into the optical fiber 211 for transmission, and when the fluorescent light comes out from the other end of the optical fiber 211, it is collimated by the collimating lens 301 of the optical filter, and then A specific incident angle is incident on the first dispersing prism, namely the dispersing unit 303, and the first dispersing prism will disperse the light signal and spatially expand its spectrum. The spectrally expanded dispersive light is then incident on the second dispersive prism, namely the collimating unit 304, at a specific incident angle.
- the second dispersive prism will collimate the dispersive light, and the collimated dispersive light will finally be irradiated on the digital micromirror. device (DMD) 305.
- DMD digital micromirror. device
- the light beam slightly deviates from the original optical path and returns, that is, after passing through the second dispersive prism and the first dispersive prism again, it is irradiated on the reflecting mirror 302, and the reflecting mirror 302 changes the beam direction It is focused by the filter focusing lens 307 .
- a fixed slit 306 can also be placed in front of the filter focusing lens 307 for filtering out stray light in the optical path.
- the desired fluorescent signal is received by detector 308 .
- FIG. 4 of the description shows a schematic diagram of the principle of collimating the dispersed light beam by the dispersion unit 303 (the first dispersion prism) and the collimation unit 304 (the second dispersion prism). Since the dispersive prism can generally achieve the largest clear aperture when used at the smallest deflection angle, it is preferable that the fluorescence collimated by the filter collimating lens 301 is incident into the first dispersive prism with the smallest deflection angle. The placement of the prisms is shown in Figure 4 of the description.
- the first dispersive prism and the second dispersive prism need to be placed in parallel, that is, the light exit surface of the first dispersive prism and the light entrance surface of the second dispersive prism are both parallel.
- the collimated beam is spectrally expanded after passing through the first dispersive prism and forms a dispersive fringe.
- the dispersive fringes become a beam of collimated dispersive light after passing through the second dispersive prism, that is, the length of the fringes in the unfolding direction does not change due to the observation distance, so that the beams with different wavelength components are parallel to each other although the spatial positions are different. .
- the collimated scattered light can ensure that the beam is also a beam of collimated light after returning to the original path, and will not be expanded by dispersion.
- the digital micromirror device (DMD) 305 can reflect the light beam corresponding to the wavelength band that needs to be retained by loading a specific pattern through the control unit shown in FIG. 5b of the specification. The generation and loading of the specific pattern are described later.
- the incident light is perpendicular to each micromirror in the DMD, the reflected light will coincide with the incident light, so it is not easy to accurately select the wavelength band selected by the DMD, so it is necessary to make some small adjustments to some devices to make the reflection Light can be separated from incident light.
- the method adopted in this embodiment is to fine-tune the angle of the DMD305.
- the reflection direction of the micromirror corresponding to a specific wavelength band (selected and reserved wavelength band) in the DMD305 can be fine-tuned in the vertical direction and/or the horizontal direction, so that the selected part of the DMD is reserved.
- a small reflector 302 can be placed between the filter collimating lens 301 and the dispersion unit 303, so as to reflect the selected and retained reflected light returned from the original path to other directions for subsequent detection.
- the selection of each component is described by taking the application of the filter in the confocal microscope as an example. Fluorescence is emitted from the small hole of the confocal microscope. In general, the size of the small hole is between a dozen micrometers to several tens of micrometers. In order to achieve better matching, the diameter of the fiber 211 can preferably be Within the above range, preferably the fiber diameter of the optical fiber 211 is slightly larger than the diameter of the small hole, so as to ensure that the light from the small hole can be received by the optical fiber 211 . In a specific embodiment, the optical fiber 211 may be a jumper MM62.5/125 of HannStar Optical Fiber Communication.
- the filter collimating lens 301 is the preferred objective lens, and the NA of the objective lens must be greater than the NA of the fiber, so that the objective lens can well collect the beam emitted from the fiber.
- the filter collimating lens 301 is a Nikon lens.
- the present application uses a dispersion prism.
- the first two parameters to be determined are the wavelength range and the Abbe number.
- the applicable range of wavelength should match the fluorescence signal, and the Abbe number reflects the dispersion ability of the prism.
- the smaller the Abbe number the greater the dispersion ability, that is, the more obvious the spectral effect.
- the size of the prism also needs to be considered in the actual system design.
- the collimating unit 304 can also select the same prism, that is, the first dispersing prism and the second dispersing prism of the dispersing unit 303 and the collimating unit 304 are Thorlabs' N-SF11 equilateral dispersing prism, the specific model for PS853.
- DMD target size (diagonal length), resolution (corresponding to the number of micromirrors), spectral range, and efficiency should be considered.
- DMDs with a diagonal length of 0.65 to 0.9 inches are acceptable.
- the minimum standard of resolution is 1024*768.
- the spectral range should match the optical signal to be filtered, for example, for a confocal microscope imaging in the visible range, the spectral range is 400nm to 700nm.
- the digital micromirror device 305 may be TI's DLP discovery 4100 development kit (resolution 1024*768).
- the reflector 302 is selected to have a higher reflectivity in the wavelength band of the signal to be filtered, and the shape may be, for example, a rectangle to facilitate the reception of the returned light.
- GMEH-30*30-AL of Hengyang Optics can be selected.
- the filter slit 306 may be a self-made slit, and may specifically be a rectangular filter slit whose shape matches the shape of the returned light.
- the filter focusing lens 307 selects a double cemented achromatic lens with high transmittance corresponding to the waveband of the signal to be filtered.
- the diameter of the lens is preferably slightly larger, such as 30 mm, which is beneficial to the reception of the back detector.
- Hengyang Optical GLH31-030-050-VIS a spectrometer with a large detection range and high resolution can be selected, for example, a spectrometer from Princeton Instruments can be used as the detector 308 to check the filtering performance of the filter.
- the detector 308 is the corresponding wavelength band PMT with high detection efficiency, such as the PMT of Beijing Hamamatsu Photonics (model CH345).
- Fig. 6 it is the spectrum diagram of the experimental LED light source, wherein, Fig. 6a is the ideal spectrum of the LED light source, but the spectrogram of the LED light source in the actual experiment is shown in Fig. 6b, which is also the object of subsequent experiment filtering .
- Optical signals such as 1: 400 ⁇ 500nm; 2: 500 ⁇ 600nm; 3: 600 ⁇ 700nm; Expanded.
- the length of the dispersion fringes in the spreading direction will vary with the observation distance.
- a biprism structure is formed. Therefore, when the composite light passes through the biprism, the stripes shown in FIG. 7 will be formed, and Since the dispersed optical signal is also collimated, the dispersion fringes will not change in the spreading direction.
- the dispersion prism is used in this embodiment, the dispersion of the prism is non-linear. Therefore, the degree of dispersion of the optical signals in different wavelength bands is different, which is reflected in the figure, that is, the optical signals in different wavelength bands have along the direction of dispersion expansion. different lengths.
- the grating is used as the dispersion unit, since the grating is linearly dispersed, the optical signals of different wavelength bands will have the same degree of dispersion, that is to say, the optical signals of different wavelength bands will have the same length along the direction of dispersion expansion.
- dashed line 1, dashed line 2 and dashed line 3 respectively represent the demarcation line between two states in which the micromirror in the DMD can reflect the optical signal back to the detection optical path and cannot reflect the optical signal back to the detection optical path.
- the above-mentioned dividing line preferably can reach the state of dotted line 2 in the figure, that is, the dividing line is perpendicular to the dispersion fringes, so that the optical signals of different wavelength bands can be accurately separated.
- the dotted line 1 or dotted line 3 because it is not perpendicular to the stripes, in these two cases, the filtered light will be mixed with the light of other wavelength bands or the light of the desired wavelength band will be lost, which will cause impure filtering or filtering.
- a series of problems such as low efficiency will eventually lead to the reduction of spectral resolution. Therefore, the pattern loaded into the DMD described later needs to be such that the demarcation line between the two states of the micromirror that can reflect the optical signal back to the detection optical path and cannot reflect the optical signal back to the detection optical path is perpendicular to the direction in which the optical signal is unfolded ( That is, the length direction of the dispersion fringes, or the spreading direction of the dispersion fringes).
- the dotted line represents the horizontal plane
- the dotted line represents the diagonal line of the DMD target surface.
- the DMD needs to comply with the requirement that a certain diagonal of the target surface is parallel to the horizontal plane, so that the target surface of the DMD can be fully used. If the optical path is adjusted correctly, the dispersion fringes from the prism are also parallel to the horizontal plane. Therefore, the dispersion fringes are also parallel to the diagonal of the target. In general, it is sufficient to make the dispersion fringes coincide with a certain diagonal of the DMD target surface. In this way, the diagonal part with the longest dimension is used to correspond to the dispersion fringes, which can reduce the requirements for DMD. In order to form the state represented by the dotted line 2 in FIG.
- a loading pattern hereinafter. Once a suitable loading pattern is found, it can be loaded into the DMD by the pattern loading unit in the control unit to meet specific filtering needs.
- the DMD has a large number of pixels, so there will be many patterns that can be loaded on the DMD. In order to mark each pattern well, certain parameters are required to define these patterns. In the present invention, the parameters used are The angle and top length of the hypotenuse of the pattern. Referring to Fig. 9a, it is a pattern that can be loaded on the DMD, and the dotted line represents the diagonal line of the target surface. In this embodiment, the pattern can be understood as the black part. Black and white represent the two states of the DMD micromirror respectively. Black indicates that the DMD micromirror is deflected to -12 degrees, and white indicates that the DMD micromirror is deflected to 12 degrees.
- the black area on the DMD target is set here to reflect the spectrum of the area back and be detected by the detector 308, while the white represents the spectrum of this area on the DMD target. is not reflected back to the detector 308, ie is filtered out.
- the black part of the pattern has a hypotenuse, and the acute angle between this hypotenuse and the edge of the entire target surface is the parameter of the angle of the hypotenuse, and the length of the edge of the black pattern part on the target surface is the length of the upper edge. Two parameters can determine each pattern.
- Figure 9b is a diagram illustrating the effect of loading the pattern of Figure 9a onto a DMD.
- the rendering after loading to the DMD is equivalent to the original image in Figure 9a rotated clockwise by a certain angle.
- the hypotenuse of the black part will be perpendicular to the diagonal of the target, that is, also perpendicular to the dispersion fringes, so the effect of the dotted line 2 in FIG. 7 can be achieved.
- the angle of the hypotenuse is optimally between 40 and 50 degrees, different angles will have different spectral resolutions, and different upper edge lengths will correspond to different filtering bands.
- 10a to 10c are examples of patterns with the same upper edge length of 510 pixels, but with different angles of the hypotenuse.
- the angles of the hypotenuse are 40 degrees, 45 degrees, and 50 degrees, respectively.
- 11a to 11c are examples of patterns in which the angle of the hypotenuse is the same as that of 42 degrees, but the lengths of the upper edge are 100 pixels, 200 pixels and 600 pixels, respectively. These patterns all correspond to different spectral resolutions and different filter bands, which means that these patterns have different filter performances after being loaded on the DMD.
- the appropriate angle of the hypotenuse is determined according to the requirement of spectral resolution. Then, in the case of fixing the angle of the hypotenuse, change the parameter of the length of the upper edge to obtain a series of patterns, use these patterns as the basic patterns, and analyze the filtering bands of these basic patterns.
- the above-mentioned changing of the length of the upper edge may be performed in steps of a fixed interval.
- the step size of the interval can be 1 DMD pixel, so the basic pattern will include a fixed hypotenuse angle, the upper edge length is 1 DMD pixel, 2 DMD pixels, 3 DMD pixels, 4 DMD pixels... .
- the step size of the interval can also be 10 DMD pixels, so the basic pattern will include the angle of the fixed hypotenuse, the upper edge length is 10 DMD pixels, 20 DMD pixels, 30 DMD pixels, 40 DMD pixels... . As the length of the upper edge increases, the black pattern will fill the entire DMD target surface.
- the black pattern may take on two shapes, one is a triangle as shown in Figures 10 and 11, and the other is a trapezoid as shown in Figure 12, where the 12 is the case where the angle of the hypotenuse is 46 degrees and the length of the upper edge is 1000 pixels.
- the length of some sides may be greater than the length of the target surface.
- the left vertical side of the black triangle pattern in Figure 12 exceeds the left side of the target surface. long, so it has a trapezoid shape.
- the following processing method is set in the pattern generation program: when the angle is determined, when the side length of the black area of the basic pattern is not greater than the side length of the target surface, the basic The growth rule of the black area of the pattern is that the above edge length is used as a reference to increase the fixed step size; but when a certain side length of the black area of the basic pattern is greater than a certain side length of the target surface, it is necessary to use the lower edge and the upper edge. Together as a reference to meet the pattern's growth law.
- the length of the pattern on the lower edge of the DMD target surface is used as a reference to increase the fixed step size (for example, 10 pixels) .
- the above basic patterns are generated by the pattern generation unit in the control unit through a pattern generation program.
- the pattern generation program will generate many patterns. For example, the angle of the hypotenuse is 44 degrees, and the length of the upper edge is increased by 1 DMD pixel each time.
- 1801 basic patterns of triangles or trapezoids can be generated. These 1801 patterns have the same angle, but the lengths of the upper and lower edges are different. It is incremented by 1 DMD pixel value.
- these patterned pictures are loaded into the DMD in turn, and the spectral curve corresponding to each pattern can be measured, so that the collected spectral data corresponding to the basic pattern can be used as pre-collected data.
- the above basic pattern is used to obtain a loading pattern that meets the specific filtering requirements, and the generation of the loading pattern is also realized by the pattern generation unit.
- the generated loading pattern is loaded onto the DMD by the pattern loading unit of the control unit and tested with a spectrometer to confirm that the expected filtering effect can be achieved.
- FIGs 13-18 in the description are schematic diagrams of experimental results of filtering the spectrum shown in Figure 6b using the optical filter of the present invention.
- the figure on the left is a picture loaded on the digital micromirror device 305
- the figure on the right is a corresponding spectrum after filtering by the filter.
- FIG. 15 shows the band from 524nm to 600nm.
- the two patterns on the left in Figure 15(c) are loaded on the DMD, and the spectra on the right in Figure 15c can be obtained respectively.
- the required spectrum similar to that of Fig. 15(b) can be obtained by subtracting the upper spectrum from the lower spectrum on the right side of Fig. 15(c). Therefore, facing the required spectrum of Fig. 15(b), the The pattern on the lower left side of 15(c) is subtracted from the pattern on the upper left side to obtain the pattern corresponding to Fig. 15(a) as the loading pattern.
- the spectral curve examples corresponding to Figures 13 and 17 are only the results of a single channel.
- the single-channel patterns need to be merged into one image, and then loaded into the DMD to measure the spectral results. That is, after finding several single channels, select several single channels whose spectra do not overlap, and then combine the patterns of these single channels into one pattern as the loading pattern for the required multi-channel spectral filtering.
- Figure 18 of the description shows how to easily realize five-channel filter modulation using the optical filter of the present invention.
- the loading pattern is derived from the fusion of Figure 13, Figure 15, Figure 16 and Figure 17.
- the picture can be loaded into the DMD.
- the final loaded picture is a picture with the same pixels generated according to the number of pixels of the DMD.
- the DMD used in this embodiment is 1024*768 pixels. , so it is necessary to use the software Matlab to generate a picture of 1024*768 pixels.
- DMD is the DLP discovery 4100 development kit of Texas Instruments (TI). This type of DMD can use the software developed by TI to load images to the DMD. Of course, you can also use the software Labview to load images to the DMD.
- the principles of the two loading methods are similar, because A complete DMD is composed of an FPGA and a DMD target surface, so the micromirrors of the DMD are controlled by the FPGA.
- a series of basic patterns need to be generated according to the angle of the stripes and the lengths of multiple upper edges, these basic patterns are loaded onto the DMD in turn, and the spectrum corresponding to each pattern is measured at the same time.
- the loading patterns corresponding to Fig. 13 to Fig. 17 are obtained, and then the spectrometer is used to obtain a single-channel schematic diagram with non-overlapping intervals.
- the result of multi-channel is to fuse the loading pattern of single channel together, and then use the spectrometer to measure it.
- the loading patterns corresponding to the single-channel can be added together to achieve multi-channel.
- Figure 19 of the specification shows the spectral resolution of the filter in this embodiment.
- the dispersive unit whether it is a dispersive prism or a grating, its dispersive ability is limited.
- the Abbe number of the prism and the number of grating lines can reflect the size of their dispersive ability.
- DMD's micromirrors also have sizes, not infinitely fine. Therefore, the two parameters of the dispersion capability of the dispersion unit and the size of the DMD micromirror limit the spectral resolution of the tunable filter, that is, the minimum wavelength difference that can distinguish two wavelengths.
- the dispersion prism is non-linear dispersion, that is, the degree of dispersion in different wavelength bands is different in the whole working wavelength, so in order to evaluate the resolution of the optical filter in this embodiment, it is necessary to measure several resolutions in the working wavelength range. rate value.
- rate value For example, in the embodiment of filtering the above-mentioned LED light source, four graphs a, b, c, and d in FIG. 19 are used to show the spectra of four different working wavelength bands of 440-460 nm, 510-550 nm, 580 nm, and 660 nm, respectively. Resolutions, 3.4131, 7.3065, 9.6063, and 11.6618, have good performance for most scenarios.
- the beveled edges of the black areas may also be reversible.
- Figure 20 shows a DMD with a pixel count of 1920*1080. The placement of the DMD is different from the DMD above, so different patterns can be used.
- 20a to 20c respectively show the cases where the angle of the hypotenuse is 45 degrees, and the length of the upper edge is 300 pixels, 600 pixels and 1200 pixels.
- the angle of the hypotenuse may also be other angles. In some cases, the angle of the hypotenuse may also be 90 degrees, as shown in Figures 21a-21c.
- the optical filter of the present invention can easily adjust the filtering performance of the optical filter by manipulating the pattern loaded by the digital micromirror device 305, which can realize not only single-band filtering but also any multi-band filtering. Multi-channel filtering. Since the tuning speed of the digital micromirror device 305 is extremely fast, which can reach 10 kHz, or even several tens of kHz, the speed of the optical filter of the present invention will not affect the original imaging system such as confocal, so the above can be achieved. Imaging Systems Ultra-fast multispectral imaging. Moreover, the digital micromirror device 305 in the system is fixed during the imaging process and does not require any mechanical movement, so the problem that the actual wavelength is different from the calibrated wavelength due to insufficient mechanical movement precision in the prior art will not arise.
- the optical filter it is also feasible to replace the prism of the dispersive unit with a grating.
- the spectral energy of the grating is better than that of the prism, but since the grating is only efficient at the center wavelength, the efficiency at other wavelengths is the highest. Lower, so the energy efficiency is not as high as the technical solution using the prism.
- the optical fiber 211 may not be used, and the fluorescence after passing through the small hole 209 is no longer coupled into the optical fiber. Transmission, but directly using space transmission, so it can also play the same function.
- the present invention proposes an optical filter with an arbitrarily tunable working band, which can completely replace the optical filter or filter set in the prior art.
- the key points include that because the digital micromirror device DMD can be easily controlled, the optical filter can realize the tuning of any central wavelength and (multi) bandwidth, and the speed is relatively fast; the design of the dispersion unit and the collimation unit is used to perform the dispersion light. Collimation, beams of different wavelengths travel parallel to each other in space, which can ensure that each wavelength or band can correspond to the pixels of the DMD, so that the distance of the DMD is not limited by space; the light reflected by the DMD will not be dispersed , that is, the beams of all wavelengths are concentrated together. Therefore, compared with the prior art, the optical filter and the optical filtering method of the present invention can significantly improve the imaging capability of various multispectral imaging systems such as confocal microscopes.
Abstract
Description
Claims (13)
- 一种滤光器,包括用于使包含多个波段的入射光信号进行色散的色散单元,其特征在于,还包括:准直单元,用于将经过色散单元色散的入射光信号准直;数字微镜器件,接收所述准直单元准直后的入射光信号;所述数字微镜器件经过控制单元的控制将所述入射光信号中的部分光信号作为反射光信号反射回所述准直单元;反射镜,所述反射光信号经过所述准直单元和所述色散单元之后入射到所述反射镜上,并被反射镜反射到探测光路中;其中,所述经过准直单元准直的入射光信号的光轴与所述数字微镜器件反射的反射光信号的光轴不重合。
- 根据权利要求1所述的滤光器,其特征在于,所述控制单元包括图案生成单元和图案加载单元;所述图案生成单元能够生成加载图案,所述加载图案由一个或多个基础图案组合而成;所述图案加载单元将所述加载图案加载到所述数字微镜器件中,从而实现将所述入射光信号中的部分光信号作为反射光信号反射回所述准直单元。
- 根据权利要求2所述的滤波器,其特征在于,所述图案生成单元能够生成多个所述基础图案,所述基础图案由该图案中斜边与所述数字微镜器件靶面上边缘之间的角度,以及该图案于所述数字微镜器件靶面上边缘和/或下边缘的长度来确定;所述多个基础图案具有相同的所述角度,并且具有不同的所述长度。
- 根据权利要求1-3任一项所述的滤光器,其特征在于,所述色散单元是第一色散棱镜,所述准直单元是第二色散棱镜;其中,所述第一色散棱镜的光出射面平行于所述第二色散棱镜的光入射面。
- 根据权利要求4所述的滤光器,其特征在于,所述第一色散棱镜与所述第二色散棱镜结构相同。
- 一种滤光方法,将包含多个波段的入射光信号进行色散,其特征在于,还包括如下步骤:对经过色散的入射光信号进行准直;使用数字微镜器件接收所述准直后的入射光信号;控制所述数字微镜器件以将所述入射光信号中的部分光信号作为反射光信号反射;使用反射镜将所述反射光信号反射到探测光路中;其中,所述经过准直的入射光信号的光轴与所述数字微镜器件反射的反射光信号的光轴不重合。
- 根据权利要求6所述的滤光方法,其特征在于,所述控制所述数字微镜器件以将所述入射光信号中的部分光信号作为反射光信号反射的操作具体是通过将加载图案加载到所述数字微镜器件中来实现的;其中所述加载图案通过如下步骤生成:S1、构建多个基础图案;S2、将所述多个基础图案分别加载到所述数字微镜器件中,获得每个基础图案对应的光谱数据,并将这些光谱数据作为预采集数据;S3、根据滤光的需求和所述预采集数据,选择一个或多个所述基础图案组合形成所述加载图案。
- 根据权利要求7所述的滤波器,其特征在于,所述基础图案由该图案中斜边与所述数字微镜器件靶面上边缘之间的角度,以及该图案于所述数字微镜器件靶面上边缘和/或下边缘的长度来确定;所述步骤S1中的多个基础图案具有相同的所述角度,并且具有不同的所述长度。
- 根据权利要求6-8任一项所述的滤光方法,其特征在于,使用第一色散棱镜对所述包含多个波段的入射光信号进行色散,使用第二色散棱镜对经过色散的入射光信号进行准直。
- 根据权利要求9所述的滤光方法,其特征在于,所述第一色散棱镜与所述第二色散棱镜结构相同。
- 一种多光谱成像系统,其特征在于,包括权利要求1-5任一项所述的滤光器,以及光信号生成单元。
- 根据权利要求11所述的多光谱成像系统,其特征在于,所述光信号生成单元是共聚焦显微镜。
- 根据权利要求12所述的多光谱成像系统,其特征在于,所述共聚焦显微镜与所述滤光器之间通过光纤传输或者空间传输的方式传输光信号。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/107160 WO2022027336A1 (zh) | 2020-08-05 | 2020-08-05 | 一种滤光器、滤光方法及多光谱成像系统 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/107160 WO2022027336A1 (zh) | 2020-08-05 | 2020-08-05 | 一种滤光器、滤光方法及多光谱成像系统 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022027336A1 true WO2022027336A1 (zh) | 2022-02-10 |
Family
ID=80118677
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/107160 WO2022027336A1 (zh) | 2020-08-05 | 2020-08-05 | 一种滤光器、滤光方法及多光谱成像系统 |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2022027336A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114441495A (zh) * | 2022-02-13 | 2022-05-06 | 中国科学院长春光学精密机械与物理研究所 | 多色荧光显微成像系统 |
CN116148811A (zh) * | 2023-04-18 | 2023-05-23 | 锋睿领创(珠海)科技有限公司 | 基于波长异位分布平行光源的多光谱分段成像视觉系统 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103454774A (zh) * | 2013-09-16 | 2013-12-18 | 华中科技大学 | 一种光的滤波方法及光滤波器 |
US8810884B1 (en) * | 2012-02-14 | 2014-08-19 | Artur Olszak | Light sources for spectrally controlled interferometry |
CN105628200A (zh) * | 2015-12-24 | 2016-06-01 | 南京理工大学 | 计算光谱成像装置 |
CN107167929A (zh) * | 2017-06-12 | 2017-09-15 | 华南师范大学 | 基于dmd的双模式光学超分辨显微成像装置及方法 |
CN109683307A (zh) * | 2018-12-28 | 2019-04-26 | 中国科学院深圳先进技术研究院 | 一种基于dmd的调谐滤光装置及调谐滤光方法 |
-
2020
- 2020-08-05 WO PCT/CN2020/107160 patent/WO2022027336A1/zh active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8810884B1 (en) * | 2012-02-14 | 2014-08-19 | Artur Olszak | Light sources for spectrally controlled interferometry |
CN103454774A (zh) * | 2013-09-16 | 2013-12-18 | 华中科技大学 | 一种光的滤波方法及光滤波器 |
CN105628200A (zh) * | 2015-12-24 | 2016-06-01 | 南京理工大学 | 计算光谱成像装置 |
CN107167929A (zh) * | 2017-06-12 | 2017-09-15 | 华南师范大学 | 基于dmd的双模式光学超分辨显微成像装置及方法 |
CN109683307A (zh) * | 2018-12-28 | 2019-04-26 | 中国科学院深圳先进技术研究院 | 一种基于dmd的调谐滤光装置及调谐滤光方法 |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114441495A (zh) * | 2022-02-13 | 2022-05-06 | 中国科学院长春光学精密机械与物理研究所 | 多色荧光显微成像系统 |
CN114441495B (zh) * | 2022-02-13 | 2024-03-22 | 中国科学院长春光学精密机械与物理研究所 | 多色荧光显微成像系统 |
CN116148811A (zh) * | 2023-04-18 | 2023-05-23 | 锋睿领创(珠海)科技有限公司 | 基于波长异位分布平行光源的多光谱分段成像视觉系统 |
CN116148811B (zh) * | 2023-04-18 | 2023-06-20 | 锋睿领创(珠海)科技有限公司 | 基于波长异位分布平行光源的多光谱分段成像视觉系统 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10591417B2 (en) | Systems and methods for 4-D hyperspectral imaging | |
JP5953230B2 (ja) | 空間的な色分離による可変フィルタリングを用いたスペクトル検出器又はレーザ走査顕微鏡 | |
US6229635B1 (en) | Light sensing device | |
CN109196333B (zh) | 用于4-d高光谱成像的系统和方法 | |
JP6096814B2 (ja) | スペクトル検出を伴う光走査型顕微鏡 | |
US7605976B1 (en) | Method and device for changing light in an adjustable manner | |
JP6061958B2 (ja) | ソフトウェア定義式顕微鏡 | |
US20100314554A1 (en) | Device to illuminate an object with a multispectral light source and detect the spectrum of the emitted light | |
DE19835072A1 (de) | Anordnung zur Beleuchtung und/oder Detektion in einem Mikroskop | |
JP2000199855A (ja) | 走査型光学顕微鏡装置 | |
JP2021505940A (ja) | 顕微鏡システムおよびこの種の顕微鏡システムを用いた顕微鏡結像のための方法 | |
US11686928B2 (en) | Light microscope | |
WO2022027336A1 (zh) | 一种滤光器、滤光方法及多光谱成像系统 | |
US8189191B2 (en) | Spectroscopic imaging microscopy | |
US20140368904A1 (en) | Software Defined Microscope | |
JP3568626B2 (ja) | 走査型光学顕微鏡 | |
CN112098378A (zh) | 一种滤光器、滤光方法及多光谱成像系统 | |
JP7079509B2 (ja) | 計測装置及び照射装置 | |
JP5871149B2 (ja) | 顕微分光システム | |
US11947097B2 (en) | Bandpass filter for light having variable lower and upper cut-off wavelengths | |
JP5787151B2 (ja) | 分光ユニット及び走査型顕微鏡 | |
US20220229307A1 (en) | Spectrally Shaped Light Source | |
JP6459070B2 (ja) | 分光器及び顕微鏡システム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20947955 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20947955 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20947955 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10/08/2023) |