WO2021019830A1 - 粒子定量装置 - Google Patents

粒子定量装置 Download PDF

Info

Publication number
WO2021019830A1
WO2021019830A1 PCT/JP2020/011638 JP2020011638W WO2021019830A1 WO 2021019830 A1 WO2021019830 A1 WO 2021019830A1 JP 2020011638 W JP2020011638 W JP 2020011638W WO 2021019830 A1 WO2021019830 A1 WO 2021019830A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sample
particle
region
particles
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/011638
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
高橋　智
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi High Tech Corp
Original Assignee
Hitachi High Tech Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi High Tech Corp filed Critical Hitachi High Tech Corp
Priority to EP20846283.8A priority Critical patent/EP4006524B1/en
Priority to KR1020227001111A priority patent/KR102659470B1/ko
Priority to JP2021536609A priority patent/JP7282894B2/ja
Priority to US17/628,440 priority patent/US12326391B2/en
Publication of WO2021019830A1 publication Critical patent/WO2021019830A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1429—Signal processing
    • G01N15/1433—Signal processing using image recognition
    • C—CHEMISTRY; METALLURGY
    • C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12M—APPARATUS FOR ENZYMOLOGY OR MICROBIOLOGY; APPARATUS FOR CULTURING MICROORGANISMS FOR PRODUCING BIOMASS, FOR GROWING CELLS OR FOR OBTAINING FERMENTATION OR METABOLIC PRODUCTS, i.e. BIOREACTORS OR FERMENTERS
    • C12M41/00—Means for regulation, monitoring, measurement or control, e.g. flow regulation
    • C12M41/30—Means for regulation, monitoring, measurement or control, e.g. flow regulation of concentration
    • C12M41/36—Means for regulation, monitoring, measurement or control, e.g. flow regulation of concentration of biomass, e.g. colony counters or by turbidity measurements
    • C—CHEMISTRY; METALLURGY
    • C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12M—APPARATUS FOR ENZYMOLOGY OR MICROBIOLOGY; APPARATUS FOR CULTURING MICROORGANISMS FOR PRODUCING BIOMASS, FOR GROWING CELLS OR FOR OBTAINING FERMENTATION OR METABOLIC PRODUCTS, i.e. BIOREACTORS OR FERMENTERS
    • C12M41/00—Means for regulation, monitoring, measurement or control, e.g. flow regulation
    • C12M41/46—Means for regulation, monitoring, measurement or control, e.g. flow regulation of cellular or enzymatic activity or functionality, e.g. cell viability
    • C—CHEMISTRY; METALLURGY
    • C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12Q—MEASURING OR TESTING PROCESSES INVOLVING ENZYMES, NUCLEIC ACIDS OR MICROORGANISMS; COMPOSITIONS OR TEST PAPERS THEREFOR; PROCESSES OF PREPARING SUCH COMPOSITIONS; CONDITION-RESPONSIVE CONTROL IN MICROBIOLOGICAL OR ENZYMOLOGICAL PROCESSES
    • C12Q1/00—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions
    • C12Q1/02—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions involving viable microorganisms
    • C12Q1/04—Determining presence or kind of microorganism; Use of selective media for testing antibiotics or bacteriocides; Compositions containing a chemical indicator therefor
    • C12Q1/06—Quantitative determination
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/06—Investigating concentration of particle suspensions
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1429—Signal processing
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1468—Optical investigation techniques, e.g. flow cytometry with spatial resolution of the texture or inner structure of the particle
    • G01N15/147—Optical investigation techniques, e.g. flow cytometry with spatial resolution of the texture or inner structure of the particle the analysis being performed on a sample stream
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N33/00—Investigating or analysing materials by specific methods not covered by groups G01N1/00 - G01N31/00
    • G01N33/48—Biological material, e.g. blood, urine; Haemocytometers
    • G01N33/50—Chemical analysis of biological material, e.g. blood, urine; Testing involving biospecific ligand binding methods; Immunological testing
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N33/00—Investigating or analysing materials by specific methods not covered by groups G01N1/00 - G01N31/00
    • G01N33/48—Biological material, e.g. blood, urine; Haemocytometers
    • G01N33/50—Chemical analysis of biological material, e.g. blood, urine; Testing involving biospecific ligand binding methods; Immunological testing
    • G01N33/5005—Chemical analysis of biological material, e.g. blood, urine; Testing involving biospecific ligand binding methods; Immunological testing involving human or animal cells
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/168—Segmentation; Edge detection involving transform domain methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/06—Investigating concentration of particle suspensions
    • G01N15/075—Investigating concentration of particle suspensions by optical means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N2015/1006—Investigating individual particles for cytology
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N2015/1486—Counting the particles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N2021/1765—Method using an image detector and processing of image signal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image

Definitions

  • the present invention relates to a particle quantifying device.
  • particles are identified and counted in units of one, so changes in the number of particles can be detected with high sensitivity even when the concentration is low or the number of particles is small, and high-sensitivity and high-precision detection is possible.
  • the conventional technology has a problem that the range of the number of particles that can accurately recognize particles is narrow.
  • the particle counting method as in Patent Document 2 for example, when the number of particles increases, the contact between particles and the overlap between particles increase stochastically, and individual identification of particles becomes difficult.
  • the distribution is usually represented by the Poisson distribution, and a simple counting model is known (Fig. 12). That is, when the number of particles existing in a certain area increases, the efficiency of counting the particles individually decreases. In the region where the number of particles is larger, the particles overlap each other in the light transmission direction. As a result, the transmitted light intensity is lowered as a whole due to multiple scattering or the like, so that the acquired image becomes unclear, the particle discrimination ability in image processing deteriorates, and counting is lost (FIG. 13).
  • the present invention has been made in view of such a situation, and an object of the present invention is to provide a particle quantification device that widens the range of the number of particles capable of accurately recognizing particles in a particulate sample.
  • An example of the particle quantifying device is An image acquisition means for acquiring a sample image representing a particulate sample, A data processing means for performing arithmetic processing on the sample image and It is a particle quantifying device equipped with The data processing means Obtain the frequency domain representation of the sample image and The frequency domain representation is separated into high frequency components and low frequency components. A high frequency image is acquired as a spatial region representation of the high frequency component. A low frequency image is acquired as a spatial region representation of the low frequency component. Recognizing the particulate sample based on the high frequency image and the low frequency image. It is characterized by that.
  • an example of the particle quantifying device is An image acquisition means for acquiring a sample image representing a particulate sample, A data processing means for performing arithmetic processing on the sample image and It is a particle quantifying device equipped with The data processing means Based on the sample image, a high numerical aperture image by light of a component that optically passes through a high numerical aperture optical path and a low numerical aperture image by light of a component that optically passes through a low numerical aperture optical path are acquired. And The particulate sample is quantified based on the high numerical aperture image and the low numerical aperture image. It is characterized by that.
  • an example of the particle quantifying device is An image acquisition means for acquiring a sample image representing a particulate sample, A data processing means for performing arithmetic processing on the sample image and It is a particle quantifying device equipped with The data processing means In the sample image, the particle region where the particles are present is extracted. Calculate the number of pixels in the particle area and The particulate sample is quantified based on the ratio of the number of pixels in the particle region to the number of pixels in the sample image. It is characterized by that.
  • the range of the number of particles that can accurately recognize the particles in the particulate sample becomes wider.
  • the schematic block diagram of the observation apparatus which concerns on 1st Embodiment.
  • the flowchart which shows an example of the operation of the observation apparatus of FIG.
  • An example of an image relating to the operation of the observation device of FIG. A histogram of the brightness of all pixels in a high frequency image.
  • the figure which shows the change according to the culture elapsed time of the total number of high-intensity pixels in a high-frequency image.
  • FIG. 1 is a schematic configuration diagram showing the configuration of the observation device 1 according to the first embodiment.
  • the observation device 1 is a device for observing a particulate sample.
  • a particulate sample means, for example, a sample containing particles, cells, or bacteria.
  • the meaning of "particle” is not particularly limited, but may be defined as including an organism such as a cell or a bacterium, or may be an inanimate object such as latex particles or polystyrene beads.
  • the observation device 1 has, as main components, an optical system 101 for illumination, a sample container 102, a pedestal 103, an XY stage 104, an objective lens 105, an objective lens actuator 106, and the like.
  • An image pickup camera 107 and a computer 108 are provided.
  • the illumination optical system 101 uniformly illuminates the particulate sample. For example, when the particulate sample is arranged on the bottom surface of the sample container 102, the bottom surface of the sample container 102 is uniformly illuminated.
  • the illumination optical system 101 is constructed from a light source, a Koehler illumination optical system, and the like.
  • the sample container 102 shall have a storage portion capable of holding a particulate sample.
  • Particulate samples can be provided, for example, as one or more sample solutions.
  • As the sample container 102 for example, a petri dish, a dish, or a microtiter plate or the like is used.
  • the sample container 102 holds a biologically related particulate sample such as cells or bacteria inside or in a well thereof.
  • the sample container 102 can be used for work such as cell culture or bacterial culture, and in particular, can be used for identification culture, culture for drug susceptibility testing, and the like.
  • the pedestal 103 can hold the sample container 102. It is preferable that the pedestal 103 has a structure in which the upper surface and the lower surface (that is, upstream or downstream in the optical path) of the measurement sample surface in the sample container 102 transmit light.
  • a transparent member may be used, or a void without a shielding structure or the like may be used.
  • the XY stage 104 can move the pedestal 103 on which the sample container 102 is placed in the X and Y directions.
  • the XY stage 104 may include a heater or the like for controlling the temperature of the sample container 102 (not shown).
  • a heater for example, a transparent glass heater can be arranged on the bottom surface or the periphery thereof.
  • the entire optical system may be surrounded by a heat insulating material, and the temperature inside may be controlled by a heater.
  • the objective lens 105 is held by the objective lens actuator 106.
  • the objective lens actuator 106 is an actuator that moves the objective lens 105 in the Z direction (illumination optical axis direction), and can scan the focal position of the objective lens 105 in the depth direction of the sample container 102.
  • the image pickup camera 107 can be focused on the measurement sample surface of the sample container 102.
  • the image pickup camera 107 functions as an image acquisition means for acquiring an image (sample image) representing a particulate sample. Such a configuration is suitable when the particles are translucent.
  • the sample image is a transmitted image, that is, an image composed of light transmitted through the particulate sample.
  • the imaging camera 107 is installed at the focal position of the objective lens 105, that is, at a position where an image of a particulate sample is formed. If the objective lens 105 is compatible with an infinity correction optical system, an imaging lens is installed between the imaging camera 107 and the objective lens 105.
  • the imaging camera 107 captures, for example, a sample image as a microscope image.
  • the image pickup camera 107 has a function of converting an captured sample image into an electric signal and outputting or transmitting the image. In this embodiment, the captured sample image is transferred to the computer 108.
  • An optical filter such as a colored glass filter or an interference filter may be appropriately inserted between the image pickup camera 107 and the objective lens 105, if necessary.
  • the computer 108 can be configured by using a known computer, and includes a calculation unit that performs various calculation processes and controls, and a storage unit that stores information.
  • the storage unit may include a temporary, volatile or transient storage medium such as a semiconductor memory device, or may include a non-temporary, non-volatile or non-transient storage medium such as a hard disk, and includes both of these. It may be.
  • the computer 108 may include an input device (mouse, keyboard, etc.) that receives input from the user and a display device (display, etc.) that displays the measurement result.
  • the computer 108 functions as a data processing means for performing arithmetic processing on the sample image, and executes a data processing step for performing arithmetic processing on the sample image.
  • cells or bacteria which are biologically related particulate samples, are used as the observation target of the observation device 1.
  • Cells or bacteria are cultured in 96-well microtiter plates and changes over time are measured.
  • FIG. 2 is an example of changes over time in an image (transmitted light image) when a particulate sample is cultured in a constant temperature state.
  • FIG. 2A is an image immediately after the start of observation
  • FIG. 2B is an image about 2 hours after the start of observation
  • FIG. 2C is an image 6 hours after the start of observation.
  • This figure shows how the particulate sample grows with the culturing time.
  • FIG. 2 (b) the particulate samples can be individually identified, but after 6 hours (FIG. 2 (c)), the particles come close to each other or overlap each other. , The outline of each particle becomes unclear.
  • the overall brightness is reduced, making it difficult to identify each particle.
  • the particle counting method as in Patent Document 2 it becomes difficult to identify the particles, and it becomes difficult to perform accurate counting.
  • an objective lens and an imaging lens may be used, and the image may be imaged on an imaging camera for detection.
  • the transmitted light image contains various information collected by each lens.
  • the straight component not the straight component but the high frequency component mainly generated by scattering / diffraction contributes greatly.
  • the straight-ahead component contributes to the magnitude of the intensity of the image.
  • FIG. 3 is a diagram for explaining the outline of the imaging optical system and the separation of the high numerical aperture component and the low numerical aperture component.
  • numerical aperture (Numerical aperture) may be simply abbreviated as "NA”.
  • the optical path L1 near the center of the lens corresponds to light with a low NA component. If the low NA component is used, an image mainly containing the straight light component can be acquired, so that the change in the transmitted light intensity can be acquired, and information equivalent to the intensity information measured by the so-called turbidity method can be obtained. can get. Further, the optical path L2 at the peripheral edge of the lens corresponds to light having a high NA component. If a high NA component is used, a scattered or diffracted component will be imaged, so that a particle image from which the background light intensity has been removed can be extracted.
  • the observation device 1 approximately analyzes the intensity information of the image (low NA image) with the low NA component. That is, the particles are identified based on the decrease in brightness that occurs when the particles are present, and the existing region (or the pixels constituting the region) is extracted.
  • the intensity information depending on the fine structure is acquired from the image with the high NA component (high NA image), and the existing region of the particles (or the pixels constituting the region) is extracted based on this. Then, the regions of both images are combined to form a particle region. This makes it possible to accurately recognize or quantify particles for a wider range of particle numbers (or particle concentrations).
  • the present inventor can substantially separate the high NA component and the low NA component by using the frequency domain representation of the image without actually preparing two types of optical detection systems. I paid attention to. For example, if the frequency domain representation (for example, Fourier transform) of a transmitted light image is acquired and this frequency domain representation is separated into a high frequency component and a low frequency component, a component substantially corresponding to a high NA component and a low NA component (for example). Alternatively, components that roughly correspond to these) can be obtained.
  • the frequency domain representation for example, Fourier transform
  • the observation device 1 constructs two types of images from one sample image, whereby the measurement time can be shortened and the device cost can be reduced.
  • the frequency domain representation can be obtained, for example, by a two-dimensional FFT (Fast Fourier Transform) process. Further, the obtained frequency domain representation is separated into a high frequency component and a low frequency component, and IFFT (Inverse Fast Fourier Transform) processing is performed on each component to obtain a high frequency image with the high frequency component and a low frequency component. It is possible to acquire a low frequency image with a frequency component.
  • FFT Fast Fourier Transform
  • IFFT Inverse Fast Fourier Transform
  • FIG. 4 is a flowchart showing an example of the operation of the observation device 1, that is, an example of a particle recognition method and a particle quantification method executed by the observation device 1.
  • FIG. 5 shows an example of an image related to the operation of the observation device 1.
  • bacteria cultured as a particulate sample are used.
  • the process shown in FIG. 4 is started by the observation device 1 acquiring a sample image (step S1).
  • Step S1 is an image acquisition step. This process is performed, for example, by the imaging camera 107 imaging a particulate sample.
  • the imaging camera 107 may transmit the captured sample image to the computer 108, and the computer 108 may receive the sample image.
  • FIG. 5A is an example of the sample image acquired in step S1. This example is a transmission image of cultured bacteria having a culture time of about 150 minutes.
  • the computer 108 acquires a spatial domain representation of the luminance values of the pixels of the sample image. For example, the brightness values of the pixels at each position arranged in two dimensions are acquired.
  • the computer 108 executes a data processing step including the subsequent steps S2 to S5.
  • the computer 108 acquires the frequency domain representation of the sample image (step S2). This process is performed, for example, by performing a Fourier transform or other frequency domain transform on the spatial domain representation. In this embodiment, a two-dimensional FFT is used as an example of the Fourier transform.
  • FIG. 5 (b) is an image of a frequency domain representation obtained by performing a two-dimensional FFT process on FIG. 5 (a).
  • the center represents a component having a frequency (spatial frequency) of 0, and the closer to the periphery, the higher the frequency.
  • the brightness of each pixel in the image represents the magnitude of the frequency component corresponding to that position.
  • the frequency domain representation is an image is taken as an example for explanation, but the frequency domain representation is not limited to the image and can be processed in any format.
  • the computer 108 separates the acquired frequency domain representation into a high frequency component and a low frequency component (step S3). Separation can be performed, for example, as shown in FIGS. 5 (c) and 5 (d).
  • a boundary line that separates the low frequency region including the point of frequency 0 and the high frequency region not including the point of frequency 0 is defined.
  • the boundary line can be, for example, a circle having a predetermined radius (for example, a radius of 6 pixels) centered on a point having a frequency of 0.
  • the inside of this circle is the low frequency region, and the outside of the circle is the high frequency region.
  • the low frequency domain is sometimes referred to as the region of interest (ROI).
  • the affiliation of points on the circumference can be arbitrarily determined, and may be included in the low frequency region or in the high frequency region.
  • the inside of the circle, that is, the low frequency region is masked to acquire the high frequency component.
  • the masking process is executed, for example, by setting the size of the component (luminance in the image) of the region to 0.
  • the low frequency component masking the outside of the circle, that is, the high frequency region is acquired.
  • step S4 the computer 108 acquires the spatial domain representation of each component (step S4). That is, a low frequency image is acquired as a spatial region representation of a low frequency component, and a high frequency image is acquired as a spatial region representation of a high frequency component.
  • This process is performed, for example, by executing the inverse conversion of the conversion in step S2 for the frequency domain representation.
  • the two-dimensional FFT is used in step S2
  • the two-dimensional IFFT is used in step S4.
  • the spatial area representation can be expressed in the same format as the spatial area representation (that is, the image) before the FFT conversion.
  • 5 (e) is a high frequency image composed of the high frequency components shown in FIG. 5 (c)
  • FIG. 5 (f) is a low frequency image composed of the low frequency components shown in FIG. 5 (d). In this way, two types of images having different characteristics are acquired.
  • the computer 108 recognizes or quantifies the particulate sample based on the obtained high-frequency image and low-frequency image (step S5).
  • the computer 108 recognizes a region in the sample image where particles are present.
  • the observation device 1 functions as a particle recognition device that recognizes the particulate sample.
  • the computer 108 quantifies the amount of particles contained in the sample image.
  • the observation device 1 functions as a particle quantification device for quantifying the particulate sample.
  • the "quantitative quantity" includes, in addition to accurately measuring the number of particles, calculating the approximate number of particles, calculating the concentration of particles, and the like.
  • the computer 108 may start the recognition or quantification of another sample image.
  • the loop shown in FIG. 4 may be executed at predetermined time intervals. That is, the imaging camera 107 may acquire sample images at predetermined time intervals, and the computer 108 may recognize or quantify the particulate sample for each sample image. In this way, it is possible to obtain information on changes over time that represent the state of increase in particles.
  • FIG. 6 is an example of a histogram of luminance for all pixels of a high frequency image. This histogram is of the high frequency image of FIG. 5 (e).
  • the "m” in the figure represents a time unit, and in this example, it represents a "minute”.
  • "0 m” represents the brightness distribution of the image at the start of culture, that is, the state in which the elapsed culture time is 0 minutes
  • "34 m" represents the brightness distribution of the image in the state where the elapsed culture time is 34 minutes.
  • FIG. 7 is a diagram showing changes in the total number of high-intensity pixels in a high-frequency image according to the elapsed culture time.
  • the "high-luminance pixel” means a pixel whose brightness exceeds a predetermined threshold brightness.
  • FIG. 7 is an example when the threshold brightness is about 6. This threshold is shown by a broken line in FIG.
  • the brightness of each pixel is represented by 256 steps from 0 to 255, where 0 is the lowest brightness (dark) and 255 is the highest brightness (bright).
  • the number of high-brightness pixels is increasing with time in a time zone of less than about 400 minutes.
  • the number of high-luminance pixels increases as the amount of particles increases, it can be said that a change in the amount of particles can be detected with high sensitivity by using a high-frequency image.
  • FIG. 8 is an example of a histogram of luminance for all pixels of a low frequency image. This histogram is from the low frequency image of FIG. 5 (f). Although the data for each elapsed culture time are not individually specified, FIG. 8 (a) shows the elapsed culture time of 0 to 184 minutes, and FIG. 8 (b) shows the elapsed culture time of 214 to 304 minutes. 8 (c) shows the culture elapsed time of 334 to 424 minutes.
  • FIG. 9 is a diagram showing changes in the total number of low-luminance pixels in a low-frequency image according to the elapsed culture time.
  • the "low-luminance pixel” means a pixel whose brightness is smaller than a predetermined threshold brightness.
  • FIG. 9 is an example when the threshold brightness is about 96. This threshold is shown by a broken line in FIG.
  • the number of low-brightness pixels will increase over time.
  • the number of low-luminance pixels increases as the amount of particles increases, it can be said that a change in the amount of particles can be detected with high sensitivity by using a low-frequency image.
  • the number of low-brightness pixels is almost constant. It is considered that this is because the particles are present in the entire sample image and are saturated. Even in this time zone, the number of low-luminance pixels does not decrease over time, so it can be said that the possibility of erroneously recognizing the amount of particles using a low-frequency image is low.
  • the computer 108 extracts the region where the particles exist in the high frequency image (first region), extracts the region where the particles exist in the low frequency image (second region), and is based on these two regions. Quantify the particulate sample.
  • the first region is extracted based on the brightness of each pixel of the high frequency image
  • the second region is extracted based on the brightness of each pixel of the low frequency image.
  • the logical sum of the first region in the high frequency image and the second region in the low frequency image may be acquired. Since the first region and the second region represent the region where the particles exist in each image, the particles can be efficiently quantified by combining them (for example, obtaining the logical sum). That is, the computer 108 obtains the logical sum (logical sum region) of the particle existence region by taking the logical sum of the first region and the second region, and quantifies the particulate sample based on this logical sum region. You may. In particular, the particulate sample may be quantified based on the area of the OR region. The area is represented by, for example, the number of pixels.
  • FIG. 10 shows the change in the total number of pixels in the OR region according to the elapsed culture time.
  • the elapsed culture time elapses from 0 minutes to 1000 minutes or more, the state changes from a state in which the number of particles is small to a state in which the particles are densely packed and the particles overlap each other.
  • FIG. 7 (example of a high frequency image) reflects the increase of particles only up to about 400 minutes, and after that, the number of pixels decreases as the number of particles increases, which may lead to erroneous detection of the amount of particles.
  • the increase of particles is accurately reflected for a while even after exceeding 400 minutes, and the number of pixels does not decrease even after saturation, and false detection of the amount of particles is suppressed. Will be done.
  • the increase of particles is not reflected until about 200 minutes, and until then, the number of pixels decreases as the number of particles increases, which leads to erroneous detection of the amount of particles.
  • the increase of particles is accurately reflected from about 100 minutes, and the number of pixels does not decrease until then, and erroneous detection of the amount of particles is suppressed.
  • a calibration curve obtained in advance can be used to determine the specific value of the amount of particles.
  • the computer 108 may obtain a calibration curve that associates a value representing the number of pixels in the OR region with the amount of particulate sample. Such a calibration curve may be stored in advance by the computer 108.
  • the computer 108 may quantify the particulate sample based on such a calibration curve and the number of pixels in the OR region.
  • the relationship between the culture time and the amount of particulate sample is obtained by a separate experiment or the like, and the culture time and the logical sum are obtained from the information shown in FIGS. 6 and 8. Find the relationship with the number of pixels in the area. Based on these two types of relationships, it is possible to create a calibration curve that associates the number of pixels in the OR region with the amount of the particulate sample. Then, as described above, the amount of the particulate sample in the sample image can be determined based on the calibration curve and the number of pixels in the logical sum region with respect to the desired sample image.
  • the particulate sample is quantified based on the number of pixels in the OR region as shown in FIG. 10, the range of the number of particles that can accurately recognize the particles becomes wider.
  • the extraction process of each area can be easily performed only by setting the threshold value, and the cost can be reduced.
  • the processing speed can be increased. Further, since two different types of images are created from one sample image, the time such as the exposure time for imaging is shortened as compared with the configuration in which each image is imaged under different optical conditions, and the device The size can be reduced, the processing speed can be improved, and the cost can be reduced.
  • the computer 108 may output information indicating a change over time for each loop. For example, information indicating the time-dependent change in the brightness of each pixel in the first region in the high-frequency image and information indicating the time-dependent change in the brightness of each pixel in the second region in the low-frequency image may be output. Alternatively, the computer 108 may output information representing the time-dependent change in the number of pixels in the first region in the high-frequency image and information representing the time-dependent change in the number of pixels in the second region in the low-frequency image.
  • Each threshold can be arbitrarily designed. For example, in the frequency domain representation (FIGS. 5B to 5D), it is not necessary to set the radius of the circle that separates the low frequency domain and the high frequency domain to 6, and when extracting high-luminance pixels in a high-frequency image. It is not necessary to set the threshold brightness (FIG. 6) of 6 to 6, and it is not necessary to set the threshold brightness (FIG. 8) to 96 when extracting low-luminance pixels in a low-frequency image.
  • the particulate sample was quantified based on the OR region, but it is not necessary to use the OR region.
  • the computer 108 acquires the non-zero-order frequency domain representation by removing the 0th-order diffraction component from the frequency domain representation of the sample image, and acquires the non-zero-order image as the spatial domain representation of the non-zero-order frequency domain representation.
  • the particulate sample may be recognized or quantified based on a non-zero order image.
  • the computer 108 uses a sample image to obtain a high numerical aperture (NA) image due to the light of the component that optically passes through the optical path having a high numerical aperture, and the light of the component that optically passes through the optical path having a low numerical aperture.
  • NA numerical aperture
  • the particulate sample may be recognized or quantified based on the high NA image and the low NA image.
  • the computer 108 extracts the particle region in which the particles are present in the sample image, calculates the number of pixels in the particle region, and prepares the particulate sample based on the ratio of the number of pixels in the particle region to the number of pixels in the sample image. It may be recognized or quantified.
  • the computer 108 may calculate the ratio of the number of pixels in the OR region to the total number of pixels of the sample image, and recognize or quantify the particulate sample based on this ratio.
  • the computer 108 may recognize or quantify the particulate sample based on the number of pixels in the first region and the number of pixels in the second region.
  • a correction value based on the brightness of the pixel may be determined in advance, this correction value may be multiplied by each pixel, and each component may be separated based on the result. It is also possible to apply a method in which the average number of pixels per particle is acquired in advance and the number of pixels in the OR region is divided by this average to calculate the number of particles. These methods also have the same effects as those in the first embodiment.
  • the amount of the particulate sample may be corrected based on the amount of decrease in the overall brightness (for example, the average value of the brightness of each pixel). Even an image or partial image in which the number of pixels in the OR region is saturated (that is, an image or partial image in which the number of low-luminance pixels no longer increases even if the number of particulate samples increases) is particulate. Increasing the number of samples may result in a decrease in overall brightness.
  • a function may be used that corrects the amount of the particulate sample according to the statistical amount of pixel brightness of a certain sample image or a specific part thereof.
  • the statistical amount of pixel brightness may be, for example, the peak brightness value in the brightness histogram as shown in FIG. 8 or the average value of pixel brightness.
  • a function that corrects to increase the amount of particulate sample as the peak brightness decreases can be used. Since the decrease in brightness is information that the overlap of particles is increasing, the measurement range of the amount of the particulate sample can be further expanded by making a correction based on the amount of decrease in brightness, and a wider dynamic range can be obtained. be able to.
  • the format of the sample image can be arbitrarily designed by those skilled in the art. For example, an 8-bit grayscale image may be used, or a 16-bit grayscale image may be used. Further, the color image may be converted into a grayscale image before use.
  • a transmitted light image is used as the sample image, but the sample image is not limited to the transmitted image, and any image that can be converted into a frequency domain representation can be used.
  • a phase contrast image, a differential interference contrast image, or the like that can be obtained by a known means can be used as a sample image. The process is the same in this case as well.
  • the particulate sample has translucency, but a sample having no translucency may be used. Further, as the particulate sample, for example, particles, cells or bacteria are used, but samples other than these may be used.
  • Image correction may be performed on the sample image. For example, image processing such as shading correction, sensitivity unevenness correction, noise removal, smoothing, etc. may be performed. Further, the light source unevenness, the detector sensitivity unevenness, and the like may be corrected. These corrections allow for more accurate quantification.
  • the optical system of the observation device does not have to have the configuration as shown in FIG. 1, and may be any configuration as long as it can provide an image acquisition means for acquiring a sample image. Further, the sample image may be acquired via a communication network, a portable storage medium, or the like without including an optical system.
  • the observation device may not be used to quantify the particulate sample.
  • it may be a device that identifies a region where particles exist and outputs the position or shape of the region.
  • the present disclosure is not limited to the above-described embodiments and modifications, but also includes various other modifications.
  • Each embodiment is described in detail in order to explain the present disclosure in an easy-to-understand manner, and is not necessarily limited to the one including all the configurations described. It is also possible to replace a part of the configuration of one embodiment with a part of the configuration of another embodiment, and it is also possible to add a part of the configuration of another embodiment to the configuration of one embodiment. Is. In addition, it is possible to add / delete / replace the configurations of other embodiments with respect to a part of the configurations of each embodiment.

Landscapes

  • Chemical & Material Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Immunology (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Physics & Mathematics (AREA)
  • Pathology (AREA)
  • Organic Chemistry (AREA)
  • Dispersion Chemistry (AREA)
  • Wood Science & Technology (AREA)
  • Zoology (AREA)
  • Signal Processing (AREA)
  • Microbiology (AREA)
  • Biotechnology (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Hematology (AREA)
  • Proteomics, Peptides & Aminoacids (AREA)
  • Urology & Nephrology (AREA)
  • Genetics & Genomics (AREA)
  • General Engineering & Computer Science (AREA)
  • Cell Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Toxicology (AREA)
  • Biophysics (AREA)
  • Sustainable Development (AREA)
  • Medicinal Chemistry (AREA)
  • Food Science & Technology (AREA)
  • Tropical Medicine & Parasitology (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
  • Apparatus Associated With Microorganisms And Enzymes (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)
PCT/JP2020/011638 2019-07-29 2020-03-17 粒子定量装置 Ceased WO2021019830A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP20846283.8A EP4006524B1 (en) 2019-07-29 2020-03-17 Particle quantifying device
KR1020227001111A KR102659470B1 (ko) 2019-07-29 2020-03-17 입자 정량 장치
JP2021536609A JP7282894B2 (ja) 2019-07-29 2020-03-17 粒子定量装置
US17/628,440 US12326391B2 (en) 2019-07-29 2020-03-17 Particle quantifying device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-138526 2019-07-29
JP2019138526 2019-07-29

Publications (1)

Publication Number Publication Date
WO2021019830A1 true WO2021019830A1 (ja) 2021-02-04

Family

ID=74228687

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/011638 Ceased WO2021019830A1 (ja) 2019-07-29 2020-03-17 粒子定量装置

Country Status (5)

Country Link
US (1) US12326391B2 (https=)
EP (1) EP4006524B1 (https=)
JP (1) JP7282894B2 (https=)
KR (1) KR102659470B1 (https=)
WO (1) WO2021019830A1 (https=)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115170641A (zh) * 2022-07-18 2022-10-11 合肥工业大学 一种碳烟积聚颗粒投影面积的测量方法
JPWO2023022117A1 (https=) * 2021-08-18 2023-02-23
WO2023032320A1 (ja) * 2021-09-03 2023-03-09 ソニーグループ株式会社 粒子解析システム、粒子解析方法及びフローサイトメータシステム
JP2023156889A (ja) * 2022-04-13 2023-10-25 公立大学法人大阪 蛍光ｘ線分析装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005218379A (ja) * 2004-02-06 2005-08-18 Olympus Corp 培養細胞の状態計測方法及び計測装置
WO2007136074A1 (ja) * 2006-05-22 2007-11-29 Nikon Corporation 観察装置
JP2012150140A (ja) * 2011-01-17 2012-08-09 Nikon Corp 顕微鏡装置及び観察方法
WO2015174011A1 (ja) * 2014-05-14 2015-11-19 ソニー株式会社 画像処理装置、画像処理プログラム、画像処理方法及び顕微鏡システム
WO2018128080A1 (ja) * 2017-01-06 2018-07-12 オリンパス株式会社 細胞観察システム
JP2019138526A (ja) 2018-02-09 2019-08-22 タイガー魔法瓶株式会社 オーブントースター

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001307066A (ja) * 2000-04-21 2001-11-02 Matsushita Electric Ind Co Ltd 細胞画像分析装置および細胞画像分析方法
US6785411B1 (en) 1999-08-05 2004-08-31 Matsushita Electric Industrial Co., Ltd. Image analyzing apparatus and image analyzing method
JP2002214228A (ja) 2001-01-12 2002-07-31 Matsushita Electric Ind Co Ltd 生体試料の評価装置および評価方法
US7564622B2 (en) 2003-12-12 2009-07-21 Olympus Corporation Methods for implement microscopy and microscopic measurement as well as microscope and apparatus for implementing them
KR100790980B1 (ko) * 2006-02-09 2008-01-02 삼성전자주식회사 영상 신호의 주파수 성분에 따라 후처리를 달리하는 영상신호의 후처리 회로
JP5941674B2 (ja) * 2011-12-28 2016-06-29 オリンパス株式会社 細胞輪郭線形成装置及びその方法、細胞輪郭線形成プログラム
CN103278431B (zh) 2013-04-22 2015-02-25 西安交通大学 一种基于视频获取的润滑油磨粒在线监测与分析方法
FR3061297B1 (fr) * 2016-12-28 2019-05-24 Commissariat A L'energie Atomique Et Aux Energies Alternatives Procede de comptage et de caracterisation de particules dans un fluide en mouvement
EP3902457A1 (en) * 2018-12-27 2021-11-03 Johnson & Johnson Consumer Inc. Device and method for selective application of topical composition using dynamic threshold values
CN110428399B (zh) * 2019-07-05 2022-06-14 百度在线网络技术（北京）有限公司 用于检测图像的方法、装置、设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005218379A (ja) * 2004-02-06 2005-08-18 Olympus Corp 培養細胞の状態計測方法及び計測装置
WO2007136074A1 (ja) * 2006-05-22 2007-11-29 Nikon Corporation 観察装置
JP2012150140A (ja) * 2011-01-17 2012-08-09 Nikon Corp 顕微鏡装置及び観察方法
WO2015174011A1 (ja) * 2014-05-14 2015-11-19 ソニー株式会社 画像処理装置、画像処理プログラム、画像処理方法及び顕微鏡システム
WO2018128080A1 (ja) * 2017-01-06 2018-07-12 オリンパス株式会社 細胞観察システム
JP2019138526A (ja) 2018-02-09 2019-08-22 タイガー魔法瓶株式会社 オーブントースター

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4006524A4

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2023022117A1 (https=) * 2021-08-18 2023-02-23
WO2023032320A1 (ja) * 2021-09-03 2023-03-09 ソニーグループ株式会社 粒子解析システム、粒子解析方法及びフローサイトメータシステム
JPWO2023032320A1 (https=) * 2021-09-03 2023-03-09
EP4386358A4 (en) * 2021-09-03 2024-12-25 Sony Group Corporation PARTICLE ANALYSIS SYSTEM, PARTICLE ANALYSIS METHODS AND FLOW CYTOMETER SYSTEM
US12535401B2 (en) 2021-09-03 2026-01-27 Sony Group Corporation Particle analysis system, particle analysis method, and flow cytometer system
JP2023156889A (ja) * 2022-04-13 2023-10-25 公立大学法人大阪 蛍光ｘ線分析装置
JP7412794B2 (ja) 2022-04-13 2024-01-15 公立大学法人大阪 蛍光ｘ線分析装置
CN115170641A (zh) * 2022-07-18 2022-10-11 合肥工业大学 一种碳烟积聚颗粒投影面积的测量方法

Also Published As

Publication number Publication date
EP4006524A4 (en) 2023-08-23
KR20220020920A (ko) 2022-02-21
US12326391B2 (en) 2025-06-10
JP7282894B2 (ja) 2023-05-29
EP4006524A1 (en) 2022-06-01
US20220268689A1 (en) 2022-08-25
JPWO2021019830A1 (https=) 2021-02-04
EP4006524B1 (en) 2024-08-14
KR102659470B1 (ko) 2024-04-23

Similar Documents

Publication Publication Date Title
US20230324278A1 (en) Assay accuracy improvement
US11674883B2 (en) Image-based assay performance improvement
JP7282894B2 (ja) 粒子定量装置
JP2022528693A (ja) アッセイ精度及び信頼性の向上
JP7141537B2 (ja) 粒子定量装置
CN105190690A (zh) 显微图像的图像质量评估
US12092577B2 (en) Method and system for analysing FluoroSpot assays
EP2593771B1 (en) Methods, storage mediums, and systems for analyzing particle quantity and distribution within an imaging region of an assay analysis system and for evaluating the performance of a focusing routing performed on an assay analysis system
US12095980B2 (en) Automated fluorescence imaging and single cell segmentation
CN116783660A (zh) 减少测定错误
CN106574224B (zh) 用于检测生物粒子的存在或缺失的方法
WO2019230624A1 (ja) 粒子径分布測定装置及び粒子径分布測定装置用プログラム
Kotyk et al. Detection of dead stained microscopic cells based on color intensity and contrast
WO2024180774A1 (ja) 異物および粒子の凝集体の検査装置および検査方法
TWI324750B (en) Microscopic image analysis method and system of microfluidic cells
EP4339591A1 (en) Feature amount calculation device, feature amount calculation method, and program
EP4390376A1 (en) An improved method for performing fluorescence measurement on a sample
CN116685827B (zh) 表面粗糙度测量系统
AKSOY et al. Developing a Solution Density Analysis Program with Image Processing Technique
CN120070349A (zh) 一种ebv抗体滴度的评估方法、装置、介质及存储设备
JP2023543800A (ja) アーテファクトマスキングを用いて生体サンプルを分析するための方法
CN121459348A (zh) 一种基于多模态光学与光谱融合的高通量菌落检测传感器及其检测方法
CN114821578A (zh) 一种基于显微成像的微球计数方法、系统和电子设备
HK40085476A (en) Automated fluorescence imaging and single cell segmentation
TarasKotyk et al. Detection of Dead stained microscopic cells based on Color Intensity and Contrast

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20846283

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
ENP Entry into the national phase

Ref document number: 2021536609

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20227001111

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020846283

Country of ref document: EP

Effective date: 20220228

WWG Wipo information: grant in national office

Ref document number: 17628440

Country of ref document: US