US11202040B2 - System and method for monitoring a sample - Google Patents
System and method for monitoring a sample Download PDFInfo
- Publication number
- US11202040B2 US11202040B2 US17/050,213 US201917050213A US11202040B2 US 11202040 B2 US11202040 B2 US 11202040B2 US 201917050213 A US201917050213 A US 201917050213A US 11202040 B2 US11202040 B2 US 11202040B2
- Authority
- US
- United States
- Prior art keywords
- image data
- collected
- detector array
- data
- correlation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01H—MEASUREMENT OF MECHANICAL VIBRATIONS OR ULTRASONIC, SONIC OR INFRASONIC WAVES
- G01H9/00—Measuring mechanical vibrations or ultrasonic, sonic or infrasonic waves by using radiation-sensitive means, e.g. optical means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/75—Circuitry for compensating brightness variation in the scene by influencing optical camera components
-
- H04N5/2256—
-
- H04N5/23229—
-
- H04N5/238—
Definitions
- the present invention relates to techniques for monitoring a sample or tissue, and is highly relevant for optical monitoring of one or more parameters of a sample or body.
- Optical monitoring of object or body provide reliable and non-invasive monitoring with limited interference with the target monitored region.
- Various techniques have been described for optical monitoring enabling collection of one or more parameter of selected sample using temporal variation self-interference pattern of light scattered from the inspected region.
- U.S. Pat. No. 8,638,991 presents a method for imaging an object.
- the method comprises imaging a coherent speckle pattern propagating from an object, using an imaging system being focused on a plane displaced from the object.
- US 2013/0144137 and US 2014/0148658 present a system and method for use in monitoring one or more conditions of a subject's body.
- the system includes a control unit which includes an input port for receiving image data, a memory utility, and a processor utility.
- the image data is indicative of data measured by a pixel detector array and is in the form of a sequence of speckle patterns generated by a portion of the subject's body in response to illumination thereof by coherent light according to a certain sampling time pattern.
- the memory utility stores one or more predetermined models, the model comprising data indicative of a relation between one or more measurable parameters and one or more conditions of the subject's body.
- the processor utility is configured and operable for processing the image data to determine one or more corresponding body conditions; and generating output data indicative of the corresponding body conditions.
- optical monitoring provides for acquiring parameters of a sample and simple and non-invasive technique.
- Some efficient optical monitoring techniques utilize correlation in secondary speckle patterns formed in light returning from the inspected region.
- one or more inspection regions are illuminated by coherent illumination.
- Light returning from the region(s) is collected by corresponding collection module generating image data indicative of an intermediate optical plane located between the inspected region and the collection unit, providing image data, which in some configurations relate to defocused image of the inspection region.
- This image data includes patterns of self-interference of light components scattered from the region, generally known as secondary speckle patterns.
- the speckle-based monitoring techniques utilizes correlations between speckle patterns collected at a selected sampling rate for determining changes in location, orientation and curvature (generally referred to as tilt) of the inspection regions, indicating nanometric changes and nano-vibrations of the inspected regions.
- appropriate monitoring of a sample may be associated with collection of image data pieces, indicative of secondary speckle patterns, at a sufficiently high sampling rate as generally indicated by Nyquist theorem.
- determining characteristics of the sample may be associated with processing a plurality of image data pieces and determining suitable correlation functions between consecutive image data pieces, or generally between image data pieces collected at two or more different temporal instances. This may require high processing power and time and limit the system's ability to provide real time analysis of the sample properties.
- the present invention provides a system for monitoring parameters of a sample utilizing collection of light components, associated with coherent illumination, that are reflected or scattered from an inspection region for generating one or more sequences of image data pieces indicative of secondary speckle patterns in the collected light.
- the system utilizes the collected image data for determining correlations between speckle patterns collected at different times (e.g. consecutively collected image data pieces) with reduced processing power and/or time.
- the system of the present invention utilizes optical, analog and/or partially digital processing of the collected image data for determining correlation functions.
- the system of the present invention comprises a collection module comprising optical arrangement and detector array arranged for collection of defocused images of a selected inspection region.
- the detector array is associated with circuitry configured for providing output electronic data indicative of correlations between image data pieces of two or more different times in accordance with readout operation of the detector array.
- the detector array may be configured as rolling shutter detector array, and the optical arrangement comprises one or more replication elements configured for directing two or more replications of collected images toward corresponding different regions (pixel rows) of the detector array.
- the detector array is further associated with circuitry configured to collecting analog output of different pixel rows upon readout and adding analog readout data of pairs of consecutive pixel rows.
- the circuitry comprises at least two analog-to-digital (A2D) conversion modules having selected different bit thresholds and configured for performing analog-to-digital conversion of the combined analog output of pairs of consecutive pixel rows, providing output data indicative of correlation of pixels of said consecutive pixel rows.
- A2D analog-to-digital
- the collection unit may comprise two or more collection sub-units, each comprising optical arrangement and detector array.
- the collection sub-units are associated with digital signal processing units (DSP) configured for determining integer-pixel correlation functions between image data pieces collected at two or more different times (e.g. consecutive frames).
- DSP digital signal processing units
- the collection sub-units are configured to provide image data of the inspection region with different defocusing levels providing variation in measurement coefficients enabling to determined weighted average correlation function with high accuracy with given processing powers.
- the weighted averaging may utilize steering vector data indicative of variations in defocusing levels of the two or more collection sub-units.
- the weighted averaging may utilize one or more optimal signal estimation techniques, e g Kalman filtering techniques, for determining optimal data on the tilt of the inspection region.
- the collection unit may be configured for providing optical correlation of collected image data.
- the collection unit comprises a spatial light modulating unit, e.g. transmitting SLM or reflecting SLM such as digital mirror device (DMD), enabling modulating light passing through/reflected from, in accordance with selected input.
- the image data collection circuitry is configured for collecting two frames of defocused image data, provide instructions to the spatial light modulating unit for modulating light in accordance with both frames, and transmit light thereon for collecting image data associated with optical Fourier transform of the frames.
- the circuitry may be configured to apply a further optical Fourier for determining correlation pick associated with correlation between the two frames.
- the present invention provides a system for use in monitoring an object, the system comprising: a collection unit comprising at least one optical arrangement and at least one detector array arranged for defocused collection of light returning from a selected region on the object; and image data collection circuitry configured for receiving electronic signals associated with data piece collected by pixels of the at least one detector array and for generating output data indicative of correlation function between image frames collected by the detector array at two or more different temporal instances.
- the system may comprise, or be associated with, an illumination unit comprising at least one light source unit configured for providing coherent optical illumination onto the selected region on the object.
- the detector array is operable as a rolling shutter detector; the optical arrangement further comprises an image multiplying unit configured for duplicating collected images onto two or more regions of the detector array; and wherein said image data collection circuitry being associated with analog to digital conversion of output data based on combined data collected in said two or more regions of the detector array, thereby providing digital data associated with image data piece collected at two or more different temporal instances.
- the image data collection circuitry may comprise analog data summation line configured for receiving analog collection data from the two or more regions of the detector array and provide summation data of said two or more regions, the analog to digital conversion unit may be operated for converting said summation data using two or more different conversion thresholds thereby providing output data indicative of correlation between said two or more regions of the detector array.
- the collection unit comprises a spatial light modulation unit configured for applying selected modulation to collected light along optical path between collection optics and the detector array
- the image data collection circuitry is configured for receiving data indicative of collected image data pieces and for varying said spatial light modulation unit in accordance with the collected image data to thereby generate correlation between image data piece associated with two or more different temporal instances.
- the detector array may be located on Fourier plane with respect to the spatial light modulation unit.
- the spatial light modulation unit may be a digital mirror device (DMD).
- the optical arrangement may further comprise a light deflection unit configured for deflecting path of collected light to thereby enabling simultaneous collection of defocused image data indicative of the inspection region and Fourier image data associated with light modulation by the spatial light modulation unit.
- a light deflection unit configured for deflecting path of collected light to thereby enabling simultaneous collection of defocused image data indicative of the inspection region and Fourier image data associated with light modulation by the spatial light modulation unit.
- the optical arrangement may be configured for directing collected light forming at least two copies of the collected light such that one copy of the collected light is directed at the detector array for collecting image data pieces to thereby enable modulation of the spatial light modulation unit accordingly, and one other copy of collected light is directed at the spatial light modulation unit for interacting with modulation pattern thereof to provide correlation between previously collected image data.
- the collection unit comprises two or more optical arrangements and corresponding two or more detector arrays arranged with different defocusing levels with respect to the selected region on the object
- said image data collection circuitry comprises two or more digital signal processors associated with said two or more detector array and configured for determining variation in collected image data pieces, and an averaging unit configured for receiving input data from said two or more digital signal processors and determining an average variation in collected image data pieces being indicative of correlation function between image data pieces associated with two or more different temporal instances.
- the averaging unit of the image data collection circuitry may be configured for determining a weighted average variation associated with correlation function between image data pieces in accordance with pre-stored steering vector corresponding to variations in levels of defocusing between said two or more optical arrangement and the corresponding detector arrays.
- the averaging unit may also be configured for using fixed weights for determined said weighted average in accordance with said steering vector.
- the averaging unit may be configured for using adaptive weights for determining said weighted average in accordance with said steering vector.
- the adaptive weights may be determined in accordance with estimated value of variation in collected image data pieces of the two or more detector arrays.
- the adaptive weights may be determined using signal power covariance estimation associated with variation in collected image data pieces of said two or more detector arrays.
- the averaging unit of the image data collection circuitry may be configured for determining an optimal estimation of the correlation function by linear or non-linear Kalman filtering technique.
- the present invention provides a method for use in monitoring properties of an object, the method comprising: collecting light returning from a region of the object using defocused optical arrangement and generating at least two defocused image regions onto a detector array; using a rolling shutter readout mode of the detector array and generating at least two temporally shifted image readout pieces associated with said at least two defocused image regions; summing said at least two temporally shifted image readout pieces providing a combined readout data vector; applying analog to digital conversion using two or more different threshold levels and determining difference vector between said two or more conversions, thereby providing data indicative of spatial correlation between said at least two defocused image regions.
- the present invention provides a system for use in monitoring an object, the system comprising: a collection unit arranged for defocused collection of light returning from a selected region on the object and comprising at least one rolling shutter detector array and an optical arrangement, the optical arrangement comprises an image multiplying unit configured for duplicating collected images onto two or more regions of the detector array; and image data collection circuitry configured comprising analog to digital conversion module, said image data collection circuitry is configured for receiving analog electronic signals associated with data piece collected by pixels of the two or more regions of the detector array and for generating output digital associated with conversion of combined data collected in said two or more regions of the detector array.
- the image data collection circuitry may comprise analog data addition line configured for receiving analog collection data from said two or more regions of the detector array and provide summation data of said two or more regions, said analog to digital conversion module being operated for converting said summation data using two or more different conversion thresholds thereby providing output data indicative of correlation between said two or more regions of the detector array.
- the present invention provides a system for use in monitoring an object, the system comprising an optical arrangement comprising spatial light modulator unit and one or more lens units, a detector array located in Fourier plane with respect to the spatial light modulator unit, light source unit and controlling circuit; the optical arrangement is configured for selectively operating in two collection paths, wherein in a first collection path the optical arrangement is configured for collecting light returning from the object in response to coherent illumination impinging thereon from the light source unit for generating corresponding defocused image on the detector array, and in a second collection path the optical arrangement is configured for directing coherent illumination from the light source unit through the spatial light modulator unit for collecting image data associated with Fourier transformation of pattern provided by the spatial light modulator unit; and wherein said controlling circuit is connected to the detector array and the spatial light modulator unit and configured for receiving image data associated with two collected defocused images and for operating said spatial light modulator unit to thereby apply coded pattern corresponding with combination of said two collected defocused images, thereby enabling detection of optical correlation between said two defocused images.
- the present invention provides a system for use in monitoring an object, comprising: a collection unit comprising two or more optical arrangements and corresponding two or more detector arrays arranged with different defocusing levels with respect to selected region on the object, and image data collection circuitry comprising two or more digital signal processors associated with said two or more detector arrays configured for determining variation in collected image data pieces and for generating output data indicative of correlation function between image data pieces collected at two or more different times (e.g. frames of different time of collection), and an averaging unit configured for receiving input data from said two or more digital signal processors and determining a weighted average variation indicative of correlation function between image data pieces.
- FIG. 1 schematically illustrates a system for monitoring an object according to some embodiments of the invention
- FIG. 2 schematically illustrates a system for monitoring an object using combined analog to digital (A2D) conversion according to some embodiments of the invention
- FIG. 3 shows an example of A2D conversion curve and approximation according to some embodiments of the present invention
- FIG. 4 shows a flow diagram exemplifying a technique for determining correlations between image data pieces according to some embodiments of the present invention
- FIG. 5 illustrates schematically a system for monitoring an object and determining correlation function between image data pieces using two or more collection sub units according to some embodiments of the present invention
- FIG. 6 exemplifies some concepts of steering vector and weighted averaging according to some embodiment of the present invention
- FIGS. 7A to 7F show rounding errors associated with integer-value correlations with respect to reference sub-pixel correlation using small, medium and large defocusing values
- FIGS. 8A to 8C show collected defocused image data associated with an illumination spot with large, medium and small defocusing levels respectively;
- FIGS. 9A to 9D show reconstructed correlation data using fixed weighting technique by weighted averaging along x and y axes ( FIGS. 9A and 9B respectively) and corresponding errors histogram with respect to the reference sub-pixel correlation data ( FIGS. 9C and 9D respectively);
- FIGS. 10A to 10D show reconstructed correlation data using fixed weighting technique using conventional beamforming technique along x and y axes ( FIGS. 10A and 10B respectively) and corresponding errors histogram with respect to the reference sub-pixel correlation data ( FIGS. 10C and 10D respectively);
- FIGS. 11A to 11D show reconstructed correlation data using adaptive weighting technique using Capon beamforming technique along x and y axes ( FIGS. 11A and 11B respectively) and corresponding errors histogram with respect to the reference sub-pixel correlation data ( FIGS. 11C and 11D respectively);
- FIGS. 12A to 12D show reconstructed correlation data using adaptive weighting technique using maximum SNR beamforming technique along x and y axes ( FIGS. 12A and 12B respectively) and corresponding errors histogram with respect to the reference sub-pixel correlation data ( FIGS. 12C and 12D respectively);
- FIGS. 13A and 13B show error histogram for eigenvector beamforming, which is a fixed weights variation of the maximal SNR technique
- FIGS. 14A to 14C showing reference, measured and estimated correlation data using linear Kalman filtering respectively
- FIGS. 15A to 15D show enlarge plots of the reference, measured and estimated correlation data using linear Kalman filtering shown in FIGS. 14A to 14C and resemblance between the reference and estimated correlation sifts respectively;
- FIGS. 16A to 16D show reference, measured and estimated correlation data using nonlinear Kalman filtering, and comparison between reference and estimated correlation shifts respectively;
- FIGS. 17A to 17C show integer correlation shifts determined based on collected image data pieces by collection sub-units having small ( FIG. 17A ), medium ( FIG. 17B ) and large ( FIG. 17C ) defocusing levels;
- FIGS. 18A to 18D show sub-pixel reference correlation data ( FIG. 18A ), integer pixel measurement data of medium defocusing ( FIG. 18B ), estimated correlation shift using non-linear Kalman filtering based on the three-measurement series ( FIG. 18C ) and comparison to reference data ( FIG. 18D );
- FIGS. 19A to 19D show enlarged section of the results of FIGS. 18A to 18D ;
- FIG. 20 illustrates an example of optical monitoring unit configured for determining correlation data using optical Fourier transform according to some embodiments of the invention
- FIGS. 21A and 21B exemplify correlation shift data determined by convention computer processing and by optical Fourier transform
- FIGS. 22A to 22D show collected image data associated with two frames, binary converted image data, optical Fourier transform of the collected image data, and inverse optical Fourier transform showing correlation between the two frames respectively.
- FIG. 1 illustrates schematically a system 100 according to some embodiments of the invention.
- the system 100 includes a collection unit 200 configured for collecting one or more sequences of defocused image data pieces formed by light returning RL from the selected inspection region R; and image data collection circuitry 500 configured for receiving the collected image data pieces and providing output correlation data 600 indicative of spatial correlations between image data pieces of different time of collection.
- the system 100 may be associated with or include an illumination unit 300 suitable for providing coherent illumination IL of one or more selected wavelength ranges, and directing the illumination toward the inspection region R.
- defocused images of light returning from the inspection region in response to coherent illumination impinging thereon typically include data on speckle patterns formed by self-interference of light components in the returning light RL. Correlations between speckle patterns collected over time provides data on vibrations (including nano-vibrations) of the inspection region, which may be associated with various characteristics and internal processes on or around the inspection region.
- Processing of the collected image data pieces for determining correlations between them may be a complex computations task. This is due to the computational process required for determining correlation between two image data pieces as well as due to the desired sampling rate that may range from 10 to 300 frames per second.
- the system 100 of the present invention and the image data collection circuitry 500 thereof are configured to provide optical and/or electronic arrangement suitable for providing output data indicative of correlations between images utilizing one or more of optical and electronic properties.
- the collection unit includes detector array 240 configured to operate as a rolling shutter detector array, i.e. utilizing raw-by-raw readout scheme, and an optical arrangement 220 including an arrangement for image duplication exemplified herein by light splitter 224 and reflector 226 .
- the image data collection circuitry 500 includes at least one signal combiner 510 and at least one analog-to-digital (A2D) conversion unit 520 . These configurations utilize shifted image collection regions and duplication of the collected light (image) to provide correlation between image data portions associated with image data pieces collected at different times (e.g. consecutive frames).
- the image duplication arrangement provides two or more duplications of collected light onto corresponding two or more regions of the detector array 240 , this is while the rolling shutter configuration of the detector array 240 results in that image data of the two or more regions of the detector array 240 are indicative of light collected at different time slots providing different frames.
- the image data collection circuitry is configured for utilizing operation of the A2D conversion unit 520 for providing digital data indicative of correlation between such different frames.
- analog to digital conversion may be considered as nonlinear operator D providing digital conversion of an analog value A received from the sensor.
- the transistors and the logic gates that perform the A2D conversion have a continuous output function and not fully digital, the A2D operation for one bit can be presented as:
- th is a selected conversion threshold value and ⁇ is a digitizing conversion parameter.
- ⁇ is a digitizing conversion parameter.
- the value of ⁇ is assigned to be about 100 to get good physical modeling of the A2D conversion.
- a ( x ) g (t 1 ) ( x )+ g (t 2 ) ( x ⁇ x ) (equation 5) where g (t1) (x) is the spatial information of the collected image data (portion of speckle pattern) at time t 1 and g (t2) (x ⁇ x) is the spatial information of the collected image data at time t 2 shifted in space by amount of ⁇ x.
- D ⁇ ⁇ D + (1+ th 2 ⁇ 2 )+2 ⁇ 2 g (t1) ( x ) g (t2) ( x ⁇ x )+ ⁇ 2 ( g (t1) ( x )) 2 + ⁇ 2 ( g (t2) ( x+ ⁇ x )) 2 ⁇ 2 th ⁇ 2 ( g (t1) ( x )+ g (t2) ( x ⁇ x )) (equation 6)
- ⁇ ( D ⁇ ⁇ D + ) dx . . .
- Equation 7 The output provided by equation 7 is composed from a constant, terms equivalent to the energy of every speckle image and a correlation expression for shift value ⁇ x.
- the image data processing circuit may be configured for summing analog output of rows associated with the two or more different regions of the detector array 240 and transmit the resulting analog data to suitable pair of A2D conversion, operated by a single A2D conversion unit 520 or two or more such conversion units.
- the resulting output may be further summed along the columns providing output data 600 indicative of correlation between image data pieces collected with certain time difference between them.
- the general scheme exemplifying the technique for monitoring an object using the system of FIG. 2 is exemplified in FIG. 4 .
- the technique includes collecting light returning from an object 4010 in response to coherent illumination of a region of the object. The light is collected to provide defocused image on a detector array to thereby generate image data pieces associated with secondary speckle patterns, rather than real space image of the inspected region.
- the technique utilizes duplicating/splitting the collected light for forming two or more defocused images onto corresponding two or more regions of the detector array 4020 , such two or more regions generally relate to different rows of the detector array.
- rolling shutter detector array 4030 provide output image data with temporal shift between image data pieces collected from the two or more regions, from the corresponding rows Manipulating the collected readout of the detector array for adding data about two shifted images 4040 , with selected spatial shifts, and applying first and second analog to digital conversions 4050 with two different conversion thresholds.
- determining difference between the analog to digital conversions 4060 and spatial summation for different spatial shifts 4070 provides output conversion data indicative of correlation element 4080 .
- the determined correlation elements are used for providing output data for further processing 4090 enabling to determine various properties of the object.
- An additional example of the present invention utilizes two or more light collection arrangements generally including associated digital processing (e.g. such as signal processing used in optical mouse for motion tracking).
- collection unit 200 includes two or more (in this example three) collection arrangements formed by lens arrangements 220 a , 220 b and 220 c and corresponding detector arrays 240 a , 240 b and 240 c .
- the image data processing circuit 500 includes one or more digital signal processors (DSP) 550 configured for determining shift between image data pieces collected at different times, similarly to typically DSP used in optical mouse for controlling location/movement of cursor in personal computing systems.
- DSP digital signal processors
- the DSP units 550 are configured to provide output indicative of position corresponding to the x-y coordinates in integer values. More specifically, general correlation between digitized image data pieces provides pixelated correlation data (i.e. in integer step values based on discrete nature of the input data). Various techniques are often used for extrapolating sub-pixel data from the so-determined correlation data. However, such techniques typically require additional processing power, time and increased energy consumption. Further, the sensitivity achieved by integer value correlation data is generally sufficient for use as input and controlling cursor's movement. This sensitivity may be insufficient of detector nano-vibrations at the inspection region and introduce rounding errors into the correlation function. The technique of the present invention overcomes this issue, using two or more collection arrangement having different defocus levels. As shown in FIG.
- collection unit 200 includes three collection sub-units exemplified by optical arrangements 220 a , 220 b and 220 c corresponding detector arrays 240 a , 240 b and 240 c .
- the collection sub-units are formed with different defocusing levels, exemplified in the figure by different distances between the optical arrangements 220 a , 220 b and 220 c and the associated detector arrays 240 a , 240 b and 240 c.
- the output of the DSP 550 relates to a linear relation between tilting level and spatial (in x-y plane) shifts determined by correlating different image data piece along discrete pixels of the images (i.e. in integer steps).
- the level of defocusing of each collection sub-unit sets a corresponding proportional ratio to this relation.
- the use of two or more differently defocused images and corresponding DSP processing provides two or more different linear translations between x-y position of the correlation function and the tilting of the object enabling sub-pixel estimation of the tilt (based on correlation) and reduce rounding errors.
- FIG. 6 illustrates exemplary output shift data of three detector arrays 240 a , 240 b and 240 c (marked as detectors a, b, and c) having different defocusing levels.
- the image data processing circuit 500 is further configured for determining a weighted average shift data with sub-pixel resolution to determine sufficiently sensitive output correlation data 600 .
- the DSP 550 output data associated with each one of detector arrays 240 a , 240 b and 240 c is normalized in accordance with defocusing level providing both variations in the sensitivity to shifts in collected image data and to the output tilting values. Accordingly, interpolation/averaging between the two or more corresponding output values provides a non-integer estimation of the actual tilt at the inspection region. The interpolation or averaging is provided for determining actual data on variation in orientation/location of the inspection region, herein referred to as tilt, with minimization of rounding errors associated with digitizing the correlation function based on pixels of the detector arrays.
- y i a ⁇ s i +v i
- v i random error resulting from the pixelated nature of the detector array and accordingly of the determined correlation between image data pieces
- Equation 8 where d j is defocusing level of collection sub-unit j.
- the rounding error v i may typically be uncorrelated between different detector arrays, and randomly distributed. Additionally, the rounding errors may be considered independent of the desired tilt signal data. It should be noted that in cases where the tilt signal s i is relatively small, i.e. corresponding with correlation shift value of 0.5 pixel of the detector array or less, the distribution of the rounding error v i is concentrated close to zero. Reference is made to FIGS. 7A to 7F showing rounding error values for tilt along x axis ( FIGS. 7A-7C ) and y axis ( FIG. 7D-7F ) for collection using different defocusing levels.
- FIGS. 7A and 7D show rounding error values for large defocusing
- FIGS. 7B and 7E shows rounding error values for medium defocusing
- FIGS. 7C and 7F shows rounding error values for small defocusing.
- the rounding error is determined with respect to reference subpixel interpolation of the correlation function.
- the inventors of the present invention have identified the following approaches as suitable for use in determining weighting vector W.
- this technique utilizes the measured data y i j for determining the weighting values of W. This technique is based on optimization of
- FIGS. 8A to 8C exemplifying defocused image data pieces collected at three defocusing levels from a common inspection region.
- the relative defocusing levels and collection units' angles were selected arbitrarily.
- the steering vector a is typically pre-known as relating to actual design of the system.
- the values of the steering vector may be determined using collected image data in accordance with ratios of diameters of the spot sizes between each collection sub-unit.
- the steering vector may include different defocusing ratios along x and y axes as the angular location of the collection sub-units may differ.
- Correlation data is determined based on sequences of images collected by the three collection sub units as exemplified in FIGS. 8A-8C . To this end, for each sequence a small window of 64 ⁇ 64 pixels was extracted around estimated center of mass of the spot of collected light. The shifts and variation of the speckle patterns were estimated using correlation between the 64 ⁇ 64 pixels windows in adjacent time frames (implemented in 2D-to-1D correlation-based procedure) for sequence of each collection sub unit. Sub-pixel correlation estimation based on the sub-unit having lowest defocusing, exemplified in FIG. 8A , is selected as reference signal. For every sensor, integer pixel correlation is determined to provide movement estimation to determine weighted average movement estimation as described above using different weighted beamforming techniques.
- FIGS. 9A-9D, 10A-10D, 11A-11D, 12A-12D and 13A-13B showing correlation data using different weighting by configurations and corresponding errors histogram with respect to the reference sub-pixel correlation data.
- FIGS. 9A-9D show reconstructed signal results provided by fixed weighting by weighted averaging
- FIGS. 10A-10D show reconstructed signal results using conventional beamforming fixed weights as described above
- FIGS. 11A-11D show reconstructed signal results using adaptive weighting for minimum noise power
- FIGS. 12A-12D show reconstructed signal results using adaptive weighting for maximal SNR.
- FIGS. 13A and 13B show error histogram relating to reconstructed signal using for eigenvector beamforming along X and Y axes respectively. As indicated above, eigenvector beamforming is a fixed weights variation of the maximal SNR technique.
- FIGS. 9A and 9B and FIGS. 10A and 10B show estimated correlation shifts along the x and y axes for fixed weighting by weighted averaging and using conventional beamforming respectively;
- FIGS. 9C and 9D and FIGS. 10C and 10D show respective error histograms with respect to sub-pixel correlation reference data.
- the correlation functions show general agreement between the two weighting techniques and the error histograms show normal distribution with standard deviation of 0.31 and 0.23 for the weighted averaging and 0.51 and 0.38 for the conventional beamforming along the x and y axes.
- the structure of the weighting vector W is almost similar, the weighted averaging provided slightly better results (in the sense of error variance) than conventional beamforming. This is due to different distribution of weights between the collection sub-units.
- FIGS. 11A and 11B and FIGS. 12A and 12B show similar estimated correlation shifts based on Capon beamforming and maximum SNR beamforming respectively;
- FIGS. 11C and 11D and FIGS. 12C and 12D as well as FIGS. 13A and 13B show respective error histograms with respect to the sub-pixel correlation reference data.
- the integer-value correlation data may be used to provide high accuracy output correlation data using one or more optimal value estimation techniques.
- integer-value correlation data associated with image data collected by the two or more collection sub-units may be combined using suitable optimal value estimation techniques, such as Kalman Filter, for determining output correlation data indicative of tilt of the inspection region.
- the image data collection circuitry 500 utilizes integer value correlation data determined by the DPS 550 for each of the two or more collection sub-units, e.g. received from detector arrays 240 A, 240 B and 240 C.
- the image data collection circuitry 500 utilizes a suitable model pre-stored in a corresponding storage utility and determines optimal estimated data on correlation function between image data pieces collected at different times using selected optimal value estimation techniques such as linear or nonlinear Kalman filtering technique.
- the pre-stored model may be based on almost constant velocity (ACV) model including one or more noise terms associated with at least one of measurement noise and rounding noise (resulting from pixelated nature of the correlations functions determined by the DSP 550 ).
- the noise terms may generally be modeled as zero mean Gaussian noise having selected variance ⁇ w 2 . More specifically, the ACV model may be as follows:
- x k is the tilt value (desired correlation shift)
- v k is the modeled velocity
- zit time increment between image data pieces
- w k is modeled measurement noise.
- the measurement channel model may be linear or non-linear, and may generally include a rounding noise term indicating the pixelated nature of the correlation data.
- a linear representation of the measurement data indicates that
- y k [ 1 0 ] [ x v ] k + n k , ( equation ⁇ ⁇ 11 )
- y k is the determined correlation shift between different image data pieces and n k is the corresponding rounding noise, which may typically have uniform probability distribution in the range [ ⁇ 0.5,0.5] as indicated above.
- the estimation technique utilizes input data in the form of measurement results y k from the DSP 550 in accordance with image data pieces collected by the two or more collection sub-units, and utilizes the input measurement data for estimating optimal correlation shift data x k .
- the processing technique of Kalman filtering is generally known and this will not be described herein in details, other than to note that the process is based on main two operational steps including prediction or time-update, in which estimated values of the desired signal x k+1 are predicted based on the selected model and data on x k ; and an update, or measurement update, in which the predicted value and corresponding factors are updated in view of the measured data y k+1 .
- the estimation technique using two or more measurement inputs is based on introduction of the two or more measurement data elements in the measurement update stage.
- the two or more measurement data elements are considered to follow similar model, while allowing variation in the noise elements and the velocity.
- FIGS. 14A to 14C and 15A to 15D showing reference, measured and estimated correlation data using linear Kalman filtering as described above, and to FIGS. 16A to 16D showing corresponding correlation data using nonlinear Kalman filtering techniques.
- FIG. 14A shows sub-pixel reference correlation shift
- FIG. 14B shows integer pixel measured correlation shift
- FIG. 14C shows estimated correlation shift using linear Kalman filtering.
- FIGS. 15A to 15C show enlarged images of the data of FIGS. 14A to 14C
- FIG. 15D shows the resemblance between the reference and the estimated correlation shift data.
- FIGS. 16A to 16D show reference correlation data ( FIG. 16A ), integer pixel measured correlation shift ( FIG. 16B ), estimated correlation shift ( FIG.
- FIG. 16C determined using non-linear Kalman filtering as described above, and comparison between the estimated and the reference correlation shift ( FIG. 16D ).
- a measure for accuracy and efficiency of the estimation may be provided by error energy, in the examples of linear Kalman filtering technique, the error energy of the integer correlation with respect to the reference is 247.63, and the error energy was reduced to 227.39 between the estimated correlation shift and the reference. In the non-linear example, the error energy reduced from 249.23 to 230.65 (error data provided by mean square error).
- the estimated correlation shift may be improved using input measurement data from additional sources, i.e. using two or more collection sub-units as indicated above.
- FIGS. 17A to 17C showing integer correlation shifts determined based on collected image data pieces by collection sub-units having small ( FIG. 17A ), medium ( FIG. 17B ) and large ( FIG. 17C ) defocusing levels; to FIGS. 18A to 18D showing sub-pixel reference correlation data ( FIG. 18A ), integer pixel measurement data of medium defocusing ( FIG. 18B ), estimated correlation shift using non-linear Kalman filtering based on the three measurement series ( FIG. 18C ) and comparison to reference data ( FIG. 18D ); and to FIGS.
- FIGS. 18A to 18D showing enlarged section of the results of FIGS. 18A to 18D .
- An error energy value of the integer correlation shift of FIGS. 17B and 18B with respect to the reference sub-pixel correlation is determined to be 247.63.
- the error energy for the estimated correlation shift of FIG. 18C with respect to the reference correlation is determined to be 158.16, showing high improvement in accuracy with respect to the estimation technique using single input source. Repeating similar technique on collected signal with relatively low signal to noise ratio showed further increase in accuracy and in proper estimation of the correlation shift.
- the technique utilizes optical Fourier transform of collected images reflected by (or transmitted through) digital micromirror device (DMD) configured to modulate light in accordance with previously collected image data.
- DMD digital micromirror device
- the DMD exemplified herein operates as a spatial light modulation unit where in the present example the modulation is achieved by variation in reflective properties.
- the technique may further direct coherent light provided by a light source unit onto the DMD for obtaining second (inverse) optical Fourier transform indicative of correlation between the image data pieces.
- the collection system 200 includes optical arrangement configured for providing several optical paths for collection of light returning from the inspection region, as well as for illuminating the DMD unit.
- the optical arrangement includes first lens arrangement 222 , light splitting unit BS 1 , reflecting element 228 , and second lens arrangement 226 configured as Fourier lens arrangement.
- the optical arrangement may also include light deflecting unit 224 (e.g. prism) and second optical splitting unit BS 2 .
- the second lens arrangement 226 is configured as Fourier lens, providing optical Fourier transform between the DMD and the detector array 240 , this is while defocusing is provided by combined optical operation of the first 222 and second 226 lens arrangements.
- the exemplary configuration illustrated in FIG. 20 utilizes DMD for modulation light by reflection.
- the present technique may utilizes transmission related light modulation technique such as spatial light modulation (SLM) unit where the light modulation is achieved by varying transmission of the light pattern rather than reflection thereof.
- SLM spatial light modulation
- DMD transmission related light modulation
- the exemplified configuration provides a joint transform correlator (JTC) using the DMD unit enabling to determine product of two images, and Fourier lens arrangement 226 for providing optical Fourier transform of the product of the two images.
- JTC joint transform correlator
- This technique enables determining spatial correlation using optical operation and thus omits the need of converting the collected image data to form a vector of numerical data piece (e.g. one-dimensional vector), which is typically required for computerized processing.
- the rate of optical processing is typically limited by electronic interface thereto, e.g. transmission of data from the detector array to the DMD and readout of the detector array 240 .
- optical spatial Fourier transform may typically be realized using one or more lenses appropriately positioned to provide Fourier transform relation between the optical field coming from the DMD and the field received at the detection plane of the detector array 240 .
- initial two frames are being captured, using full reflecting arrangement of the DMD (or portion thereof).
- optical illumination provided by the light source unit 300 is directed at the inspection region R and reflected light is collected through lens arrangement 222 , transmitted to the DMD using BS 1 and possibly BS 2 , and reflected from the DMD to be detected on detector array 240 .
- a suitable image processing circuit (associated with circuit 500 in FIG.
- the DMD 1 is configured for receiving image data pieces for two frames, and transmit corresponding data to the DMD, operating the DMD to provide reflection patterns corresponding to product of the two frames.
- the DMD presents reflective patterns corresponding with product of the two frames
- light from the light source is directed onto the DMD and provides Fourier transform image on the detector array 240 using Fourier lens arrangement 226 .
- an additional Fourier transform is performed is the same technique to provide correlation data between the two frames.
- the optical arrangement of the collection unit may include light deflecting unit 224 (e.g. prism) configured for deflecting light returning from the inspection region to a parallel optical axis and use different portions of the detector array and the DMD for collecting image data from the inspection region.
- light deflecting unit 224 e.g. prism
- other region of the detector array and DMD, or second detector and DMD are used for determining correlation between selected image data pieces collected at different times as described above.
- the technique may be used for determining correlations between consecutive frames, or at different time differences depending on frame rate of operation.
- the correlation in JTC configuration is realized in two temporal cycles.
- the DMD is used to display two input frames (two speckle patterns that are to be correlated with each other) while the camera captures the intensity of the Fourier transform and then in the second temporal cycle the output of the camera is transmitted to be displayed on the DMD.
- the correlation pattern data is typically collected on the detector array around the first order of diffraction (with respect to the optical Fourier transform). This technique is based on the relations between convolution and correlation operations as Fourier transform.
- FIGS. 21A and 21B showing correlation shift data long x and y axes (with respect to image data plane) determined using conventional processing of image data pieces (POS) and correlation data determined by optical correlation as described herein (JTC). Certain variations between the correlation graphs can be seen, however for various proposes, such differences are generally reproducible and may thus may be considered when processing the correlation data to determine one or more parameters of the inspection region. Furthermore, the process of determining optical correlation data may require less processing power and time with respect to processing of the image data pieces for determining correlation along two axes.
- FIGS. 22A to 22D The process of Joint transform correlation (JTC) is exemplified in FIGS. 22A to 22D .
- FIG. 22A shows two image pieces corresponding to two frames of speckle patterns, in this case the frames are consecutively collected;
- FIG. 22B shows binary mapping image pieces suitable for light modulation by the DMD;
- FIG. 22C shows optical Fourier mapping of product of the two image pieces;
- FIG. 22D shows inverse optical Fourier mapping indicative of convolution of the image pieces.
- Pick of correlation between the two image pieces is clearly seen in the image of FIG. 22D marked by arrow.
- the x-y location of the correlation pick is indicative of correlation shift data along x and y axes.
- optical correlation an additional important added value associated with the use of optical correlation as described herein relates to the resolution of the correlation processing.
- Digitally computed correlation is typically limited by geometrical resolution of detection of the relevant image data pieces and may require sub pixel interpolation for the correlation to obtain sufficient resolution, i.e. avoid loss of resolution due with respect to resolution of the detector array. This is while optical processing is associated with manipulation of light propagation and thus no interpolation is needed to provide correlation data at the optimal resolution (determined by the detector array).
- the present technique provides a system or unit for use in monitoring of an object.
- the system includes optical collection unit configured for collecting image data pieces from an inspection region at a selected sampling rate, and corresponding circuitry for one or more determining correlation functions between image data pieces collected at different times.
- the system may also include a coherent light source unit configured to provide coherent illumination of the inspection region.
- the optical arrangement is aligned for defocused collection of images from the inspection region for forming image data indicative of secondary speckle pattern in the reflected/scattered light.
- the system of the invention may be configured as an add-on unit for use with additional electronic device or system having processing power, storage utility and user interface, for processing data indicative of the one or more correlation functions and determine accordingly one or more selected parameters/properties of the inspection region.
- the system of the invention may be used for monitoring biological properties of a patient (e.g. heart rate, breathing rate, pulse speed, glucose or alcohol concentrations etc.) and/or mechanical characteristics of any sample, being biological or not, typically combined with selected suitable external stimulation applied on the inspection region.
- biological properties of a patient e.g. heart rate, breathing rate, pulse speed, glucose or alcohol concentrations etc.
- mechanical characteristics of any sample being biological or not, typically combined with selected suitable external stimulation applied on the inspection region.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Pathology (AREA)
- Biophysics (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
Description
Where th is a selected conversion threshold value and β is a digitizing conversion parameter. For simplicity, the following analysis considers normalized values, i.e. the range of values for A is between 0 and 1 and th=0.5. The value of β is assigned to be about 100 to get good physical modeling of the A2D conversion.
For A>th and
For A<th. Using two A2D conversions with slightly different thresholds th and subtracting the two obtained results can be approximated as:
D − −D +≈1+(A−th)2β2 (equation 4)
A(x)=g (t
where g(t1)(x) is the spatial information of the collected image data (portion of speckle pattern) at time t1 and g(t2)(x−Δx) is the spatial information of the collected image data at time t2 shifted in space by amount of Δx. Thus, the overall digital output we obtain equals to:
D − −D +=(1+th 2β2)+2β2 g (t1)(x)g (t2)(x−Δx)+β2(g (t1)(x))2+β2(g (t2)(x+Δx))2−2thβ 2(g (t1)(x)+g (t2)(x−Δx)) (equation 6)
Summing the pixels related to the spatial information (performing summation over the columns of the rolling shutter detector array) provides:
∫(D − −D +)dx= . . . 2β2 ∫g (t1)(x)g (t2)(x−Δx)dx+ . . . (equation 7)
The output provided by
a=[1,d 2 /d 1 ,d 3 /d 1 . . . ,d N /d 1,] (equation 8)
where dj is defocusing level of collection sub-unit j. The rounding error vi may typically be uncorrelated between different detector arrays, and randomly distributed. Additionally, the rounding errors may be considered independent of the desired tilt signal data. It should be noted that in cases where the tilt signal si is relatively small, i.e. corresponding with correlation shift value of 0.5 pixel of the detector array or less, the distribution of the rounding error vi is concentrated close to zero. Reference is made to
S i =W T y i j (equation 9)
where Si is the estimate tilt signal with minimized rounding errors. The inventors of the present invention have identified the following approaches as suitable for use in determining weighting vector W.
given that wTa=1, and provides
where R=E{yyT}. In this context, E{x} refers to expected value associated with probability distribution of the parameter x. The covariance matrix R may be replaced by an approximated sampled covariance matrix formed by the vectors yi j such that Y=[y1, y2, . . . yN] providing
where Rv=E{vvT}. The noise v, corresponding with rounding errors, generally follows known statistical behavior. More specifically, it is assumed that the rounding errors are random errors having uniform distribution in the range [0,1]. This provides the suitable estimation of Rv=σq 2I, where
It should be toned that such estimation provides this weighting technique as fixed weights. Alternative estimation for W utilizes adaptive scheme using signal power covariance estimation (eigenvector beamforming) providing R=PsaaT+σ2I, where Ps is the average signal power (energy) and σ2 is estimated by eigenvector decomposition of R as mean value of the N−1 smaller eignevalues, where the larger eigenvalue is given by Ps.
where xk is the tilt value (desired correlation shift), vk is the modeled velocity, zit is time increment between image data pieces and wk is modeled measurement noise. The measurement channel model may be linear or non-linear, and may generally include a rounding noise term indicating the pixelated nature of the correlation data. A linear representation of the measurement data indicates that
Where yk is the determined correlation shift between different image data pieces and nk is the corresponding rounding noise, which may typically have uniform probability distribution in the range [−0.5,0.5] as indicated above. Additionally, the measurement model may have non-linear quantization form including a quantization mapping M, providing:
=M(y k) (equation 12)
Claims (18)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US17/050,213 US11202040B2 (en) | 2018-04-23 | 2019-04-08 | System and method for monitoring a sample |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201862661262P | 2018-04-23 | 2018-04-23 | |
| PCT/IL2019/050395 WO2019207568A1 (en) | 2018-04-23 | 2019-04-08 | System and method for monitoring a sample |
| US17/050,213 US11202040B2 (en) | 2018-04-23 | 2019-04-08 | System and method for monitoring a sample |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20210243411A1 US20210243411A1 (en) | 2021-08-05 |
| US11202040B2 true US11202040B2 (en) | 2021-12-14 |
Family
ID=68294449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US17/050,213 Expired - Fee Related US11202040B2 (en) | 2018-04-23 | 2019-04-08 | System and method for monitoring a sample |
Country Status (2)
| Country | Link |
|---|---|
| US (1) | US11202040B2 (en) |
| WO (1) | WO2019207568A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11647497B2 (en) * | 2018-03-28 | 2023-05-09 | Telefonaktiebolaget Lm Ericsson (Publ) | Uplink scheduling based on upper and lower SNR targets |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130144137A1 (en) | 2011-01-28 | 2013-06-06 | Universitat De Valencia | Method and system for non-invasively monitoring biological or biochemical parameters of individual |
| US8638991B2 (en) | 2007-07-26 | 2014-01-28 | Bar Ilan University | Motion detection system and method |
| US20140148658A1 (en) | 2011-01-28 | 2014-05-29 | Universitat De Valencia | Method and system for non-invasively monitoring biological or biochemical parameters of individual |
| US20150323311A1 (en) | 2012-06-13 | 2015-11-12 | Koninklijke Philips N.V. | Determining a propagation velocity for a surface wave |
| WO2016103271A2 (en) | 2014-12-27 | 2016-06-30 | Guardian Optical Technologies Ltd. | System and method for detecting surface vibrations |
| WO2017191639A1 (en) | 2016-05-05 | 2017-11-09 | ContinUse Biometrics Ltd. | System and method for use in tissue monitoring and analysis |
| WO2018150427A1 (en) | 2017-02-15 | 2018-08-23 | ContinUse Biometrics Ltd. | System and method for use in remote sensing |
| US20180376086A1 (en) * | 2017-06-27 | 2018-12-27 | Canon Kabushiki Kaisha | Image capturing apparatus, control method, and storage medium |
| US20190155012A1 (en) * | 2016-05-02 | 2019-05-23 | Carl Zeiss Microscopy Gmbh | Artefact reduction for angularly-selective illumination |
-
2019
- 2019-04-08 WO PCT/IL2019/050395 patent/WO2019207568A1/en not_active Ceased
- 2019-04-08 US US17/050,213 patent/US11202040B2/en not_active Expired - Fee Related
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8638991B2 (en) | 2007-07-26 | 2014-01-28 | Bar Ilan University | Motion detection system and method |
| US20130144137A1 (en) | 2011-01-28 | 2013-06-06 | Universitat De Valencia | Method and system for non-invasively monitoring biological or biochemical parameters of individual |
| US20140148658A1 (en) | 2011-01-28 | 2014-05-29 | Universitat De Valencia | Method and system for non-invasively monitoring biological or biochemical parameters of individual |
| US20150323311A1 (en) | 2012-06-13 | 2015-11-12 | Koninklijke Philips N.V. | Determining a propagation velocity for a surface wave |
| WO2016103271A2 (en) | 2014-12-27 | 2016-06-30 | Guardian Optical Technologies Ltd. | System and method for detecting surface vibrations |
| US20190155012A1 (en) * | 2016-05-02 | 2019-05-23 | Carl Zeiss Microscopy Gmbh | Artefact reduction for angularly-selective illumination |
| WO2017191639A1 (en) | 2016-05-05 | 2017-11-09 | ContinUse Biometrics Ltd. | System and method for use in tissue monitoring and analysis |
| US20190159701A1 (en) * | 2016-05-05 | 2019-05-30 | ContinUse Biometrics Ltd. | System and method for use in tissue monitoring and analysis |
| WO2018150427A1 (en) | 2017-02-15 | 2018-08-23 | ContinUse Biometrics Ltd. | System and method for use in remote sensing |
| US20180376086A1 (en) * | 2017-06-27 | 2018-12-27 | Canon Kabushiki Kaisha | Image capturing apparatus, control method, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019207568A1 (en) | 2019-10-31 |
| US20210243411A1 (en) | 2021-08-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110646810B (en) | Speckle-optimized compressed sensing ghost imaging method and system | |
| EP0590846B1 (en) | Image processor for target detection and tracking | |
| Malik et al. | A novel algorithm for estimation of depth map using image focus for 3D shape recovery in the presence of noise | |
| KR101264955B1 (en) | Object Restoration Method and System | |
| US9989630B2 (en) | Structured-light based multipath cancellation in ToF imaging | |
| US11816259B2 (en) | Methods and systems for forming images of eye features using a non-imaging, scanning-MEMS-based eye-tracking system | |
| JP2008310828A (en) | End-to-end design of electro-optic imaging system | |
| US11867625B2 (en) | System and method for imaging via scattering medium | |
| US7113268B2 (en) | Scintillation tolerant optical field sensing system and associated method | |
| EP2585853A2 (en) | Calibration of a probe in ptychography | |
| Young et al. | Signal processing and performance analysis for imaging systems | |
| Murino et al. | A probabilistic approach to the coupled reconstruction and restoration of underwater acoustic images | |
| EP2227705B1 (en) | Method and apparatus for providing image data | |
| Fienup et al. | Comparison of phase diversity and curvature wavefront sensing | |
| US11202040B2 (en) | System and method for monitoring a sample | |
| CN115115653A (en) | Refined temperature calibration method for cold and hot impact test box | |
| JPH1063821A (en) | Super-Resolution Full Circular Aperture Scene Synthesis Using Rotating Strip Aperture Image Measurement | |
| CN111537999A (en) | A Robust and Efficient Decomposed Projection Autofocus Method | |
| US10591871B1 (en) | Method of single shot imaging for correcting phase errors | |
| Hillier et al. | Topographic cellular active contour techniques: theory, implementations and comparisons | |
| US12241731B2 (en) | Displacement measurement device, non-contact input apparatus, and biological micromotion measurement apparatus | |
| KR20230097797A (en) | Ghost Image Acquisition Method and Electronic Device Using The Same | |
| US10861171B2 (en) | System and method for optical monitoring using symmetry condition of light field | |
| US12554009B2 (en) | Image analysis device and image analysis method | |
| CN115166712B (en) | Spatial imaging method of micromotion characteristic mode based on first-order field correlation and target micromotion model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
| AS | Assignment |
Owner name: CONTINUSE BIOMETRICS LTD., ISRAEL Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZALEVSKY, ZEEV;CALIFA, RAN;GOLBERG, MARK;AND OTHERS;SIGNING DATES FROM 20190502 TO 20190527;REEL/FRAME:054160/0402 |
|
| FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: AWAITING TC RESP, ISSUE FEE PAYMENT VERIFIED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
| AS | Assignment |
Owner name: BANK LEUMI LE-ISRAEL B.M., ISRAEL Free format text: SECURITY INTEREST;ASSIGNOR:CONTINUSE BIOMETRICS LTD.;REEL/FRAME:058550/0784 Effective date: 20211129 |
|
| FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
| LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
| STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
| FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20251214 |