JP2017514108A - Multispectral ultrasound imaging - Google Patents

Abstract

Systems and methods for multispectral ultrasound imaging are disclosed. In one embodiment, the finger is scanned at multiple ultrasound scan frequencies. Each scan frequency provides a set of image information describing a plurality of pixels of the finger, including a signal intensity that indicates the amount of energy reflected from the surface of the platen on which the finger is placed. For each of the pixels, a pixel output value corresponding to each of the scan frequencies is combined to produce a combined pixel output value for each pixel. Systems and methods for improving multispectral ultrasound imaging data acquisition are also disclosed.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a US Provisional Application No. 61 / 948,778 entitled “Multi-Spectral Ultrasonic Imaging” filed March 6, 2014, the entire contents of which are incorporated herein by reference. And claims the priority and interest of US Non-Provisional Application No. 14 / 639,116, filed March 4, 2015, entitled “Multi-Spectral Ultrasonic Imaging”.

  The present disclosure relates to devices and methods using multispectral ultrasound imaging.

  Ultrasonic scanners can be constructed from various types of materials. Usually, the ultrasonic energy used in such scanners is required to pass through most of these materials. The properties of the various materials through which the ultrasound passes or impinges can have different properties with respect to dispersion, diffraction, absorption, and reflection, so that the material disperses, diffracts, absorbs, and reflects the ultrasonic energy in various ways. Thus, these differences can depend on the wavelength of the ultrasonic energy. Imaging a particular object using a single ultrasound frequency can limit information and details about the object being imaged.

  During manufacturing of an ultrasonic sensor array, tolerances can build up in the ultrasonic sensor stack, which affects the signal path and optimizes the signal and system response for which the collected data is available. You can create a situation that doesn't use. Furthermore, the data quality may be frequency dependent and the structural configuration of the object may be frequency dependent.

U.S. Patent No. 7,287,013

  Due to the normal variability resulting from the manufacture of an ultrasound scanning system, even if one ultrasound scanning system and another scanning system are manufactured within the desired tolerances and according to the same procedure, It can behave differently from a scanning system. The result of these differences may mean that one scanner collects information at the optimal frequency, while another scanner does not.

  The basic method that has been applied in the prior art is to perform a scan at a single specific frequency that maximizes the signal output as captured by a thin film transistor (TFT) array located in the sensor stack. there were. This single frequency is largely determined by the thickness and material properties of the sensor stack and can be used to distinguish between the raised and valley areas of the fingerprint of the finger being imaged. In a manufacturing situation (without reference to fingerprints), the frequency is determined by choosing the frequency at which the sensor array output is maximized between the two cases, when the ultrasonic transmitter excitation voltage is on and when the transmitter is off. Decisions can be made. This method may produce image information that may not meet the expected results for the sharpness of the fingerprint image in more real-world situations. It may be necessary to tune the operating frequency throughout normal use, which can lead to inconsistent results.

  One aspect of the present invention can be described as a method of scanning a finger. The method may include scanning a finger placed on the imaging surface of the ultrasonic sensor with a plurality of ultrasonic scan frequencies. The penetration depth of the ultrasound signal into the tissue region can be different at different frequencies and can result in variations in the signal level reflected when captured by the TFT array. A plurality of scan frequencies can be selected by scanning at a plurality of test frequencies with no finger on the imaging surface and specifying a peak test frequency. The peak test frequency may be a test frequency at which the test frequency immediately below and the test frequency immediately above returns less energy than the peak test frequency.

  The method may include generating an ultrasound image information set from a plurality of pixels of the ultrasound sensor for each of the scan frequencies. The image information set may include pixel output values from each of the plurality of pixels, each pixel output value indicating the amount of energy reflected from the imaging surface. Each scan frequency may provide a set of image information describing a plurality of pixel output signal levels associated with the fingerprint. Each pixel output value may indicate a signal intensity indicative of the amount of ultrasonic energy reflected from the surface of the platen on which the finger is placed. As used herein, the term “image” refers to a form of image information set.

  The method may further comprise the step of combining the image information sets corresponding to each of the scan frequencies to generate a combined image information set. The combined image information set may include a combined pixel output value from each of the plurality of pixels. The steps of combining the image information set include adding pixel output values to generate a sum, dividing the sum by the number of scan frequencies to generate an average value for each of the pixels, and combining the average values. Used as a value. As used herein, the term “synthesized” means mathematically synthesized.

  In some embodiments, the method may further include using a plurality of ultrasound image information sets to make a biological determination and providing a biological output signal indicative of the biological determination.

  In some embodiments, the method further provides converting each pixel output value to a grayscale value and providing the grayscale values for the plurality of pixels as a combined image information set representing a finger fingerprint. Steps.

In some embodiments, the step of synthesizing the image information set includes identifying a weighting factor for each scan frequency and multiplying each pixel output value by the corresponding weighting factor to obtain a product of the pixel output values. A step of generating, adding a product of pixel output values to generate a sum, dividing the sum by the number of scan frequencies to generate an average value for each of the pixel output values, and combining the average values. Using as a pixel output value. The weighting factor can be calculated using the following formula:
w (f _i ) = (e ^{(avgi * fi)-e (avgi * fmax)} ) / (e ^{(avgi * fmin)-e (avgi * fmax)} )
Where w (f _i ) is a weighting factor for the i-th scan frequency, avg _i is the average pixel output value at the i-th scan frequency and the next lower scan frequency, and f _min is The lowest scan frequency, and f _max is the highest scan frequency.

  In another embodiment, synthesizing the image information set may include creating a covariance matrix for each of the scan frequencies. A covariance matrix may be created from the pixel output values in the image information set. The covariance matrix can be combined to provide a combined matrix with a combined value for each pixel output value. In one embodiment, synthesizing the covariance matrix includes interpolating between components in the covariance matrix.

In one embodiment, the method includes, for each scan frequency, identifying a weighting factor and multiplying each component in the covariance matrix by a corresponding weighting factor before mathematically synthesizing the covariance matrix. Can be included. The weighting factor can be calculated using the following formula:
w (f _i ) = (e ^{(avgi * fi)-e (avgi * fmax)} ) / (e ^{(avgi * fmin)-e (avgi * fmax)} )
Where w (f _i ) is a weighting factor for the i-th scan frequency, avg _i is the average pixel output value at the i-th scan frequency and the next lower scan frequency, and f _min is The lowest scan frequency, and f _max is the highest scan frequency.

  The method may further include correlating each synthesized value for each of the pixels with a grayscale value. The method may further include providing a grayscale value as a finger or fingerprint representation.

  The method may include scanning at a plurality of ultrasonic test frequencies with no finger on the imaging surface of the ultrasonic sensor. The method may further include selecting one or more peak test frequencies. Each selected peak test frequency may have a reflected signal that is higher than most of the other peak test frequencies. The method may further include using the selected peak test frequency as a plurality of scan frequencies. Additional scan frequencies can be identified by adding or subtracting a predetermined offset to a selected one of the peak test frequencies. In another embodiment, the additional scan frequency may be selected by identifying a range that includes the selected one of the peak test frequencies and selecting the scan frequency to be within the identified range. In one embodiment, the additional scan frequency may be selected by identifying harmonics of the selected peak test frequency. In another embodiment, the method may further include evaluating the image quality of the peak test frequency and selecting a peak test frequency that has a better image quality than other peak test frequencies.

  One aspect of the present invention may be described as a system for generating an automatically co-registered image information set of objects. The system can also be described as a system for scanning a finger. The system can include an imaging surface configured to accept a finger. The imaging surface can be substantially flat. The system can also include a planar ultrasound transmitter. The planar ultrasound transmitter can generate one or more planar ultrasound waves in response to the signal generator. The signal generator may be capable of generating electrical signals with different discrete frequencies within the ultrasonic frequency range.

  The system may further include a transmitter driver amplifier. The amplifier may be configured to receive an electrical signal from the signal generator and use the electrical signal to drive an ultrasonic transmitter. The ultrasound may be directed by the transmitter to the imaging surface, one or more ultrasound signals may be reflected from the imaging surface to the ultrasound sensor array, and the object is in contact with the ultrasound sensor array. The ultrasonic sensor array may be configured to detect one or more reflected ultrasonic waves. In some implementations, the system may further include a set of bandpass filters for separating one or more detected ultrasound waves into frequency components.

  The system may further include an electronic subsystem for forming or generating an image information set of the object for each received signal at each frequency of interest. The electronic subsystem may include a processor or logic circuit. The electronic subsystem can also be configured to synthesize the image information set. The image information set can be heuristically synthesized using a Neiman-Pearson multimodal fusion system or stochastically synthesized to produce an output representation of the object, such as an image.

  One aspect of the invention may also be described as a non-transitory computer-readable medium that stores computer-executable code. The executable code may include instructions for scanning a finger placed on the imaging surface of the ultrasonic sensor with a plurality of ultrasonic scan frequencies. The executable code may further include instructions for generating an ultrasound image information set from a plurality of pixels of the ultrasound sensor for each of the scan frequencies. The image information set may include pixel output values from each of the plurality of pixels. Each pixel output value may indicate the amount of energy reflected from the imaging surface. The executable code may further include instructions for combining the image information sets corresponding to each of the scan frequencies to generate a combined image information set. The combined image information set may include a combined pixel output value from each of the plurality of pixels. The executable code may further include instructions for converting each pixel output value to a grayscale value and providing the grayscale value for the plurality of pixels as a composite image information set representing the fingerprint of the finger. The executable code may further include instructions for making a biological determination using a plurality of sets of ultrasound image information and providing a biological output signal indicative of the biological determination.

  One aspect of the invention can also be described as a system for scanning a finger. The system responds to a signal generator capable of generating electrical signals of different discrete frequencies within the ultrasonic frequency range and provides a means for generating one or more planar ultrasound (`` MFG '') May be included. The system may further include means for driving the MFG in response to an electrical signal from the signal generator. The system may further include means for contacting the finger and reflecting the ultrasound from the MFG as an ultrasound signal to the ultrasound sensor array means, the ultrasound sensor array means receiving the reflected ultrasound signal. Configured to receive. The system may further include means for forming a finger image information set for each received reflected ultrasound signal at each frequency of interest and synthesizing the formed image information set. In one embodiment, the system may further include means for separating one or more received ultrasound signals into frequency components. The means for synthesizing the formed image information set may be configured to heuristically generate the output image or to generate the output image stochastically using Neiman-Pearson multimodal fusion. .

  For a deeper understanding of the nature and purpose of the present disclosure, reference should be made to the accompanying drawings and the following description. The present disclosure will now be described by way of non-limiting example with reference to the accompanying drawings and figures.

FIG. 6 is a plot with information corresponding to six different ultrasonic waves each passing through an ultrasonic sensor array, each at a different frequency. FIG. 2 shows a method for generating an ultrasound image information set, based on sequentially transmitting, processing and repeating this method for several transmitter excitation signals of different frequencies. It is a figure which shows the method for producing the ultrasonic image information set based on a chirp type transmitter excitation signal. 6 is a flowchart illustrating a method of multispectral ultrasound imaging using an arithmetic average for each pixel with optional weighting. It is a block diagram which shows the ultrasonic sensor system for producing | generating the ultrasonic image information set corresponding to a target object. 1 is a diagram illustrating a first configuration of a system for generating an ultrasonic image information set of an object that contacts an outer surface of an ultrasonic sensor array. FIG. FIG. 3 is a diagram showing a second configuration of a system for generating an ultrasonic image information set of an object in contact with an outer surface of a platen arranged on an ultrasonic sensor array. FIG. 4 shows transmitter and receiver signals for a set of two or more excitation frequencies applied sequentially to a transmitter, with image information sets acquired for each applied frequency. FIG. 4 shows transmitter and receiver signals for a set of two or more excitation frequencies applied sequentially to a transmitter, with image information sets acquired for each applied frequency. FIG. 4 shows a transmitter signal and a receiver signal for a set of two or more excitation frequencies applied sequentially to a transmitter, together with a set of image information acquired for the set of excitation frequencies. FIG. 2 shows transmitter and receiver signals for a chirp transmitter excitation sequence (up chirp or down chirp) starting at a first frequency and ending at a second frequency. It is a figure which shows the arithmetic average for every point of two or more sets of ultrasonic image information. FIG. 6 is a flowchart illustrating a method for creating a composite image information set using two or more covariance matrices. FIG. FIG. 6 illustrates a method for creating a combined image information set using covariance-based interpolation. 6 is a flowchart illustrating a method for providing a representation of a fingerprint. It is a graph corresponding to selection of various excitation frequencies. Fig. 6 is a graph corresponding to selection of additional excitation frequencies. It is a graph which shows the method of calibration. 6 is a graph showing an additional method of calibration. 2 is a flowchart illustrating a method of multispectral ultrasound imaging with chirped excitation. FIG. 6 is a flowchart illustrating a method of multispectral ultrasound imaging for chirped excitation of multiple information sets with arithmetic average per pixel and optional weighting. 2 is two graphs showing the determination of a chirp sequence. It is a graph which shows various chirp sequences. It is a graph which shows various chirp sequences. It is a graph which shows FFT of the transmitter signal by which chirp coding was carried out. It is a graph which shows FFT of the transmitter signal by which chirp coding was carried out. It is a graph which shows FFT of the transmitter signal by which chirp coding was carried out. It is a graph which shows FFT of the transmitter signal by which chirp coding was carried out. It is a figure of an ultrasonic sensor array. It is a figure of the some structure of an ultrasonic fingerprint sensor. It is a block diagram of an ultrasonic sensor system. FIG. 6 shows the frequency response of the difference in the output of the sensor array between the transmitter on condition and the transmitter off condition. 6 is a histogram of sensor output signal amplitude obtained for a finger in two cases. FIG. 6 is a diagram showing a contour of a sample image of a finger and a corresponding histogram plot. FIG. 6 shows a variability plot showing a comparison of analog voltages between a selected ridge region and a valley region. It is a figure which shows the negative peak at the time of plotting the difference which pulled the voltage when there exists a target object from the voltage in the case of air with respect to the frequency of an ultrasonic transmitter excitation signal. It is a figure which shows the method based on multiple frequencies with respect to determining the biological property of a target object. It is a figure which shows the method based on multiple frequencies with respect to determining the biological property of an object. It is a figure which shows the method based on multiple frequencies with respect to determining the biological property of a target object. It is a figure which shows the method based on multiple frequencies with respect to determining the biological property of a target object. FIG. 6 is a plot of signals reflected from various portions of an object in one configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in one configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in one configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 39B is a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 39A. FIG. 39B shows a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 39B. FIG. 39B is a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 39C. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 40B is a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 40A. FIG. 40B shows a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 40B. FIG. 40D shows a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 40C. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 6 is a plot of signals reflected from various portions of an object in another configuration of a system for multispectral ultrasound imaging. FIG. 41B is a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 41A. FIG. 41B is a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 41B. FIG. 41D shows a plot of integrated receiver output according to various distance gates implemented in the system of FIG. 41C. It is a flowchart of the biometric detection method. 3 is a flowchart of a method for scanning a finger.

  One aspect of the invention generally relates to an ultrasonic sensor system for providing information about an object. In some implementations, this information may be obtained from multiple excitation signals applied to the ultrasound transmitter, each at a different frequency. By using multiple ultrasonic frequencies, more information about the object can be provided than can be provided by utilizing a single excitation frequency.

  An ultrasonic fingerprint sensor can function by generating and transmitting ultrasonic waves toward a platen type imaging surface. There can be an object on the platen for which information is desired. When the object is a finger, the desired information may relate to the fingerprint. A portion of the ultrasonic energy that reaches the platen is reflected and this reflected energy can be detected. The intensity of the reflected energy and the location where that energy is received can be obtained. The acquired signal can be recorded in the form of a data set. The data set may be used to create a data stream that may be used to generate a visual image of the object, which visual image may be provided via a monitor or printer. In some implementations, the acquired signals can form a data set, also referred to as an ultrasound image information set, which is further processed to generate a composite image information set. obtain. The combined image information set may be utilized in registration, verification, and authentication of a user of a mobile device such as a mobile phone, tablet computer, or portable medical device that incorporates an ultrasonic fingerprint sensor, for example.

  One aspect of the present invention may be embodied in a system and / or method for multispectral ultrasound imaging to better match the system-specific maximum limits. For example, ultrasonic sensors manufactured according to the same design and manufactured from the same manufacturing facility can have differences that can affect the performance of each sensor.

  For example, during the manufacture of an ultrasonic sensor, there are several material boundaries and bulk material of the material that the ultrasound traverses. Due to normal variations in the manufacturing process, each sensor may have a slightly different effect on the resonant frequency and the ultrasonic signal passing through the sensor. These resonance differences can show as much as 50% change for fairly small frequency changes. As a result, the same system that obtains good output at a 20 MHz transmitter excitation frequency may show only half of the output at 19 or 21 MHz.

  Individual sensor differences can be addressed by using more than one scan frequency and then combining the set of image information from each scan frequency. In addition, applying sound waves at multiple frequencies may allow data collection for objects that provide a better representation of the object than applying single wavelength sound waves. Applying sound waves at multiple frequencies in an ultrasound system, either as a single sequential signal with a spectrum of excitation frequencies or a composite signal, can improve imaging with less manufacturing tolerances in the system. As it becomes possible, more cost-effective manufacturing techniques can be utilized.

  In some implementations, the multispectral ultrasound sensor system generates multiple data sets or image information sets corresponding to the object, each image information set being generated with information obtained at different ultrasound frequencies. . The term multispectrum generally refers to a system that uses two, three, four, or more frequencies or wavelengths in constructing an image information set of an object. A multispectral system may also be referred to as a hyperspectral system. When the ultrasonic receiver is capable of detecting ultrasonic energy at many different frequencies and is at a constant distance from the object, the step of generating an image information set can be performed. For example, the surface of the platen on which the user can place a finger may be at an invariable distance from the pixel circuit of the underlying ultrasonic sensor array. The desired ultrasound can be generated by driving the ultrasound transmitter with a transmitter excitation signal to generate discrete frequency ultrasound. In some implementations, the transmitter may be driven to generate an ultrasound waveform that is simultaneously added as a composite energy ultrasound waveform that is the sum of the desired frequencies and with multiple desired frequencies.

  Multispectral scanning at discrete frequencies may be implemented as “chirp”. A chirp is a signal whose frequency increases ("up chirp") or decreases ("down chirp") over time and can be continuous. In ultrasound, the excitation signal may be shaped to represent a chirp so that the generated wave interacts with the dispersion properties of the material, increasing the overall dispersion as the ultrasound signal propagates or Decrease. By using the chirped excitation signal, it is possible to collect data with more information content.

  The use of the chirp excitation signal allows the sensor system to apply sound waves and collect information about the object over a wide range of frequencies. An ultrasonic sensor system is required to obtain a pixel output signal from sensor pixels in an ultrasonic sensor array, digitize the pixel output signal, and pass the digitized pixel output signal (or value) through a series of filters. Data can be extracted. Alternatively, discrete frequency pulses may be used to apply sound waves to the object and collect data from the reflected signal, and then the transmitter excitation frequency may be changed, and the object at multiple different frequencies. This process is repeated to obtain data about the object. This process can be completed very quickly in situations where the excitation signal is transmitted in very small time increments and the reflected signal is received and processed. Depending on the distance from the transmitter to the platen surface and to the opposite ultrasonic sensor array, this process can be completed in a few microseconds (or milliseconds if longer distances are involved).

  An additional advantage of using a chirp-based method when operating an ultrasound system is that it allows more flexibility in manufacturing an ultrasound transmitter-receiver system. Using a chirp-based system can accommodate manufacturing differences and allow improved response from each system.

FIG. 1 shows a plot with information corresponding to six different ultrasound waves passing through the ultrasonic sensor array, each at a different frequency. The vertical axis is transmissive and the horizontal axis is the thickness of the polystyrene layer, which is a typical material used as a platen for an ultrasonic sensor. The thickness (xi) of the polystyrene layer corresponding to the various curves in FIG. 1 ranges from approximately 0 to approximately 1/5000 inch. Transparency (tr) is shown for 6 ultrasonic frequencies: 5 MHz, 10 MHz, 15 MHz, 20 MHz, 25 MHz, and 30 MHz. FIG. 1 confirms that at a thickness of about 1.7 mils, 30 MHz gives nearly 100% transparency, and both 15 MHz and 5 MHz each give about 65% transparency. Other frequencies at 1.7 mil thickness give less than 50% permeability. Note that the peak transmission for the other ultrasonic frequencies shown in FIG.
25MHz-1.9mil thickness 15MHz-3.5mil thickness
20MHz-2.4mil thickness 10MHz-4.7mil thickness

  It can also be seen that the slope of the permeability function at 30 MHz is very large near the peaks at about 1.7 mils and 3.5 mils. This indicates that the permeability at 30 MHz decreases sharply near the peak permeability due to small changes in the thickness of the polystyrene layer. The same is true for the peak transparency for each of the 25 MHz, 20 MHz, and 15 MHz excitation frequencies. That is, the permeability rapidly decreases due to a small change in the thickness of the polystyrene layer. Figure 1 shows that 5MHz and 10MHz have smaller slopes, but these frequencies are useful because they have high transparency, but it is a limited part of the polystyrene thickness range shown in Figure 1 Only true.

  An additional advantage of multiple discrete frequency sensor systems or chirp based systems is that they can distinguish between objects on the platen being observed and those on the platen not being observed. For example, assume that an ultrasonic multispectral system is built into the display of a mobile device. It is also assumed that the system uses a 22 MHz sound wave application signal. Raindrops on the display interfere with information corresponding to the object. However, if the frequency is lowered to, for example, 15 MHz, the rainwater droplets do not have proper resonance and therefore do not appear in the ultrasound observation. Multispectral sensors (ie, chirp sensors) avoid detecting raindrops by providing multiple frequency dependent image information sets. The best one of the information sets may be selected for further use, or a composite of information sets may be created and used.

  FIG. 2 shows a method for generating an ultrasound image information set based on sequentially transmitting, processing and repeating this method for several transmitter excitation signals of different frequencies. In this case, a system using a plurality of discrete frequencies is employed. A first frequency is generated (21), amplified (22), and can be used to drive an ultrasound transmitter that generates planar ultrasound (23). (23) The ultrasonic signal generated and emitted from the transmitter passes through the binding medium (if any) and the material of the ultrasonic sensor stack and interacts with the surface of the platen in contact with the object ( twenty four). A portion of the ultrasonic energy may be reflected, for example, at a location where the platen interacts with air (e.g., a fingerprint valley) (25), and then the reflected energy is transmitted to the ultrasonic sensor array where the energy is Detected (26). An ultrasound image information set or data set corresponding to the detected energy may be generated and stored for later use (27). Later use may include creating a data stream that causes an image of the object to be displayed via a monitor or for fingerprint registration, verification, and authentication. This process can be repeated for the second frequency and a second set of image information corresponding to the detected energy can be generated and stored for later use. This process can be repeated N times to create N image information sets (29). The plurality of image information sets may be combined to generate a multispectral combined image information set (28).

  Another type of ultrasonic multispectral imaging system is illustrated in FIG. FIG. 3 shows a method for creating an ultrasound image information set based on a chirp-type transmitter excitation signal. A chirp-type signal is generated (either up-chirp (rising frequency) or down-chirp (falling frequency)) over a range of frequencies appropriate to the required resolution of the system (31). The amplifier 32 can amplify the excitation signal and drive the planar ultrasound generator. Ultrasound signals, sometimes referred to as pulses or tone bursts (TB), exit the transmitter (33), pass through the coupling medium (if any) and other materials in the sensor stack and contact the object Interacts with the surface of the platen (34). A portion of the ultrasonic energy may be reflected, and the reflected energy is then transmitted to the ultrasonic sensor array (35) where it is detected (36) and converted to an electrical signal. The converted electrical signal from the pixels in the ultrasonic sensor array can be sent to a bandpass filter 37 that separates the components of the signal according to frequency components. Component separation can be achieved using fast Fourier processing instead of discrete bandpass filters. The output obtained from the filtering process can be used to construct a plurality of ultrasound image information sets, each representing an object at a different frequency (38). Multiple information sets may be combined (39) to generate a multispectral image information set.

  FIG. 4 is a flowchart illustrating a method of multispectral ultrasound imaging using an arithmetic average for each pixel with optional weighting. Fingerprints can be scanned at multiple ultrasound frequencies (“scan frequencies”). Each scan frequency may provide a set of image information that describes the object being imaged, such as a fingerprint. The information set may consist of scan value data, and each data in the information set may indicate the signal strength of the pixel. The signal strength of each pixel indicates the amount of energy reflected back to that pixel from the surface of the platen on which the finger is placed. Scanning at multiple frequencies generates multiple data for each of the pixels. Scan value data corresponding to each of the scan frequencies may be mathematically synthesized to generate a synthesized value for each pixel. The combined value for each pixel can be correlated with the grayscale value to provide a grayscale value for each pixel. These gray scale values can be provided collectively to represent the fingerprint.

  Mathematically synthesizing the image information set for a pixel includes adding pixel output values for the pixel to produce a sum, and dividing the sum by the number of scan frequencies to average each of the pixels Generating a value. This average value can be used to calculate (87) a combined image information set from the acquired image information sets 81, 83, 85 using an average per pixel with optional weighting. .

  The process of combining individual co-registered information sets may be performed by heuristic addition, averaging, comparison, or selection of different information sets. The process of synthesizing the information set can use a stochastic synthesis system such as the Neyman-Pearson multimodal fusion system (see, eg, US Pat. No. 7,287,013). While heuristic systems may be less computationally complex, Neiman-Pearson multimodal fusion systems can produce more accurate output at the cost of increased complexity.

The step of mathematically synthesizing the scan value data includes specifying a weighting factor for each scan frequency and multiplying each scan value data by the corresponding weighting factor to generate a product of the scan values. May be included. The product of scan values can be added to produce a sum, and the sum can be divided by the number of scan frequencies to produce an average value for each of the pixels. This average value can be used as the synthesized value mentioned above. The weighting factor can be calculated using the following formula:
w (f _i ) = (e ^{(avgi * fi)-e (avgi * fmax)} ) / (e ^{(avgi * fmin)-e (avgi * fmax)} )
here,
w (f _i ) is a weighting factor for the i-th scan frequency,
avg _i is the average value of the scan value data at the i-th scan frequency and the next lower scan frequency,
f _min is the lowest scan frequency,
f _max is the highest scan frequency.

  Another method for mathematically synthesizing the scan value data is to create a covariance matrix for each of the scan frequencies from the scan value data in the information set and to mathematically synthesize the covariance matrix for each pixel. Providing a synthesized matrix having synthesized values for. To synthesize the covariance matrix, the corresponding components in each of the covariance matrices may be interpolated to provide a synthesized covariance matrix, and the components of the synthesized covariance matrix are interpolated Value.

One or more of the covariance matrices may be weighted. If weighting for a particular scan frequency is desired, a weighting factor for the corresponding covariance matrix may be specified, and each component in the corresponding covariance matrix may have its components before mathematically synthesizing the covariance matrix. Can be multiplied by a weighting factor. The weighting factor can be calculated using the following formula:
w (f _i ) = (e ^{(avgi * fi)-e (avgi * fmax)} ) / (e ^{(avgi * fmin)-e (avgi * fmax)} )
here,
w (f _i ) is a weighting factor for the i-th scan frequency,
avg _i is the average value of the scan value data at the i-th scan frequency and the next lower scan frequency,
f _min is the lowest scan frequency,
f _max is the highest scan frequency.

  A scan frequency can be selected by scanning in the absence of a finger at multiple test frequencies and identifying a peak test frequency. The peak test frequency is the test frequency at which the test frequency immediately below and the test frequency immediately above returns less energy than the peak test frequency. Once several peak test frequencies have been identified, the peak test frequency used to evaluate the fingerprint can be selected. The selection may have a higher reflected energy than most of the other peak test frequencies. That is, if an information set of three (or some other number) peak frequencies is used to evaluate a fingerprint, three (or some other number) peak test frequencies may be selected as the scan frequency. In some implementations, the scan frequency range can range from less than 8 MHz to more than 12 MHz. In some implementations, the scan frequency range can range from less than 5 MHz to more than 25 MHz. In some implementations, the scan frequency range can range from less than 1 MHz to more than 100 MHz. Other ranges are possible. The number of scan frequencies within the selected range can range from only 2 to 50 or more. As described in more detail below, the scan frequency separation may also vary. Hyperspectral ultrasound imaging involves imaging at multiple frequencies, usually in multiple scans over different frequencies or wavelengths. Hyperspectral ultrasound imaging is considered an extension of multispectral imaging.

  Alternatively, the scan frequency can be selected based on the quality of the information set. For example, for each of the peak test frequencies, the information set quality may be evaluated and the peak test frequency with the best information set quality may be selected. For example, if three (or some other number) peak test frequencies are to be selected as the scan frequency, three (or some other number) peak tests with better quality than the other peak test frequencies The frequency can be selected and used as the scan frequency. The quality of the image information set at a particular frequency can be evaluated in various ways. For example, the quality of the information set can be determined by evaluating the image contrast ratio between the ridges and valleys of the fingerprint image. A higher quality information set may have a higher contrast ratio. Another quality measure may be related to ambiguity, ie an image with a clear depiction of ridges and valleys may have a higher quality than an image with blurred edges. The quality of the image information set can be determined for the entire image or for selected regions within the image. For example, image quality may be evaluated within the contours of the fingers, avoiding areas without fingers. The quality of the information set may be affected by the object being imaged. For example, diffraction effects can occur due to the distance between certain ridges, which can be related to the age of the person, the size of the finger, or the pattern of swirls and ridges in the finger. The diffraction effect can vary with the scan frequency. By using multiple scan frequencies in multispectral ultrasound imaging, some of the effects of diffraction can be mitigated, for example, by selective synthesis of image information sets generated at different frequencies.

  In some implementations, the initial scan frequency (e.g., the peak test frequency with the highest average amplitude or best quality) is selected as one of the scan frequencies, and then the predetermined offset is the first selected scan frequency. It may be beneficial to select additional scan frequencies by adding to and / or subtracting from them. For example, if the first selected scan frequency has a frequency of X and the predetermined offset is Y, the second of the scan frequencies may be X + Y and 3 of the scan frequencies. The second may be XY.

  Alternatively, the initial scan frequency can be selected, for example, by selecting the highest average value or peak test frequency with the best quality, and then identifying the range that includes the initially selected scan frequency. The additional scan frequency may be selected from frequencies that are within a range that includes the peak test frequency. In some implementations, the additional scan frequency may be specified to be a frequency that is a harmonic of the initially selected scan frequency, such as an integer multiple of the selected scan frequency.

  In some implementations, when multiple information sets are created, the information sets can also be used to determine whether a fingerprint has been provided by a living thing. In the method for determining biometricity, the normalized multi-frequency response of each fingerprint pixel can be formed as a vector. A first information set of information sets (e.g., FoIS (first one of the information sets)) may be selected, pixels in the FoIS corresponding to ridges (`` bulge pixels '') may be identified, and valleys may be identified. The pixels in FoIS corresponding to (“valley pixels”) may be identified. The vectors can be clustered together to form a cluster of valley pixels. For each of the other information sets, signal strength histogram distribution information (“SSHDI”) may be calculated for the raised pixels and SSHDI may be calculated for the valley pixels. The SSHDI feature value of the raised pixel may be specified, and the SSHDI feature value of the valley pixel may be specified. In some embodiments, feature values of frequency response intensity histogram distribution information (FSHDI) of raised pixels may be identified, and FSHDI feature values of valley pixels may be identified. The features mentioned above are: (a) the signal strength most commonly appearing in FSHDI or SSHDI, (b) the median signal strength appearing in FSHDI or SSHDI, (c) the statistical energy of FSHDI or SSHDI, (d ) FSHDI or SSHDI statistical entropy, or (e) FSHDI or SSHDI statistical variance.

  For each of these other information sets, the difference between the raised pixel feature and the valley pixel feature may be determined to obtain a separation value. A determination may then be made as to whether any of the separation values identifies a spatial location previously identified as corresponding to a living organism.

  FIG. 42 shows that an example of detection of a living body by multispectral / hyperspectral imaging is separation of ridges and valleys by multiple frequencies. In some embodiments, the object being scanned (eg, a finger) may not move during processing. In one embodiment of multi-frequency ridge and valley separation, ridge and valley separation is calculated. The fingerprint image is taken at the optimal frequency 420 (selected from scans at other frequencies 421) to extract (422) the region of interest (ROI) of the fingerprint. The ROI is extracted (422) and binarization can be performed on the one or more extracted ROIs. ROI binarization can extract uplift and valley maps (423). The extracted (423) ridge-valley map at the ROI and optimal frequency 420 can be applied to all other frequency 421 scans. For the scan at each frequency 421, a histogram of raised and valley pixels may be calculated separately (424). Features such as the peak bin value of the ridge-valley histogram, the median value of the ridge-valley pixel, or the energy, entropy, or variance of the ridge-valley histogram can be extracted from the ridge and valley histogram ( 425). To obtain a ridge and valley separation at a given frequency, the valley feature can be subtracted from the ridge feature. For example, peak bin values may be used as extracted features from the histogram, and ridge and valley separation curves 426 may be shown across multiple frequencies (see generally FIGS. 33-36). Different materials have different acoustic impedances, resulting in different ridge and valley separation curves 426. Thus, the ridge and valley separation curve 426 can be used to determine the biological properties of an object.

  FIG. 5 is a block diagram showing an ultrasonic sensor system for generating an ultrasonic image information set corresponding to an object. The sensor system may include an ultrasonic transmitter 55 for generating ultrasonic waves. The system may further include a signal generator 57 capable of generating a transmitter excited electrical signal to cause the transmitter 55 to provide multiple waves at a desired frequency. The system may further include an amplifier 56 for amplifying the electrical signal from the signal generator 57 and for driving the ultrasonic transmitter 55. The system may further include an ultrasonic sensor array 51 capable of detecting the reflected ultrasonic energy and providing a pixel output signal corresponding to the detected energy. The system may further include a sensor controller 53 for controlling the ultrasonic sensor array 51 and receiving pixel output signals from the sensor array 51. The system may further include an electronic subsystem 54 (eg, an application processor) for forming image information sets, each corresponding to a different frequency, and then combining the image information sets into a combined image information set. The system may further include a bandpass filter 52 for separating the received signal into frequency components.

Exemplary Embodiment FIG. 6 is a diagram illustrating a configuration of a system 60 for generating an ultrasound image information set of an object 61 that contacts an outer surface of an ultrasound sensor array 62. In this configuration, the sensor stack 62 includes a TFT substrate 63 sandwiched between an ultrasonic transmitter (bottom Tx65) and a receiver (top Rx64). The TFT substrate 63 has a TFT circuit including a pixel circuit disposed on the upper surface and a piezoelectric layer disposed on the pixel circuit. The transmitter 65 may include a piezoelectric layer with one or more electrodes disposed on each side of the piezoelectric layer. The top surface (not shown) can be coated with a protective film that can function as a platen (ie, parylene, acrylic, hard coat, diamond-like coating (DLC), impedance matching layer, or other suitable coating). The controller 66 can provide a transmitter excitation signal to the electrode associated with the ultrasonic transmitter 63. The controller 66 can send various control signals to the TFT pixel circuit to control the acquisition of ultrasound signals reflected from the top surface of the sensor array 62 and to extract pixel output signals from the sensor pixels. . Sensor array 62 and controller 66 may be in communication with one or more processors 67, such as an application processor in a mobile device that may function to process pixel output signals from sensor array 62. Alternatively, a TFT substrate comprising thin film transistors and associated circuitry formed thereon, as described throughout this disclosure, is in and in contact with a silicon substrate rather than a glass or plastic TFT substrate. It can be replaced by a silicon-based ultrasonic sensor array, such as a CMOS sensor array with formed transistors and associated circuitry.

  FIG. 7 shows a diagram illustrating another configuration of a system 70 for generating an ultrasound image information set of an object 71 that is in contact with an outer surface of a platen 72 disposed on an ultrasound sensor array 73. In this configuration, the sensor array 73 has a TFT substrate 74 sandwiched by an ultrasonic transmitter (bottom Tx76) and receiver (top Rx75) with a cover material (i.e., glass or plastic) on the top surface. The cover material may function as a cover glass or platen 72. In some implementations, the platen 72 may include a layer of polycarbonate (PC), polyethylene terephthalate (PET), polyethylene naphthalate (PEN), polyimide (PI), or other polymeric material. In some implementations, the platen 72 may be made of sapphire, alkali-aluminosilicate sheet glass, aluminum, stainless steel, alloy, polycarbonate, polymeric material, metal-filled plastic, or other suitable platen material. In some implementations, the platen 72 may include one or more coatings or acoustic impedance matching layers on one or both sides of the platen 72. The controller 77 can provide a transmitter excitation signal to the electrode associated with the ultrasonic transmitter 76. The controller 77 can send various control signals to the TFT pixel circuit to control the acquisition of the ultrasound signal reflected from the top surface of the sensor array 73 and to extract the pixel output signal from the sensor pixel. . Sensor array 73 and controller 77 may be in communication with one or more processors 78, such as an application processor in a mobile device that may function to process pixel output signals from sensor array 73.

  Different frequencies or chirp sequences for the controllers 66, 77 to excite the transmitters 65, 76 to capture pixel output data in both the first configuration (FIG. 6) and the second configuration (FIG. 7) Can be generated. Optionally, processors 67, 78 may be provided for further processing. In some implementations, the processors 67, 78 can instruct the controllers 66, 77 to excite the transmitters 65, 76. In some implementations, the processors 67, 78 may be provided on a mobile platform, such as a mobile phone, tablet computer, laptop computer, or portable medical device. In some implementations, one or more processors 67, 78 in a mobile device, such as an application processor, process data and image information sets from the sensor arrays 62, 73 to unlock the phone or user May serve to provide output for other functions such as authenticating.

  For example, the teachings herein include, but are not limited to, mobile devices, display devices, telephones, multimedia internet enabled mobile phones, mobile television receivers, wireless devices, smartphones, bluetooth devices, personal digital assistants (PDAs), Wireless email receiver, handheld or portable computer, netbook, notebook, smart book, tablet, printer, copier, scanner, facsimile device, GPS receiver / navigator, camera, MP3 player, camcorder, game console, medical equipment Wearable electronic devices, mobile health equipment, watches, watches, calculators, television monitors, flat panel displays, e-book devices (e.g., electronic readers), computer monitors, May be implemented in various electronic devices such as vehicle displays (e.g., odometer displays), cockpit controls and / or displays, camera view displays (e.g., rear camera displays of vehicles), or cash dispensers Or it may be associated with them.

8A and 8B illustrate a method of operation for multispectral ultrasound imaging. FIGS. 8A and 8B show image information sets obtained for each applied frequency with transmitter and receiver signals for a set of two or more excitation frequencies applied sequentially to the transmitter. Shown with. Two or more sets of excitation frequencies can be sequentially applied to the transmitter (f ₁ in FIG. 8A and f _n in FIG. 8B) with the information set acquired for each applied scan frequency. As shown in the upper part of each figure, one or more periods of the excitation signal at a given scan frequency may be applied to the transmitter (Tx). Two cycles are shown here. During ongoing ultrasound transmission, the control signal to the receiver bias electrode can be set to a blocking mode. The reflected ultrasound signal may be captured during the sample mode, where the control signal to the receiver bias electrode is sampled. In order to prevent detection of unwanted internal reflections, the receiver bias electrode can be returned to blocking mode for a short period of time. During the hold mode, the signal stored in each sensor pixel of the ultrasonic sensor array can be transmitted according to the clock. As shown in FIG.8A, the ultrasound image information set may be acquired with the first applied excitation frequency, and as shown in FIG.8B, the second ultrasound image information set is the second (more (High) may be acquired along with the applied excitation frequency.

  Another such method is shown in FIG. 9A where transmitter and receiver signals for two or more sets of excitation frequencies are sequentially applied to the transmitter. FIG. 9A shows the transmitter and receiver signals for two or more sets of excitation frequencies applied sequentially to the transmitter, along with the set of image information acquired for the set of excitation frequencies. In this method, a peak detector formed as part of a pixel circuit is used to detect peaks during sample mode as reflected ultrasound signals from various scan frequencies are detected within a single sample interval. Can be used to obtain signal amplitude. FIG.9B shows the transmitter and receiver signals for a chirp transmitter excitation sequence (up-chirp or down-chirp) starting at the first frequency and ending at the second frequency (up-chirp sequence is shown) ). In this method, a peak detector formed as part of a pixel circuit detects a reflected ultrasound signal from a chirped excitation signal applied to an ultrasound transmitter within a single sample interval. As such, it can be used to obtain the peak signal amplitude during the sample mode.

  An example of the present invention can utilize a synthesis method that uses discrete frequencies. FIG. 4 is a flow chart illustrating one such method utilizing an arithmetic average per pixel with optional weighting. The method uses a pixel-by-pixel average to calculate a composite value (87) for each pixel from the acquired (81, 83, 85) information set, each derived from a different frequency. Accompany. Using this method, an ultrasound information set having respective frequency components is acquired at blocks 81, 83, and 85 by the ultrasound sensor array. For each of these acquired information sets, the pixel output signal or value may be digitized at each image pixel at coordinates (x, y), where x is the horizontal dimension of the sensor array; y is the vertical dimension of the sensor array. The digitized value of the pixel at each location (x, y) can be summed across all selected information sets. Based on the frequency components, these pixel values can be multiplied by pre-assigned weights. This sum of pixels at each x, y location for all selected information sets may be divided by the number of information sets acquired. This operation can be repeated for some or all of the pixel locations. The resulting data may be referred to as a composite representation or a composite image information set.

  FIG. 10 shows the arithmetic mean for each point of two or more sets of ultrasound image information. In FIG. 10, the pixel sizes of a plurality of selected pixels from information sets at different frequencies are added together and optionally normalized to form a combined representation or image information set. Using point-by-point arithmetic averaging with weighted averages, the magnitudes from the information set at different frequencies for the same pixel are multiplied by their respective frequency-dependent weights, added together, and optionally normalized. A combined representation or image information set can be formed.

  One example of the present invention can utilize covariance-based interpolation with optional weighting. FIG. 11 is a flowchart illustrating a method for generating a combined image information set using two or more covariance matrices. In such a method, a first received image may be acquired with a first scan frequency (101). A second received image at a second scan frequency different from the first scan frequency may be acquired (103). Additional images for other scan frequencies may also be acquired (105). The horizontal and vertical size corresponding to the acquired image using the image pixel at coordinates (x, y), where x is the horizontal dimension of the image and y is the vertical dimension of the image A covariance matrix of the first image with can be computed (107). A second covariance matrix may be calculated from the second image (109). An additional covariance matrix may be calculated from the additional acquired images (111). Covariance matrices may be generated from images acquired at different frequencies, and the matrices are synthesized using interpolation (e.g., linear interpolation, cubic interpolation, bicubic interpolation, or spline interpolation) (113) , And optionally can be normalized to form a composite representation.

FIG. 12 shows a method for generating a composite image information set using covariance-based interpolation. The covariance matrix of each acquired image at each excitation frequency can be synthesized using interpolation (e.g., linear interpolation, cubic interpolation, bicubic interpolation, or spline interpolation) with optional weighting. . The elements of each covariance matrix at each coordinate (x, y) can be interpolated using the respective elements at the same location across all acquired information sets. Each of these element values can be multiplied by the assigned weight based on the excitation frequency. The multiplication (139) operation may be repeated for some or all pixel locations to form a composite (resulting) image. A block of information set data a ⁱ may be represented as a ⁱ = F * a ^o + n, where F is a known NxN matrix and a ⁰ is a block of the original information set (e.g., a row or Nx1 vector of pixel values to be estimated, representing a column), and n is a Nx1 noise vector whose average can be zero. The estimated block of information set data may be calculated as a ^e = (F ^T C ⁻¹ F) ⁻¹ F ^T C ⁻¹ a ⁱ . The covariance matrix C for the block of initial image data a ⁰ can be expressed as:
C _f1 C _f1 C _f1 C _f2 … C _f1 C _fn
C = C _f2 C _f1 C _f2 C _f2 … C _f2 C _fn
::::
C _fn C _f1 C _fn C _f2 … C _fn C _fn

  After estimating the statistics for blocks in the original information set, interpolation may be used to estimate statistics centered on each pixel in the information set. After calculating the statistics around each pixel, estimated image data for that pixel can be calculated. For example, each synthesized value can be correlated with a grayscale value (140). The estimated image data can be obtained by combining the results from each block of estimated image data. A combined representation may be obtained by combining the estimated image data results from each set of initial image data (eg, from various excitation frequencies). For example, the grayscale value may be given as a representation of a fingerprint (141).

  Some implementations may utilize a method for generating weights based on the transmission frequency. The frequency used for ultrasound transmission generally has an attenuation in the material used in the sensor stack that varies exponentially. One approach to generating weights for multi-frequency ultrasound imaging is to associate various frequencies with exponentially derived weighting factors. There can be n-1 consecutive weights for as many as n information sets generated using different excitation frequencies. The information set may be arranged in descending order of excitation frequency, the image with the highest frequency being weighted with a first weight (e.g. 1) and the information set with the second highest frequency being the second exponential And so on, and so on.

  Spatial registration may be used to obtain a composite representation (ie, a composite image information set) from a set of image information acquired using various excitation frequencies. Spatial registration may involve realignment of features from each image using techniques such as block-by-block distortion. Alternatively, spatial registration can be realized using motion compensation techniques. Methods such as normalized cross-correlation, mean square error, sum of absolute difference, or mutual information can be used to synthesize two or more images from different excitation frequencies. Resizing, rotating, nearest neighbor, linear, cubic, or spline techniques can be used to combine two or more image information sets to obtain a combined image information set. Other methods for obtaining a combined information set may include edge detection or gradient based methods.

  Based on the frequency response of the ultrasonic sensor array (which depends in part on the components and placement of the sensor stack during the evaluation, preparation, or calibration procedure), the frequency for multispectral ultrasound imaging is Can be selected. More than one frequency can be used. In some implementations, the system can be calibrated or self-calibrated to determine a preferred set of frequencies for imaging.

  FIG. 13 is a flowchart illustrating one or more methods for providing a target value representation in accordance with the present disclosure. One method may include selecting (131) a peak test frequency that has a higher reflected energy than the other test frequencies. The method may include evaluating (132) the image quality of the peak test frequency and selecting (133) the scan frequency based on the peak frequency with high image quality. Additional scan frequencies can be identified (134) by applying a predetermined offset to the selected (133) frequency. The method may further include identifying (135) additional scan frequencies by identifying harmonics of the selected (133) scan frequency. The method shown in FIG. 13 may further include scanning (136) the fingerprint at a plurality of scan frequencies. The scan value data from the scanning step (136) can be mathematically synthesized to correspond to each of the scan frequencies. In some implementations, a weighting factor for the scan value data may be identified (138). Each component in the covariance matrix may be multiplied by a corresponding weighting factor (139). Each synthesized value may be correlated with a grayscale value (140). In some implementations, each synthesized value may be correlated without specifying a weighting factor or multiplying each component in the covariance matrix by the weighting factor. The grayscale value may be given as a representation of a fingerprint or other object such as a stylus (141).

An example of the frequency response of an ultrasonic sensor array is shown in FIG. 14, where multiple peaks (local resonances labeled as f _r1 , f _r2 ,... F _r5 ) and valleys are visible.

  The step of calibrating the ultrasonic sensor system may be performed by varying the frequency (eg, from about 1 MHz to about 25 MHz) to cause the ultrasonic transmitter to emit ultrasonic waves to determine the system response. The system may operate with and without transmitter excitation, a background information set with transmitter excitation off, transmitter excitation on to determine system response. Subtracted from the state image information set. Acquisition of the image information set may be performed, for example, pixel by pixel or as an arithmetic average (average) of some or all pixels in the ultrasonic sensor array.

Six graphs are shown in FIGS. 14A-14F, each showing how the operating frequency can be selected. In the upper left graph (FIG. 14A), the frequency with the highest amplitude response at f _r2 is selected. In the lower left graph (FIG. 14D), the frequencies (f _r2 and f _r3 ) with the two highest amplitude responses are selected. In the upper graph (FIG. 14B), the frequencies (f _r1 to f _r5 ) with the five highest amplitude responses are selected. In the lower middle graph (FIG. 14E), the frequency with the best response quality corresponding to _fr3 and _fr4 is selected. In the upper right graph (FIG. 14C), the preferred operating frequency (f _r2 ) is selected, and one or more frequencies lower or higher than the preferred operating frequency are selected (eg, selected in equal steps). In the lower right graph (FIG. 14F), one or more preferred operating frequencies are selected (eg, f _r3 ), and a range of frequencies below, including, or above the preferred operating frequency is selected. In some implementations, the selected frequencies can have equal spacing between the frequencies.

15A-15B show graphs corresponding to the selection of additional excitation frequencies. In the upper graph (Figure 15A), one or more frequencies are selected for transmitter excitation, and the information set is one or more harmonics of the excitation frequency (e.g., an integer of the first selected frequency). Times, this may be at certain peaks in the frequency response, such as 5 MHz, 10 MHz, 15 MHz, and 20 MHz, as shown). In the graph below (Figure 15B), the frequency with the highest amplitude response or response quality (e.g., _fr2 ) is selected for transmitter excitation and the image is one or more harmonics of the excitation frequency. Acquired in the waves.

  For some materials, the sound may travel faster during the compression phase of the wave compared to the dilute phase, causing non-linear propagation of the sound waves. This non-linearity of the sound traveling through the medium can produce a received signal with various harmonics of the excitation frequency. Alternatively, ultrasound nonlinearity can produce a response as the sum or difference of frequencies when two or more excitation frequencies are used, such as a carrier frequency and a frequency modulated portion. The harmonics generated as a received signal are not dominant in the near field, but may still be present and detectable. During multispectral imaging, various harmonics can be received by the ultrasonic sensor array. In a thickness mode in which ultrasound propagates in a direction perpendicular to the surface of the ultrasound transmitter, the sensor stack can resonate at the fundamental frequency and the associated odd multiples. A chirp transmission sequence generated at or near the excitation frequency at or near the fundamental frequency, or a band covering the fundamental frequency, may be transmitted to cause resonance and associated harmonics. The information set formed by the harmonic components of the applied frequency can be used as input for a pixel-by-pixel average or covariance based interpolation method to generate a combined representation or image information set. These approaches can increase the resolution and contrast of the ultrasound imaging system, and the fundamental frequency can be filtered out during signal processing.

The system can be calibrated or self-calibrated to determine the preferred frequency for capturing the representation of the object. 16A to 16B are graphs showing a calibration method. In the first method (Figure 16A), a lower scan frequency `` 1 '' (e.g. 1 MHz) and a higher scan frequency `` 2 '' (e.g. 24 MHz) may be selected to determine the system response and resonance peak In order to do so, a scan is performed between a lower scan frequency and a higher scan frequency. In this example, the frequency “3” (f _r2 ) has the greatest response and can be selected for operation. In the second method (FIG. 16B), the test object may be placed on the sensor platen, which includes protrusions and indentations representing various separations between the fingerprint ridges and valleys. A scan may be performed and the frequency “3” with the best image quality may be selected for operation (this may be at a frequency other than the peak frequency). For various calibration methods, changes in the selected frequency with temperature changes can be modeled by temperature testing and calibration over a range of temperatures, or by modeling the typical temperature response of an ultrasonic sensor for local temperature measurements. Can be determined by modifying the applied excitation frequency based on it.

  17A-17B show graphs showing additional methods of calibration. In the third method (Figure 17A), a lower scan frequency `` 1 '' (e.g. 1 MHz) and a higher scan frequency `` 2 '' (e.g. 24 MHz) may be selected to determine the system response and resonance peak In order to do so, a scan is performed between a lower scan frequency and a higher scan frequency with the test object. The first frequency `` 3 '' indicating the lowest output signal when a test object such as skin is placed on the sensor platen (representing a fingerprint ridge) may be determined, and air (representing a fingerprint valley) In this case, the second frequency “4” indicating the highest output signal may be determined. Note that the highest output signal for air and the lowest output signal for test subjects such as skin do not always occur with the highest system peak and the lowest system peak. In this example, two determined frequencies 3 and 4 can be selected for operation. Changes in frequency 3 and 4 with temperature changes can be included. In the fourth method (Figure 17B), a lower scan frequency `` 1 '' (e.g. 1 MHz) and a higher scan frequency `` 2 '' (e.g. 24 MHz) may be selected to determine the system response and resonance peak In order to do so, a scan is performed between a lower scan frequency and a higher scan frequency with the test object. For example, the first frequency “3” may be determined based on a low output signal when a test object such as skin is placed on the sensor platen, eg, high when air is in contact with the platen. Based on the output signal, the second frequency “4” may be determined, and some frequencies “5” between 3 and 4 are determined (eg, based on an increment of the allowed system frequency). May be. In a frequency band, the intermediate frequency 5 along with the two determined frequencies 3 and 4 can then be selected for operation. A change in frequency with a change in temperature may be included. In some implementations, a second band of scan frequencies may be included. The second band of scan frequencies may be determined in a similar manner, but note a frequency range that is different from the first frequency range.

  FIG. 18 is a flowchart illustrating one method of multispectral ultrasound imaging with chirped excitation. Specifically, FIG. 18 illustrates a method of chirped excitation of a single information set. In such a method, only a single information set is acquired. This method works with a pixel circuit based on a peak detector and uses a controlled distance gate time delay between the start of transmitter excitation and the start of sample mode, this time delay being at the desired time. It may be adjusted to receive the reflected signal. This chirp sequence may be generated to generate a single information set, allowing higher frame rates. In one embodiment, a chirp sequence is determined (181). The chirp sequence is applied to the transmitter (183). The receiver captures a peak signal at each pixel from the chirped transmitter excitation (185). The chirped image is then acquired (187), for example, by a processor operating on the captured (185) peak signal.

  FIG. 19 is a flowchart illustrating a method for chirped excitation of multiple images with an arithmetic average per pixel (and optional weighting). In one embodiment, the chirp sequence is determined (191), eg, by a processor or through previous configuration. A first image is acquired using a first chirp sequence (192). A second image is then acquired using the second chirp sequence (193). Additional images may be acquired for various other chirp sequences (194). A first covariance matrix may be calculated from the first image (195). A second covariance matrix may be calculated from the second image (196). An additional covariance matrix may be calculated (197) from any additional acquired (194) images. The covariance matrix may be synthesized to form a synthesized image, for example, by using interpolation (eg, linear interpolation, cubic interpolation, bicubic interpolation, or spline interpolation) (198). The covariance matrix composition 198 may be weighted.

  FIG. 20 shows two graphs that help illustrate the determination of the chirp sequence. The upper graph of FIG. 20 shows a linear chirp that starts at a first frequency and ends at a second frequency. The lower graph of FIG. 20 shows an exponential chirp that starts at a first frequency and ends at a second frequency. One or more information sets may be acquired during any scan (eg, at the frequency indicated by the arrows). A chirp sequence may be, for example, linear, quadratic, exponential, logarithmic, or may include individual frequencies. A chirp generally has a start frequency and an end frequency. Some chirps can start at a low frequency and end at a high frequency, while other chirps can start at a high frequency and end at a low frequency. The start chirp frequency and end chirp frequency may be the frequency at the peak amplitude, the frequency at the second peak amplitude, the frequency at or near the peak frequency, or other frequencies of the ultrasonic sensor.

  Several different types of chirp sequences that can be used are: 1) extended chirp with an extended range of frequency components, 2) frequency extending between the highest peak frequency of the receiver array and the second highest peak frequency 3) a neighboring chirp with a frequency around one of the system peaks, and 4) a gap with two or more bands of frequencies extending through one or more peaks of the ultrasonic sensor array It is chirp with. The chirp sequence can be selected based on the highest peak of the system response. A chirp sequence may be selected based on the image obtained therefrom, and the chirp sequence is determined from an evaluation of image quality or other measure. One or more chirps may be applied sequentially (eg, repeated). A single information set can be obtained using a chirp sequence with multiple frequency components covering the maximum frequency response of the receiver. Multiple information sets may be obtained using one or more chirp sequences, and these information sets are combined. Ultrasonic sensors can be calibrated using these chirp sequences.

  21A-21B show graphs showing various chirp sequences. In the first method (extended chirp, see Figure 21A), a lower chirp frequency `` 1 '' (e.g. 5 MHz) and a higher chirp frequency `` 2 '' (e.g. 20 MHz) are selected, with a lower chirp frequency and higher Chirping is performed between the chirp frequency. In some implementations, a lower chirp frequency and a higher chirp frequency may be determined based on measurement of the frequency response of the system. In some implementations, a lower chirp frequency and a higher chirp frequency may be selected based on testing a similarly constructed sensor array. After the chirp is complete, an information set can be obtained. Alternatively, one or more information sets can be obtained during the chirp. In the second method (peak-to-peak chirp, see FIG. At or near, and chirped between a lower chirp frequency and a higher chirp frequency. After the chirp is complete, an information set can be obtained. Alternatively, one or more information sets can be obtained during the chirp.

A linear chirp signal has a frequency that varies linearly with time, for example, for 0 <time <T,
Chirp (time) = sin [2π (f ₀ + (B / 2T) * time) time]
Where f ₀ is the starting frequency, B is the frequency bandwidth, and T is the chirp duration.

  22A-22B show graphs illustrating various other chirp sequences. In the third method (neighboring chirp, see Figure 22A), a lower chirp frequency `` 1 '' (e.g. 4 MHz) and a higher chirp frequency `` 2 '' (e.g. 6 MHz) covering the system peak are selected and more A scan is performed between a lower chirp frequency and a higher chirp frequency. After the chirp is complete, an information set can be obtained. Alternatively, one or more information sets can be obtained during the chirp. The fourth method (gapped chirp, see Figure 22B) has a lower chirp frequency `` 1 '' (e.g. 4 MHz) and a higher chirp frequency `` 2 '' (e.g. 6 MHz) around the peak of the first resonance. A second lower chirp frequency “3” (eg, 15 MHz) and a second higher chirp frequency “4” (eg, 17 MHz) are selected around the peak of the second resonance. A scan is performed between the lower and higher chirp frequencies of the first range (1 to 2), followed by the lower and higher chirp frequencies of the second range (3 To 4) can be scanned. After the chirp is complete, an information set can be obtained. Alternatively, one or more information sets can be obtained during the chirp.

  A chirp coded transmitter signal can be generated using a linear frequency band around the peak amplitude response of the ultrasound system. Both the broadband pulse and the chirp pulse may have the same peak amplitude, but the chirp pulse may have a much larger pulse energy due to its greater length. In general, the more signal energy that is transmitted, the greater the reflected signal. Chirp pulses can be formed with amplitudes and frequencies that change during the pulse. A shorter chirp pulse may allow for a faster sensor frame rate. A chirped pulse may use a single transmitted pulse to mitigate motion artifacts that may be caused by object motion during multipulse, multifrequency transmission pulses.

  FIG. 23A and FIG. 23B show graphs showing the FFT of the chirp coded transmitter signal. FIG. 23A shows an FFT of a chirp coded “extended chirp” transmitter signal, with a linear frequency band from 5 MHz to 20 MHz. FIG. 23B shows an FFT of a chirp-coded “peak-to-peak chirp” transmitter signal with a linear frequency band from 7.5 MHz to 12.5 MHz.

  FIG. 24A and FIG. 24B show graphs showing the FFT of the chirp coded transmitter signal. FIG. 24A shows an FFT of a chirp coded “neighboring chirp” transmitter signal with a linear frequency band from 7 MHz to 8.5 MHz. FIG. 24B shows an FFT of a chirp coded “gap chirp” transmitter signal with linear frequency bands of 7.5 MHz to 8.5 MHz and 11 MHz to 12.5 MHz.

  FIG. 25 is a diagram showing an ultrasonic sensor array. The sensor array 243 may include a TFT substrate 241 and a receiver 247. The sensor array 243 may be in physical communication with a display / cover 242 such as a glass cover or LCD display. The receiver 247 can include one or more receiver biases 245 and one or more receiver electrodes 246. The sensor array 243 can also be positioned with respect to the transmitter 250. The transmitter 250 can include a plurality of transmission electrodes 248 and 249. The sensor array 243 may have a pixel pitch of about 50 μm. The sensor array 243 can range in size from about 15 mm x 6 mm to full display size. Other sizes can be 11 mm x 11 mm, and 1 inch x 1 inch sizes. The sensor array 243 may have a low profile (about 1 mm). The sensor array 243 may have a high operating frequency (5-25 MHz). The sensor array 243 can be on the periphery of the fingerprint scanning device, behind a portion of the display 242, behind the entire display, or elsewhere in the device housing.

  FIG. 26 illustrates some possible configurations of an ultrasonic fingerprint sensor according to the present disclosure. The sensor may have a separate or common TFT substrate for the display and fingerprint sensor. A common cover glass or touch screen may be shared between the sensor elements. An ultrasonic fingerprint sensor array (and optional coding or cover layer) may be placed on the bezel, side or back of the mobile device housing. The sensor may be placed on a button (mechanical or non-mechanical button, authentication or non-authentication button) or may be placed as part of the button. For example, the fingerprint sensor 264 may be at the periphery of the display (including the display color filter glass 262 and the display TFT substrate 263). In this example, the fingerprint sensor 264 is placed under the cover glass 261 of the display. In another example, the fingerprint sensor 264 can be placed separately from the display and coded with a cover layer to protect the sensor 264. In another example, fingerprint sensor 264 may be placed under (or behind) the display (including display color filter glass 262 and display TFT substrate 263). In another example, the fingerprint sensor 264 can be incorporated into the TFT substrate 263 of the display.

  FIG. 27 shows a block diagram of one such ultrasonic sensor system. The ultrasonic sensor system of FIG. 27 includes an ultrasonic transmitter 271 having an ultrasonic sensor pixel circuit array 272. The ultrasonic transmitter 271 is in electrical communication with the transmitter driver 276 (eg, through one or more electrical connections). For example, the transmitter driver 276 may have a positive polarity signal and a negative polarity signal in electrical communication with the ultrasonic transmitter 271. The transmitter driver 276 may be in electrical communication with the control unit 279 of the sensor controller 278. The control unit 279 can provide a transmitter excitation signal to the transmitter driver 276. The control unit may also be in electrical communication with the receiver bias driver 274 through a level select input bus. The receiver bias driver 274 can provide a receiver bias voltage to a receiver bias electrode disposed on the surface of the piezoelectric receiver layer that can be attached to the ultrasonic sensor pixel circuit array 272. Control unit 279 may also be in electrical communication with one or more demultiplexers 277. Demultiplexer 277 may be in electrical communication with a plurality of gate drivers 275. The gate driver 275 may be in electrical communication with the ultrasonic sensor pixel circuit array 272 in the ultrasonic transmitter 271. The gate driver 275 may be located external to the ultrasonic sensor pixel circuit array 272 or may be included on the same substrate as the ultrasonic sensor pixel circuit array 272 in some implementations. A demultiplexer 277 that can be external to or included with the ultrasonic sensor pixel circuit array 272 can be used to select a particular gate driver 275. The ultrasonic sensor pixel circuit array 272 may be in electrical communication with one or more digitizers 273. The digitizer 273 can convert the analog signal from the ultrasonic sensor pixel circuit array 272 into a digital signal suitable for the data processor 280 in the sensor controller 278. The sensor controller 280 can provide a digital output to an external system, such as an application processor of a mobile device.

  When an information set is acquired at a particular frequency, the resulting information set may have a reversed fingerprint sharpness (e.g., a region of ridges that normally appear bright in the sensor array output image appears dark and the finger The opposite is true for the valley region). These observations can occur at several specific frequencies within the operating range of 5-20 MHz, but can have the largest output in a narrow range for a pre-defined optimal frequency . The hypothesis for this behavior is that a standing wave is generated by the resonance of the transmit and receive signals in the sensor stack, and that the standing wave interferes to strengthen or weaken at a particular frequency. It produces patterns.

  An example of such behavior is shown in FIG. FIG. 28 shows the frequency response of the difference between the output of the array with tone burst on and the output of the array with off. The fingerprint image is superimposed at the frequency at which it was acquired, and the image marked by the thick black box appears to be inverted. Usually, the peak response is determined to be the optimum operating frequency (optimum frequency) that produces the maximum signal transmission. The overlay plot includes fingerprint images taken at each frequency. An image whose outline is a thick black box emphasizes the inversion of the image (the grayscale value for the fingerprint ridge and the grayscale value for the valley are interchanged). Note also that this behavior is much stronger, sensitive and observable around the peak frequency. This behavior can be used to process the information set and improve the overall signal-to-noise ratio of the output image. In this proposed method, the fingerprint image is above the previously determined `` optimal frequency '' (the frequency at which the difference between the tone burst on signal read by the array and the tone burst off signal is the largest) and It can be taken simultaneously at the bottom two to four frequencies. Once the information set is obtained, there can be one of several ways to improve the identification of distinct fingerprints and valley patterns.

  One such example is shown in FIG. 29, where two images were acquired, one at the optimal frequency (12.6 MHz) and the other somewhat resulting in complete inversion of the raised and valley regions. Acquired at low frequency (10.4MHz). These individual images are then subtracted from the corresponding background images (images taken at the same frequency without a finger) for better separation. As an example, in post-normalization, data captured at two different frequencies are added, which results in an increase in the overall signal due to the ridge and valley regions. There can be several other image processing methods that can be implemented that serve the same purpose of improving separation in response. FIG. 29 shows the signal amplitudes obtained in two cases (one at a single optimal frequency and one at two frequencies, one of which resulted in an inversion of the image). The histogram plot reveals a typical voltage output distribution for the two cases, the latter showing a significantly higher output distribution.

  One object of the present disclosure is to use a specific target frequency that leads to improved fingerprint sharpness by utilizing multiple frequency-related signal inversions. There may be several processing methods to improve the sharpness and can be chosen based on the specific problem. There are several interrelated factors related to image capture based on sampling parameters. Important factors that will affect the observation of such reversal behavior are the delay between sampling and burst start, the number of ultrasonic pulses used, and the frequency of the ultrasonic pulses. However, with appropriate tuning of the sensor, these parameters (ie, number of pulses, delay, burst start, and frequency) can be adjusted.

  For example, one way to improve the distinction between distinct ridges and valley patterns is to use two distinct frequency settings (one is "normal" where the ridges appear bright and the other is darker than the valley region) With the step of acquiring a fingerprint image in “invert”). FIG. 30 shows sample image contour and histogram plots, which show the voltage corresponding to fingerprint ridges and post-transformation by valley ADCs (analog-to-digital converters). The histogram and image plots in FIG. 30 have the same scale for easy comparison. In FIG. 30, the contrast between the ridge and valley regions appears to be equivalent, but the histogram corresponding to each of the frequencies shows a frequency (6.5 MHz) with a distribution of information sets corresponding to the ridge and valley regions. ) To other frequency (8.5MHz).

  Further review of the data distribution is shown in FIG. FIG. 31 is a variability plot showing a comparison of analog voltages at selected raised and valley points. Here, randomly selected groups of points corresponding to the finger ridge and valley regions are tracked for the two operating frequencies. It can be seen that the region representing the finger ridge shows the greatest change between the two operating frequencies, but the “valley” region of the finger remains largely unchanged.

  By obtaining measurement results at a selected frequency, the difference between the raised and valley regions can be amplified by tracking the image or pixel region based on the change in output with frequency. For effective processing, regions of ridges whose output is more likely to change with frequency can be identified using subsequent thresholding with appropriate gradient region processing of the information set. Another potential advantage is an improvement in the SNR (signal to noise ratio) of the acquired image. Identifying the region with the largest gradient and the region with the smallest gradient between the two operating frequencies can potentially improve the SNR compared to single frequency image acquisition processing.

  In order to determine the optimal operating frequency, standard factory-like calibration methods can be utilized by using a target material (eg, rubber) whose acoustic characteristics are similar to a finger. Two sets of measurements can be taken, one set with the platen completely covered by the target material (simulating a finger) and the other with no target on the platen (`` air '' Measurement). The frequency of the tone burst signal may be swept and a TFT response is captured for both cases (with and without the object). The difference between the two signals is then the best inversion behavior, given by the negative maximum and the positive maximum of the difference signal in the case of air and in the presence of the object (`` air minus object ''). Used to determine the optimal point to be observed. FIG. 32 shows a negative peak and a positive peak when the difference obtained by subtracting the voltage when there is an object from the voltage when air is plotted against the frequency of the ultrasonic signal.

  Another embodiment of the present disclosure may relate to determining fingerprint biometrics. Fingerprints have been found to be an effective biometric characteristic for distinguishing the identity of an object. Fingerprint authentication is widely used. However, fingerprint authentication is vulnerable to impersonation. A fake finger (also known as a “spoof”) can be made from a registered real finger mold and used to obtain unauthorized authentication. This type may be created with or without user cooperation. In order to protect against the use of spoofs, attempts may be made to determine whether an object is alive. Existing biological tests can be divided into two groups. One group is an image-based approach that distinguishes between real and fake fingers using faint features that are visually perceptible in a fingerprint image. This technique requires a fairly high resolution (500-1000 dpi) in order to properly assess biological properties. The second group of biometric tests is a hardware based approach that requires hardware other than a fingerprint sensor to capture biological features such as blood pressure, pulse, conductivity, and the like.

  One embodiment of the present disclosure incorporates testing biogenicity by using a multi-frequency ultrasound information set. At the optimal operating frequency, both real, fake and fake fingers look the same and may offer very little in image-based biometric features. However, different materials have different ultrasonic reflectance over different frequencies. Differences over a range of frequencies can be used to identify the spoof. For each pixel, a vector of biological features can be extracted. The vector can be normalized using the reference frequency response. The normalized frequency response vector can then be processed to generate a multi-frequency signature of the material and thus a good indication for biological properties. FIGS. 33-36 show the results of the multi-frequency approach for biological testing.

  One method of determining biogenicity using a multi-frequency approach includes selecting a first information set (“FoIS”) of information sets, where the group of information sets includes multi-frequency ultrasound. Contains a set of information captured by the sensor. The method may further include identifying a pixel in the FoIS that corresponds to the fingerprint ridge (“protrusion pixel”). The method may further include identifying a pixel in the FoIS that corresponds to the valley of the fingerprint (“valley pixel”).

  For each of the other information sets, the method may further include calculating SSHDI or FSHDI for the raised pixels and SSHDI or FSHDI for the valley pixels. For each of these other information sets, the method may further include identifying an SSHDI or FSHDI feature for the raised pixel and an SSHDI or FSHDI feature for the valley pixel. For each of the other information sets, the method may further include the step of determining a difference between the feature values of the raised pixels and the feature values of the valley pixels to obtain a separation value. For each of the other information sets, the method may further include determining whether the separation value specifies a spatial location previously identified as corresponding to the organism.

  In one embodiment, the feature is the signal strength that most commonly appears in SSHDI or FSHDI. In another embodiment, the feature is the median signal strength that appears in SSHDI or FSHDI. However, the feature quantity may be SSHDI or FSHDI statistical energy, statistical entropy, or statistical variance.

  The following describes operational information for a particular sensor that uses multiple frequencies and ultrasound to obtain information about an object such as a fingerprint according to the present disclosure. The operational information may include the material type and other characteristics of the sensor. Note that although this particular sensor uses an integrator to detect the signal peak, other devices may be used to detect the signal peak.

For this particular sensor, a tone burst generator function is created. In the following equation, f = frequency, n = number of pulses, t = time, t ₀ = start time, and A = amplitude. The tone burst generator function may be represented by the following equation:

  The reflected tone burst can be represented by the following equation:

  The speed of sound in PVDF, parylene, and polycarbonate can be as follows:

The PVDF, parylene, and polycarbonate thicknesses in this particular sensor can each be expressed as:
δ _pvdf : _28μm , δ _pary : = 25μm, δ _pcar : = 254μm

The distance gate function can be represented by the following equation:
RangeGate (t, rgstart, rgstop, X): = if [(t ≧ rgstart) Λ [t ≦ (rgstop)], X, -X]

The index, the sequence of times, and the number of pulses in the tone burst can be expressed as:
j: = 0..2000, t _j : = jns, n: = 4

  The parylene coating on the piezoelectric layer and the top of the piezoelectric layer can first be observed. The following equation represents a possible observation.

The following parameters may be relevant for this particular sensor.
rg _on : = 150ns, rg _off : = 600ns (arbitrary gate start and distance gate end)
p (f, τ): = η (f, n, τ, 0ns, 1) (main pulse)
r (f, τ, δt): = rη (f, n, τ, δt, 1) (reflected pulse)
x (f, τ, δt, σ): = p (f, τ) + r (f, τ, δt)-σ
(Interferometrically modulated pulse (a pulse that satisfies the intrinsic reflection at the receiver layer))
q (f, τ, δt, σ): = if (x (f, τ, δt, σ) <0, 0, x (f, τ, δt, σ))
(Rectified electrical signal due to pulse and reflection)

Film (t, δt, X): = if (t <δt, -X, X) (film thickness marker function)

  The following equation represents the critical distance gate point, δt is the thickness of the platen (start of echo),

Is the length of TB (TB end),

Is the end of the echo.

  A frequency sweep of the output can be captured from the receiver. For example, the frequency sweep starts at 1 MHz and gradually increases in 0.1 MHz increments until the upper frequency, eg 33 MHz, is reached. The following configurations are used to capture signals as shown in FIGS. 37A-37C and 39A-39C.

Capture (f, t, gs, ge, σ): = if [(t ≧ gs) Λ [t ≦ ge], p (f, t) + r (f, t, δt) σ, 0]
Rectifier (f, t, gs, ge, σ): = if (Capture (f, t, gs, ge, σ) ≤0, 0, Capture (f, t, gs, ge, σ))

  39D-39F show the receiver output integrated by frequency based on various distance gates.

  Using the following configuration (with a 254 μm polycarbonate platen added), signals as shown in FIGS. 38A-38C are captured.

Capture (f, t, gs, ge, σ, δt): = if [(t ≧ gs) Λ [t ≦ ge], p (f, t) + r (f, t, δt) σ, 0]
Rectifier (f, t, gs, ge, σ, δt): = if (Capture (f, t, gs, ge, σ, δt) ≤0, 0, Capture (f, t, gs, ge, σ, δt ))

  The following integration functions can be used.

  To represent the integrated valley and ridge difference, the following equation can be used:

  In this particular sensor, the ultrasonic signal enters the piezoelectric film and reflects down through the film. When the signal encounters a fingerprint valley (air), both incident and reflected pulses excite the piezoelectric film to generate an electrical signal. In situations where the signal passes through finger tissue, such as a fingerprint ridge, only the incident pulse excites the piezoelectric film. The signal can be delayed between the film and the object by a delay line, such as a 254 μm polycarbonate (see FIGS. 40A-40C) or a 500 μm platen (see FIGS. 41A-41C). FIGS. 40D-40F show the frequency-integrated receiver output based on various distance gates for a 254 μm polycarbonate layer. FIGS. 41D-F show the receiver output integrated by frequency based on various distance gates for a 500 μm platen layer.

  FIG. 43 is a flowchart of a multispectral method for scanning a finger. The method may include a step (431) of scanning the finger with a plurality of ultrasound waves at various scan frequencies. The method may further include generating (432) an ultrasound image information set from the plurality of pixel output values for each of the scan frequencies. The method may further include the step (433) of combining the image information sets corresponding to each of the scan frequencies to generate a combined image information set. The method may further include converting (434) each pixel output value to a grayscale value. The method may further include providing (435) the grayscale values as a combined image information set. The method may further include the step (436) of making a biometric determination using the plurality of sets of ultrasound image information. The method may further include providing (437) a biological output signal indicative of the biological determination.

  While this disclosure has been described with respect to one or more specific embodiments, it will be understood that other embodiments of this disclosure may be implemented without departing from the spirit and scope of this disclosure. . Accordingly, the present disclosure is deemed limited only by the appended claims and the reasonable interpretation thereof.

22 Amplifier
32 Amplifier
51 Sensor array
52 Bandpass filter
53 Sensor controller
54 Electronic subsystem
55 Ultrasonic transmitter
56 Amplifier
57 Signal generator
60 system
61 objects
62 Sensor stack
63 TFT substrate
64 pixel output data
65 Transmitter
66 Controller
67 processor
70 system
71 objects
72 platen
73 Ultrasonic sensor array
74 TFT substrate
75 Ultrasonic receiver
76 Ultrasonic transmitter
77 Controller
78 processor
241 TFT substrate
242 Display / Cover
243 Sensor array
245 Receiver bias
246 Receiver electrode
247 receiver
248 Transmitter electrode
249 Transmitter electrode
250 transmitter
261 Display cover glass
262 Display color filter glass
263 Display TFT substrate
264 fingerprint sensor
271 Ultrasonic transmitter
272 Ultrasonic Sensor Pixel Circuit Array
273 Digitizer
274 Receiver bias driver
275 gate driver
276 Transmitter Driver
277 Demultiplexer
278 Sensor controller
279 Control unit
280 data processor
420 Optimal frequency
421 other frequencies
426 Separation curve of uplift and valley

Claims (30)

A method of scanning a finger,
Scanning a finger placed on the imaging surface of the ultrasonic sensor with a plurality of ultrasonic scan frequencies;
Generating an ultrasound image information set from a plurality of pixels of the ultrasound sensor for each of the scan frequencies, the image information set including pixel output values from each of the plurality of pixels; Each pixel output value indicates the amount of energy reflected from the imaging surface; and
Combining the image information sets corresponding to each of the scan frequencies to generate a combined image information set, wherein the combined image information set is combined from each of the plurality of pixels; Including a pixel output value. Converting each pixel output value to a grayscale value;
2. The method of claim 1, further comprising: providing the grayscale values for the plurality of pixels as the combined image information set representing a fingerprint of the finger. Performing a biological determination using the plurality of ultrasound image information sets;
Providing a biogenic output signal indicative of the biogenic determination.   Combining the image information set includes adding the pixel output values to generate a sum, and dividing the sum by the number of ultrasound scan frequencies to generate an average value for each of the pixel output values. The method of claim 1, comprising: and using the average value as the synthesized pixel output value. Synthesizing the image information set comprises:
Identifying a weighting factor for each scan frequency;
Multiplying each pixel output value by the corresponding weighting factor to produce a product of pixel output values;
Adding a product of the pixel output values to generate a sum; dividing the sum by a number of scan frequencies to generate an average value for each of the pixel output values; and combining the average values. Using the output as a pixel output value. The weighting factor is
w (f _i ) = (e ^{(avgi * fi)-e (avgi * fmax)} ) / (e ^{(avgi * fmin)-e (avgi * fmax)} )
Is calculated using the formula
here,
w (f _i ) is the weighting factor for the i th scan frequency,
avg _i is the average value of the pixel output values at the i-th scan frequency and the next lower scan frequency,
f _min is the lowest scan frequency,
f _max is the highest scan frequency,
6. The method according to claim 5.   A method of synthesizing the image information set includes creating a covariance matrix for each of the scan frequencies from the pixel output values in the image information set, and synthesizing the covariance matrix to output each pixel output value. Providing a synthesized matrix having synthesized values for.   8. The method of claim 7, wherein synthesizing the covariance matrix includes interpolating between components in the covariance matrix. Identifying a weighting factor for each scan frequency;
9. The method of claim 8, further comprising multiplying each component in the covariance matrix by the corresponding weighting factor before combining the covariance matrix. The weighting factor is
w (f _i ) = (e ^{(avgi * fi)-e (avgi * fmax)} ) / (e ^{(avgi * fmin)-e (avgi * fmax)} )
Is calculated using the formula
here,
w (f _i ) is the weighting factor for the i th scan frequency,
avg _i is the average value of the pixel output values at the i-th scan frequency and the next lower scan frequency,
f _min is the lowest scan frequency,
f _max is the highest scan frequency,
The method of claim 9. The plurality of scan frequencies are
Scanning at a plurality of ultrasonic test frequencies with no finger on the imaging surface of the ultrasonic sensor,
The method of claim 1, wherein the method is selected by identifying a peak test frequency, wherein the peak test frequency is a test frequency at which the test frequency immediately below and the test frequency immediately above returns less energy than the peak test frequency. .   Selecting a peak test frequency, each selected peak test frequency having a reflected energy higher than a majority of the other test frequencies, and scanning the selected peak test frequency with the plurality of scans. 12. The method of claim 11, further comprising using as a frequency. Evaluating the image quality of the peak test frequency;
Selecting a peak test frequency, each selected peak test frequency having better image quality than the other peak test frequencies; and selecting the selected peak test frequencies to the plurality of scan frequencies. 12. The method of claim 11, further comprising using as a step. Selecting one of the peak test frequencies;
12. The method of claim 11, further comprising using the selected peak test frequency as one of the plurality of scan frequencies.   15. The method of claim 14, further comprising identifying additional scan frequencies of the plurality of scan frequencies by adding or subtracting a predetermined offset to the selected one of the peak test frequencies. Identify a range including the selected one of the peak test frequencies;
15. The method of claim 14, further comprising identifying an additional scan frequency of the plurality of scan frequencies by selecting the scan frequency to be within the identified range.   15. The method of claim 14, further comprising identifying additional scan frequencies of the plurality of scan frequencies by identifying harmonics of the selected peak test frequency.   4. The method of claim 3, wherein generating the ultrasound information set further comprises generating a biological vector for each pixel. A system for scanning a finger,
An imaging surface configured to accept a finger;
A planar ultrasound transmitter for generating one or more planar ultrasound waves directed to the imaging surface in response to a plurality of electrical signals;
A transmitter driver amplifier configured to receive an electrical signal from the signal generator and use the electrical signal to drive the ultrasound transmitter;
An ultrasonic sensor array configured to receive one or more reflected ultrasonic signals from the imaging surface;
An electronic subsystem for generating and synthesizing the image information set corresponding to the one or more reflected ultrasound signals at each frequency of interest.   20. The system of claim 19, further comprising a signal generator configured to transmit a plurality of different discrete frequency electrical signals within an ultrasound frequency range to the planar ultrasound transmitter.   20. The system of claim 19, further comprising a signal generator configured to transmit electrical signals at a plurality of varying frequencies within an ultrasound frequency range to the planar ultrasound transmitter.   20. The system of claim 19, further comprising a set of bandpass filters for separating the one or more received ultrasound waves into frequency components.   23. The system of claim 22, wherein the information set collected by the system is heuristically synthesized to produce an output image.   24. The system of claim 23, wherein the information set collected by the system is probabilistically synthesized using a Neiman-Pearson multimodal fusion system to generate an output image. A non-transitory computer readable medium storing computer executable code, the executable code comprising:
Instructions for scanning a finger placed on the imaging surface of the ultrasonic sensor with a plurality of ultrasonic scan frequencies;
Instructions for generating an ultrasound image information set from a plurality of pixels of the ultrasound sensor for each of the scan frequencies, wherein the image information set includes a pixel output value from each of the plurality of pixels. Instructions, each pixel output value indicating the amount of energy reflected from the imaging surface;
An instruction for combining the image information sets corresponding to each of the scan frequencies to generate a combined image information set, wherein the combined image information set is combined from each of the plurality of pixels. A non-transitory computer readable medium comprising instructions including a programmed pixel output value. Instructions for converting each pixel output value to a grayscale value;
26. The method of claim 25, further comprising instructions for providing the grayscale values for the plurality of pixels as the composite image information set representing the fingerprint of the finger. Instructions for making a biological determination using the plurality of ultrasound image information sets;
26. The method of claim 25, further comprising instructions for providing a biological output signal indicative of the biological determination. A system for scanning a finger,
Means for generating one or more planar ultrasound waves (`` MFG '') in response to a signal generator capable of generating electrical signals of different discrete frequencies within the ultrasonic frequency range;
Means for driving the MFG in response to an electrical signal from the signal generator;
A means for touching the finger and reflecting ultrasonic waves from the MFG as an ultrasonic signal to an ultrasonic sensor array means, the ultrasonic sensor array means receiving the reflected ultrasonic signal Means configured to, and
Forming a finger image information set for each received reflected ultrasound signal at each frequency of interest, and means for synthesizing the formed image information set.   30. The system of claim 28, further comprising means for separating the one or more received ultrasound signals into frequency components.   The means for synthesizing the formed image information set is configured to heuristically generate an output image or to generate an output image stochastically using Neiman-Pearson multimodal fusion 30. The system of claim 29, wherein:

Images