EP4734834A1

EP4734834A1 - Optical sensors and audio device for determining biometric information

Info

Publication number: EP4734834A1
Application number: EP23742551.7A
Authority: EP
Inventors: Dongeek Shin; Jonathan Hsu; Anupam J. Pathak
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2023-06-28
Filing date: 2023-06-28
Publication date: 2026-05-06
Also published as: WO2025005911A1

Abstract

A wearable computing device includes one or more optical sensors, an audio device, one or more memories configured to store one or more instructions, and one or more processors configured to execute the one or more instructions stored in the one or more memories to obtain, via the one or more optical sensors, first heart rate information associated with a user wearing the wearable computing device, obtain, via the audio device, second heart rate information associated with the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information.

Description

OPTICAL SENSORS AND AUDIO DEVICE FOR DETERMINING BIOMETRIC

INFORMATION

FIELD

[0001] The disclosure relates generally to computing devices. More particularly, the disclosure relates to wearable computing devices which are used to obtain and store biometric information of a user.

BACKGROUND

[0002] Some existing methods for determining a pulse transit time metric or blood pressure value can require separate devices (e.g., a smartwatch and a smartphone) to take separate biometric measurements at different locations of the body. Some existing methods also require active user interaction with biometric devices to actively take biometric measurements. Other methods also obtain biometric information for determining a pulse transit time metric at sporadic times, according to an invocation by a user for example, rather than taking continuous and/or passive biometric measurements.

SUMMARY

[0004] Aspects and advantages of embodiments of the disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the example embodiments.

[0005] In an example embodiment, a wearable computing device (e.g., a biometric computing device, a smartwatch, a tracker, wearable jewelry having biometric measurement capabilities, and the like) is provided. The wearable computing device includes one or more memories configured to store one or more instructions; and one or more processors configured to execute the one or more instructions stored in the one or more memories to: obtain, via the one or more optical sensors, first heart rate information associated with a user wearing the wearable computing device, obtain, via the audio device, second heart rate information associated w ith the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information

[0006] In some implementations, the audio device includes a transmitter configured to emit a sound wave and a receiver configured to receive a reflected sound wave that corresponds to a sound wave reflected off a first location on the user.

[0007] In some implementations, the transmitter emits an ultrasonic sound wave.

[0008] In some implementations, the one or more optical sensors include one or more photoplethysmography (PPG) sensors, the one or more PPG sensors including a light source configured to emit a light and a photodetector configured to receive reflected light that reflected off a body part of the user.

[0009] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: generate a PPG signal to obtain the first heart rate information based on the reflected light, generate a ballistocardiogram (BCG) signal to obtain the second heart rate information based on the reflected sound wave, and determine the biometric information associated with the user based on a first peak associated with the PPG signal and a second peak associated with the BCG signal.

[0010] In some implementations, the biometric information includes at least one of a pulse transit time, a blood pressure value, or a heart rate.

[0011] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: implement one or more machine-learned models to determine whether a ballistocardiogram (BCG) signal generated based on the reflected sound wave is indicative of an expected BCG signal associated with the first location on the user, when a confidence level associated with an output of the one or more machine-learned models is less than a threshold confidence level, exclude the BCG signal as a measurement associated with the second heart rate information for determining the biometric information, and when the confidence level associated with the output of the one or more machine-learned models is greater than the threshold confidence level, include the BCG signal as a measurement associated with the second heart rate information for determining the biometric information.

[0012] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: determine a distance between the wearable computing device and a body part associated with measuring the second heart rate information, when the distance exceeds a threshold distance value, disable the audio device from obtaining the second heart rate information, and when the distance does not exceed the threshold distance value, enable the audio device to obtain the second heart rate information.

[0013] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: when the distance exceeds a threshold distance value, disable the one or more optical sensors from obtaining the first heart rate information, and when the distance does not exceed the threshold distance value, enable the one or more optical sensors to obtain the first heart rate information.

[0014] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: determine a distance between the wearable computing device and a body part associated with measuring the second heart rate information, when the distance exceeds a threshold distance value, exclude the measurement associated with the second heart rate information for determining the biometric information, and when the distance does not exceed the threshold distance value, include the measurement associated with the second heart rate information for determining the biometric information.

[0015] In some implementations, the wearable computing device further includes a motion sensor. In some implementations, the one or more processors are configured to: determine motion information associated with the wearable computing device via the motion sensor, for a first duration of time in which the motion information associated with the wearable computing device exceeds a threshold motion value while the second heart rate information associated with the user is obtained, exclude the second heart rate information corresponding to the first duration of time as second heart rate information for determining the biometric information, and for a second duration of time in which the motion information associated with the wearable computing device does not exceed the threshold motion value while the second heart rate information associated with the user is obtained, include the second heart rate information corresponding to the second duration of time as second heart rate information for determining the biometric information.

[0016] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: determine whether the motion information associated with the wearable computing device exceeds a threshold motion value, when the motion information exceeds the threshold motion value, disable the audio device from obtaining the second heart rate information, and when the motion information does not exceed the threshold motion value, enable the audio device to obtain the second heart rate information.

[0017] In some implementations, the one or more processors are configured to obtain, via the one or more optical sensors, the first heart rate information while the second heart rate information is being obtained via the audio device.

[0018] In some implementations, the one or more processors are configured to activate the audio device to obtain the second heart rate information in response to the one or more processors detecting the user is engaged in a predetermined activity. [0019] In some implementations, the one or more processors are configured to detect the predetermined activity of the user sleeping.

[0020] In some implementations, the wearable computing device includes a smartwatch or a tracker.

[0021] In an example embodiment, a computer-implemented method is provided. The computer-implemented method includes obtaining, via one or more optical sensors of a wearable computing device, first heart rate information associated with a user wearing the wearable computing device; obtaining, via an audio device of the wearable computing device, second heart rate information associated with the user; and determining biometric information associated with the user based on the first heart rate information and the second heart rate information.

[0022] In some implementations, the first heart rate information is passively obtained via the one or more optical sensors, and the second heart rate information is passively obtained via the audio device.

[0023] In some implementations, obtaining, via the one or more optical sensors of the wearable computing device, the first heart rate information, includes emitting light and detecting reflected light.

[0024] In some implementations, obtaining, via the audio device of the wearable computing device, the second heart rate information, includes transmitting an audio signal and receiving an audio reflection.

[0025] In some implementations, determining the biometric information associated with the user based on the first heart rate information and the second heart rate information comprises: generating a photoplethysmography (PPG) signal based on the reflected light, generating a ballistocardiogram (BCG) signal based on the audio reflection, and determining an interval of time between a first peak associated with the PPG signal and a second peak associated with the BCG signal to determine a pulse transit time.

[0026] The computer-implemented method may include further operations to execute other aspects and operations of the wearable computing device as described herein.

[0027] In an example embodiment, a non-transitory computer-readable medium which stores instructions that are executable by one or more processors of a computing device is provided. The non-transitory computer-readable medium stores instructions which are executable by one or more processors of the computing device. The instructions include: instructions to cause the one or more processors to: obtain, via one or more optical sensors of a wearable computing device, first heart rate information associated with a user wearing the wearable computing device; obtain, via an audio device of the wearable computing device, second heart rate information associated with the user; and determine biometric information associated with the user based on the first heart rate information and the second heart rate information.

[0028] The non-transitory computer-readable medium may store additional instructions to execute other aspects and operations of the wearable computing device and computer- implemented method as described herein.

[0029] These and other features, aspects, and advantages of various embodiments of the disclosure will become better understood with reference to the following description, drawings, and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples of the disclosure and, together with the descnption, serve to explain the related principles.

BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Detailed discussion of example embodiments directed to one of ordinary skill in the art is set forth in the specification, which makes reference to the appended drawings, in which:

[0031] FIG. 1 is an example system including block diagrams of a wearable computing device, a server computing system, and an external computing device, according to one or more examples of the disclosure;

[0032] FIG. 2 is an example illustration of a wearable computing device, according to one or more examples of the disclosure;

[0033] FIG. 3 is an example block diagram of a biometric measurement application, according to one or more examples of the disclosure;

[0034] FIG. 4 is an example illustration of a process flow for determining a ballistocardiogram signal, according to one or more examples of the disclosure;

[0035] FIG. 5 is an example graph illustrating data collected from a photoplethysmogram measurement and a ballistocardiogram measurement, according to one or more examples of the disclosure;

[0036] FIG. 6 is an example graph illustrating data collected from an audio device, according to one or more examples of the disclosure; and

[0037] FIG. 7 is a flow diagram of an example, non-limiting computer-implemented method according to one or more examples of the disclosure. DETAILED DESCRIPTION

[0038] Reference now will be made to embodiments of the disclosure, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the disclosure and is not intended to limit the disclosure. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made to the disclosure without departing from the scope or spirit of the disclosure. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the disclosure covers such modifications and variations as come within the scope of the appended claims and their equivalents.

[0039] Terms used herein are used to describe the example embodiments and are not intended to limit and / or restrict the disclosure. The singular forms “a,” “an” and “the” are intended to include the plural fonns as well, unless the context clearly indicates otherwise. In this disclosure, terms such as "including", "having", “comprising”, and the like are used to specify features, numbers, steps, operations, elements, components, or combinations thereof, but do not preclude the presence or addition of one or more of the features, elements, steps, operations, elements, components, or combinations thereof.

[0040] It will be understood that, although the terms first, second, third, etc., may be used herein to describe various elements, the elements are not limited by these terms.

Instead, these terms are used to distinguish one element from another element. For example, without departing from the scope of the disclosure, a first element may be termed as a second element, and a second element may be termed as a first element.

[0041] The term "and / or" includes a combination of a plurality of related listed items or any item of the plurality of related listed items. For example, the scope of the expression or phrase "A and/or B" includes the item "A", the item "B", and the combination of items "A and B”.

[0042] In addition, the scope of the expression or phrase "at least one of A or B" is intended to include all of the following: (1) at least one of A, (2) at least one of B, and (3) at least one of A and at least one of B. Likewise, the scope of the expression or phrase "at least one of A, B, or C" is intended to include all of the following: (1) at least one of A, (2) at least one of B, (3) at least one of C, (4) at least one of A and at least one of B, (5) at least one of A and at least one of C, (6) at least one of B and at least one of C, and (7) at least one of A, at least one of B, and at least one of C.

[0043] Example aspects of the disclosure are directed to a computing device, for example, a wearable computing device (e.g., a smartwatch, a tracker, etc.), that can be used to capture biometric information associated with a user. The biometric information can include, for example, a heart rate, a blood pressure, a pulse transmit time, and the like, which may be used for lifestyle management, health monitoring, and as an indicator for overall cardiovascular risk and/or health.

[0044] According to example embodiments of the disclosure, a wearable computing device includes a biometric measurement application which can be executed to record biometric information (e.g., in the form of raw data) and also to determine distance information associated with a distance between the wearable computing device and a body part of the user wearing the wearable computing device. The biometric information may include, or be based on, a ballistocardiogram (BCG) signal, an optical photoplethysmography (PPG) signal, a raw accelerometer signal, and the like. The biometric information may include a pulse transit time, a blood pressure value, a heart rate, and the like.

[0045] In some implementations, the wearable computing device may be configured to obtain the PPG signal via one or more optical sensors In some implementations, the wearable computing device may be configured to obtain the BCG signal via an audio device including one or more transmitters (e.g., speakers) and one or more receivers (e.g., one or more microphones).

[0046] For example, the wearable computing device may be configured to passively measure biometric information (e.g., a BCG signal, a PPG signal, etc.) while the user is engaged in a predetermined activity. For example, the predetermined activity may include sleeping or remaining stationary for a predetermined duration of time (e.g., w hile sitting in a chair at work, while sitting in a recliner while watching a movie, etc.).

[0047] For example, the wearable computing device may be configured to automatically activate the one or more optical (PPG) sensors to obtain first heart rate information (e.g., a heart rate of a user or PPG signal) and/or to automatically activate the audio device to obtain the second heart rate information (e.g., a heart rate of a user or BCG signal), in response to the wearable computing device detecting the user is engaged in a predetermined activity (e.g., the user sleeping or the user remaining stationary for a predetermined duration of time). The first heart rate information and the second heart rate information may be obtained simultaneously (e.g., the first heart rate information is obtained via the one or more optical sensors while the second heart rate information is being obtained via the audio device).

[0048] Generally, a BCG signal corresponds to a mechanical measurement of the ballistic forces of each heartbeat. Previous methods utilize a biometric computing device like a smartphone or a watch with a sensitive motion sensor such as an inertial measurement unit (IMU) that is placed near the chest of a subject who remains still to measure the BCG signal. However, this measurement method requires user attention and thus is not a passive measurement. In contrast, according to examples of the disclosure, one or more processors may be configured to enable an output device (e.g., an audio device with an existing speaker and microphone) to obtain remote BCG measurements based on a sound wave (e.g., an ultrasonic sound wave outside the range of hearing of a human) that is reflected off of a user’s body (e.g., the user’s chest), for example, while a user is sleeping.

[0049] In some implementations, the wearable computing device may implement a continuous optical PPG heart rate algorithm via one or more PPG sensors to measure the heart rate of the user based on the optical PPG signal. For example, the one or more PPG sensors may be located on a rear side of the wearable computing device to come into contact with the user’s skin when the wearable computing device is worn by the user.

[0050] In some implementations, the one or more PPG sensors may be enabled or activated after the wearable computing device determines the user’s movement is not excessive and/or that the wearable computing device is within a predetermined distance (e.g., less than 0.5 meters) from a body part (e.g., the chest) of the user. In some implementations, the one or more PPG sensors may be enabled or activated after or in response to the wearable computing device determining the audio device has begun obtaining the BCG signal.

[0051] According to examples of the disclosure, a BCG signal may be measured via the audio device using existing components which are also used for other tasks (e.g., a speaker which provides audible information to a user or is used for voice conversations and a microphone which may be used for voice conversations, as a voice assistant, etc.). The BCG signal may be subjected to noise and instability. According to examples of the disclosure, various processes may be implemented to obtain an accurate BCG signal.

[0052] For example, the wearable computing device may be worn on a first body part (e.g., an arm or wrist) of the user that is relatively close to a second body part (e.g., the chest) of the user. If the wearable computing device is located too far from the second body part (e.g., more than a predetermined distance such as 0.5 meters), BCG measurements obtained via the audio device may be excluded for purposes of determining biometric information associated with the user.

[00531 For example, the audio device may utilize an ultrasonic band of the speaker to transmit inaudible sound waves toward or onto the second body part (e g., the chest). As an example, for a 96 kHz speaker digitization rate, the audio device may emit an ultrasonic transmission in the band of 40 kHz to 45 kHz. For example, the transmission frequency may correspond to a frequency that is not audible to a human and which is upper bounded by the Nyquist rate associated with the speaker.

[0054] According to examples of the disclosure, the microphone is configured to receive transmission waveforms which are reflected from the second body part (e g., the chest). The reflected waveform includes ballistocardiographic information that reflect subtle movements of the human body caused by the heart’s pumping action. The microphone may be configured to record the reflected sound waves in a digital signal format for further processing (e.g., filtering out unwanted noise and amplifying the BCG signal).

[0055] For example, the wearable computing device may be configured to perform a pulse matched filtering method to compare the received reflected waveform with the transmitted waveform (which may be stored in a memory device of the wearable computing device such as a flash memory) to mitigate all other sound sources in the environment and avoid false positives which may be caused by the presence of noise, interference, or other distortions.

[0056] In some implementations, the wearable computing device may be configured to perform the comparison through a correlation approach by which the wearable computing device makes a binary decision regarding whether the transmitted waveform has reflected off the user and is now recorded by the microphone. For example, if the correlation score is above a threshold correlation value, the wearable computing device may be configured to determine that the transmitted waveform has reflected off the user and is now recorded by the microphone, and if the correlation score is below the threshold correlation value, the wearable computing device may be configured to determine that the transmitted waveform has not reflected off the user or that the reflected waveform has been distorted too greatly to be useful for determining an accurate BCG signal.

[0057] The audio device may be configured to perform the above-described transmit and receive process over many periods to create an ultrasonic pulse train over time. For each period, the wearable computing device may be configured to determine a correlation and can stack up a plurality of detections over time at a common maximum point. This refers to finding the maximum correlation value or peak that occurs consistently across the plurality of periods and indicates a significant reflection that persists over time. This operation (the process of combining the plurality of detections at the common maximum point over time) may be referred to as a waterfall concatenation and involves aligning and stacking the detections to create a concatenated or merged representation of the reflected waveforms. This technique may enhance the detection and analysis of persistent reflections and reduce the impact of noise or unwanted signals. The wearable computing device may be configured to analyze the waterfall to determine information about a distance to the user as the time of flight of the pulse normalized by the speed of sound carries information about the distance between the wearable computing device and the second body part (e.g., the chest) and the subtle ultrasonic phase shifts carried at that distance value. This phase shift carries micrometer level information about movement of the second body part. For example, a very subtle periodic change in the amplitude of the received reflected waveform may represent or correspond to a BCG signal.

[0058] The wearable computing device may be configured to perform various postprocessing operations such as range gating and/or motion gating to enhance signal / phase fidelity.

[0059] The wearable computing device may be configured to determine biometric information associated with the user based on the obtained PPG signal and the BCG signal. For example, in some implementations, the wearable computing device may be configured to enable the one or more optical sensors to obtain the PPG signal in response to obtaining the BCG signal. The wearable computing device may be configured to time-synch the PPG signal and BCG signal to (passively) obtain continuous heart rate information (e.g., while the user is sleeping).

[0060] The wearable computing device may be configured to account for movement of the wearable computing device and/or user which may interfere with or distort the PPG signal and/or BCG signal. For example, the wearable computing device may include one or more motion sensors to determine whether the motion information associated with the wearable computing device exceeds a threshold motion value. [0061] In some implementations, the wearable computing device may be configured to, when the motion information exceeds the threshold motion value, disable the one or more optical sensors from obtaining the first heart rate information (PPG signal) and/or disable the audio device from obtaining the second heart rate information (BCG signal), and when the motion information does not exceed the threshold motion value, enable the one or more optical sensors to obtain the first heart rate information and/or enable the audio device to obtain the second heart rate information.

[0062] In some implementations, the wearable computing device may be configured to exclude or omit biometric information associated with the PPG signal and/or BCG signal when the motion information indicates a degree of movement which is greater than a threshold motion value. For example, the wearable computing device may implement one or more machine-learned models (e.g., a neural network such as a custom convolutional neural network) trained on the inertial measurement unit (e.g., the accelerometer and/or gyroscope) of the wearable computing device to perform a binary motion detection classification which may exclude or omit portions (e.g., certain durations of time) of the PPG signal and/or BCG signal collected over time due to motion artifacts, from being used for determining the biometric information.

[0063] The wearable computing device may be configured to, for each time period, compute a time delay between a representative BCG signal peak to a representative PPG signal peak to get a single pulse transit time (PTT) measurement. The wearable computing device may be configured to repeat this process for all pulse periods (with the exception of those which are excluded due to motion artifacts or which are taken at too great a distance). The wearable computing device may be configured to create a histogram feature based on the resulting PTT measurements, and to train a machine-learned model (e.g., a fully connected neural network) that maps the PTT histogram data to a blood pressure value. That is, the machine-learned model takes the PTT histogram data as an input and maps the input PTT histogram data to corresponding blood pressure values based on various internal parameters to minimize a loss function that reflects the difference between a predicted blood pressure value and an actual blood pressure value.

[0064] In some implementations, the wearable computing device may be configured to implement various machine-learned models to perform certain aspects of the disclosure. For example, the wearable computing device may be configured to implement one or more machine-learned models to determine whether a ballistocardiogram (BCG) signal generated based on the reflected sound wave is indicative of an expected BCG signal associated with the second body part of the user, when a confidence level associated with an output of the one or more machine-learned models is less than a threshold confidence level, exclude the BCG signal as a measurement associated with the second heart rate information for determining the biometric information, and when the confidence level associated with the output of the one or more machine-learned models is greater than the threshold confidence level, include the BCG signal as a measurement associated with the second heart rate information for determining the biometric information. For example, ground truth data associated with a BCG signal may be stored in a database and referenced for training the one or more machined learned models to recognize or assess whether a reflected waveform corresponds to a BCG signal. For example, ground truth BCG signal data may be collected by simulating the oscillatory BCG response for known heart rates (e.g., by placing a sonar device near the chest of the user (e.g., less than 20 cm) and measuring data).

[0065] Example aspects of the disclosure provide several technical effects, benefits, and/or improvements in computing technology and the technology of computing devices and health monitoring devices. For example, according to one or more examples of the disclosure, existing components of a wearable computing device (e.g., one or more PPG sensors, a speaker, a microphone, an inertial measurement unit, etc.), may be used to obtain a reliable PPG signal and BCG signal to obtain accurate biometric measurements (e.g., a pulse transit time metric) which can be collected and recorded in an accurate and passive manner over a continuous period of time. Therefore, specialized equipment is not needed to determine biometric information associated with the user.

[0066] For example, computer resources and power savings may be achieved by the selective activation of various components of the wearable computing device. For example, activation of the one or more PPG sensors may be delayed until a BCG signal is collected by the audio device. For example, activation of the one or more PPG sensors and/or the output device may be delayed until a motion threshold value is satisfied and/or until a distance value threshold (relating to a distance between a certain body part of the user and the wearable computing device) is satisfied. Therefore, battery usage may be conserved by activating certain components when certain conditions are satisfied.

[0067] Furthermore, according to one or more examples of the disclosure, the wearable computing device can obtain accurate biometric information by utilizing one or more machine-learned models to confidently predict or determine whether a reflected waveform corresponds to a BCG signal which can be used to determine the pulse transit time. In addition, measurements associated with the PPG signal and the BCG signal may be omitted or excluded for determining biometric information associated with the user when those measurements correspond to time periods where motion artifacts are present and/or where a distance between the second body part of the user and wearable computing device exceeds a threshold distance value. Thus, the biometric information may be determined based off of PPG signals and BCG signals which contain reliable data.

[0068] According to one or more examples of the disclosure, a wearable computing device can opportunistically obtain both PPG signal values and BCG signal values at the same time when taking measurements for obtaining biometric information. The wearable computing device can passively obtain accurate biometric information without a conscious effort by the user to activate the one or more PPG sensors or the output device. Therefore, user convenience is improved, and continuous measurements may be obtained, resulting in a greater number of measurements which can further enhance data collection, compared to previous methods.

[0069] Referring now to the drawings, FIG. 1 illustrates an example system including block diagrams of a wearable computing device, a server computing system, and an external computing device, according to one or more examples of the disclosure. FIG. 2 is an example illustration of a wearable computing device which can be used for obtaining biometric information (e g., a heart rate, a pulse transit time metric, a blood pressure value, etc.) associated with a user via a biometric measurement application, according to one or more examples of the disclosure. FIG. 3 is an example block diagram of a biometric measurement application which may be provided to the wearable computing device, according to one or more examples of the disclosure.

[0070] In FIG. 1, the example system 1000 includes a wearable computing device 100, a server computing system 300, and an external computing device 400. For example, the wearable computing device 100, server computing system 300, and external computing device 400 may be connected with one another over a network 200. Any communications interfaces suitable for communicating via the network 200 (such as a network interface card) may be utilized as appropriate or desired by the wearable computing device 100, server computing system 300, and external computing device 400.

[0071] The wearable computing device 100 may include biometric wearable computing devices (e.g., a biometric smartwatch, a tracker, and the like). In example embodiments described herein, the wearable computing device 100 may be any computing device that can measure biometnc information of a user and is intended to be worn by the user. The server computing system 300 may include a server, or a combination of servers (e.g., a web server, application server, etc.) in communication with one another, for example in a distributed fashion. The external computing device 400 may include any computing device including a personal computer, a smartphone, a laptop, a tablet computer, and the like. In example embodiments described herein, the external computing device 400 may be a computing device that can communicate with the wearable computing device 100 to receive biometric information that is measured by the wearable computing device 100. The wearable computing device 100 may be configured to measure various biometrics, including biometrics associated with an ECG, PPG, BCG, heart rate, blood pressure, heart rate recovery, pulse information, BMI, heart rate variability, oxygen saturation, body temperature, sleep quality, physical activities (e.g., number of steps walked), and the like. Further, the wearable computing device 100 may be configured to generate or display information associated with an electrocardiogram, a photoplethysmogram, a ballistocardiogram, heart rate, heart rate recovery, blood pressure, oxygen saturation, respiration rate, body temperature, physical activity, a sleep metric, electrical conductance, and the like.

[0072] Referring to FIG. 2, according to some implementations of the disclosure, in the illustrated overview 2000 the wearable computing device 2100 (e.g., a wearable computing device including a smartwatch, fitness tracker, etc.) may include a body 2190, a wrist strap or band 2192 to secure the wearable computing device 2100 to a body part of the user, and a display device 2150 configured to present various information (e.g., biometric information) via a graphical user interface 2152. The wearable computing device 2100 may include one or more PPG sensors (e.g., located on a rear side of the wearable computing device 2100 as indicated by the dashed circle 2184). For example, the one or more PPG sensors may naturally contact a body part of the user (e g., a wrist or forearm) when the wearable computing device 2100 is worn by the user, such that PPG measurements may be passively taken without a conscious effort of the user. For example, the one or more PPG sensors may be configured to emit light (represented by the arrow 2200), onto the skin of the user and to measure variations in the intensity of the reflected or transmitted light caused by changes in blood flow. The wearable computing device 2100 may further include an audio device 2162 which includes a speaker and a microphone. As described herein, a BCG signal may be obtained by the speaker emitting a sound wave (represented by arrow 2300) toward a body part of the user (e.g., the chest of the user) and the microphone receiving transmission waveforms which are reflected from the body part (e.g., the chest). The reflected waveform includes ballistocardiographic information that reflect subtle movements of the human body caused by the heart’s pumping action. The microphone may be configured to record the reflected sound waves in a digital signal format for further processing (e.g., filtering out unwanted noise and amplifying the BCG signal). The audio device 2162 may be configured to emit the sound wave (e.g., an ultrasonic sound wave) while the user is asleep or is stationary for a certain duration of time, in a passive manner that does not require interaction from the user or a conscious effort of the user. As described herein, the wearable computing device 2100 may be configured to determine various biometric information based on the obtained PPG and BCG signals (which can be collected simultaneously), including a pulse transit time and a blood pressure value.

[0073] For example, the network 200 may include any type of communications network such as a local area network (LAN), wireless local area network (WLAN), wide area network (WAN), personal area network (PAN), virtual private network (VPN), or the like. For example, wireless communication between elements of the examples described herein may be performed via a wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi direct (WFD), ultra wideband (UWB), infrared data association (IrDA), Bluetooth low energy (BLE), near field communication (NFC), a radio frequency (RF) signal, and the like. For example, wired communication between elements of the examples described herein may be performed via a pair cable, a coaxial cable, an optical fiber cable, an Ethernet cable, and the like. Communication over the network can use a wide variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and/or protection schemes (e.g., VPN, secure HTTP, SSL).

[0074] The wearable computing device 100 may include one or more processors 110, one or more memory devices 120, a biometric measurement application 130, an input device 140, a display device 150, an output device 160, one or more cameras 170, and one or more sensors 180. Each of the components of the wearable computing device 100 may be operatively connected with one another via a system bus. For example, the system bus may be any of several types of bus structures that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and/or a local bus using any of a variety of commercially available bus architectures. [0075] The server computing system 300 may include one or more processors 310, one or more memory devices 320, and a biometric measurement application 330. Each of the features of the server computing system 300 may be operatively connected with one another via a sy stem bus. For example, the system bus may be any of several types of bus structures that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and/or a local bus using any of a variety of commercially available bus architectures.

[0076] The external computing device 400 may include a personal computer, a smartphone, a laptop, a tablet computer, and the like. In example embodiments described herein, the external computing device 400 may be any computing device that can communicate with the wearable computing device 100 to receive biometric information that is measured by the wearable computing device 100. The external computing device 400 can include some or all of the components described with respect to the wearable computing device 100 including the biometric measurement application 130. Therefore, descriptions of these components in the context of the wearable computing device 100 are also applicable to the external computing device 400 and will not be repeated for the sake of brevity. Each of the features of the external computing device 400 may be operatively connected with one another via a system bus. For example, the system bus may be any of several types of bus structures that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and/or a local bus using any of a variety of commercially available bus architectures.

[0077] For example, the one or more processors 110, 310 can be any suitable processing device that can be included in a wearable computing device 100 or server computing system 300. For example, such a processor 110, 310 may include one or more of a processor, processor cores, a controller and an arithmetic logic unit, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an image processor, a microcomputer, a field programmable array, a programmable logic unit, an applicationspecific integrated circuit (ASIC), a microprocessor, a microcontroller, etc., and combinations thereof, including any other device capable of responding to and executing instructions in a defined manner. The one or more processors 110, 310 can be a single processor or a plurality of processors that are operatively connected, for example in parallel.

[0078] The one or more memory devices 120, 320 can include one or more non- transitory computer-readable storage mediums, such as such as a Read Only Memory (ROM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), and flash memory, a USB drive, a volatile memory device such as a Random Access Memory (RAM), an internal or external hard disk drive (HDD), floppy disks, a blueray disk, or optical media such as CD ROM discs and DVDs, and combinations thereof. However, examples of the one or more memory devices 120, 320 are not limited to the above description, and the one or more memory devices 120, 320 may be realized by other various devices and structures as would be understood by those skilled in the art.

[0079] For example, the one or more memory devices 120 can store instructions, that when executed, cause the one or more processors 110 to obtain, via one or more optical sensors, first heart rate information associated with a user, obtain, via an audio device, second heart rate information associated with the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information, as described according to examples of the disclosure.

[0080] For example, the one or more memory devices 320 can store instructions, that when executed, cause the one or more processors 310 to obtain, via one or more optical sensors of the wearable computing device 100, first heart rate information associated with a user, obtain, via an audio device of the wearable computing device 100, second heart rate information associated w ith the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information, as described according to examples of the disclosure.

[0081] The one or more memory devices 120 can also include data 122 and instructions 124 that can be retrieved, manipulated, created, or stored by the one or more processors 110. In some examples, such data can be accessed and used as input to obtain the first heart rate information and the second heart rate information and output the biometric information associated with a user, as descnbed according to examples of the disclosure. The one or more memory devices 320 can also include data 322 and instructions 324 that can be retrieved, manipulated, created, or stored by the one or more processors 310. In some examples, such data can be accessed and used as input to obtain the first heart rate information and the second heart rate information and output the biometric information associated with the user, as described according to examples of the disclosure.

[0082] The biometric measurement application 130 can include any biometnc application which allows or is capable of determining biometric information associated with a user (e.g., based on biometric measurements obtained via the one or more sensors 180 and the audio device 162). As explained with reference to FIG. 3, in some implementations, the biometric measurement application 130 includes a first heart rate information determination application 132, a second heart rate information determination application 134, a biometric information determiner 136, a motion determiner 138, and a distance determiner 139.

[0083] For example, in some implementations the wearable computing device 100 may automatically execute the biometric measurement application 130 to measure, determine, and store a biometric measurement (e.g., a pulse transit time and/or blood pressure value that is obtained based on data associated with a PPG signal obtained via the one or more optical sensors 184 and a BCG signal obtained via the audio device 162). For example, the wearable computing device 100 may automatically execute the biometric measurement application 130 in response to determining the user is asleep, in response to determining the user has been stationary for a predetermined duration of time (e.g., 30 seconds), or in response to detecting another circumstance where the wearable computing device 100 is likely able to obtain PPG signals and BCG signals of sufficient quality and duration for measuring biometric information (e.g., a pulse transit time and/or blood pressure value). For example, the wearable computing device 100 may automatically execute the biometric measurement application 130 at a predetermined time which may be set as a default time, set by the user, or determined by the wearable computing device 100 as a time which is probabilistically likely to be successful for PPG signals and BCG signals of sufficient quality and duration for measuring biometric information (e.g., a pulse transit time and/or blood pressure value).

[0084] For example, in some implementations the wearable computing device 100 may be configured to receive an input to the wearable computing device 100 via input device 140 to execute the biometric measurement application 130 to measure, determine, and store a biometric measurement. For example, the user may be prompted to execute the biometric measurement application 130 at a predetermined time of day (e g., at a time corresponding to a normal bedtime associated with the user), or in response to the wearable computing device 100 determining the user has been stationary for a predetermined duration of time.

[0085] The biometric measurement application 330 of the server computing system 300 can also include similar features as the biometric measurement application 130 (e.g., as shown in FIG. 3) which perform similar functions and operations, and therefore a description of those features will not be repeated for the sake of brevity. [0086] The wearable computing device 100 may include an input device 140 configured to receive an input from a user and may include, for example, one or more of a keyboard (e.g., a physical keyboard, virtual keyboard, etc.), a mouse, a joystick, a button, a switch, an electronic pen or stylus, a gesture recognition sensor (e.g., to recognize gestures of a user including movements of a body part), an input sound device or voice recognition sensor (e.g., a microphone to receive a voice command), a track ball, a remote controller, a portable (e.g., a cellular or smart) phone, and so on. The input device 140 may also be embodied by a touch-sensitive display device having a touchscreen capability, for example. The input device 140 may be used by the user of the wearable computing device 100 to provide an input to execute the biometric measurement application 130, to provide information about the user (e.g., biometric information, demographic information, user preferences, etc.). The input device 140 may be used by the user of the wearable computing device 100 to request a biometric measurement, to transmit biometric information of the user to the server computing system 300, external computing device 400, user biometric information data store 350, etc. For example, the input may be a voice input, a touch input, a gesture input, a click via a mouse or remote controller, and so on.

[0087] The wearable computing device 100 may include a display device 150 which presents information viewable by the user, for example on a user interface (e.g., a graphical user interface). For example, the display device 150 may be a touch sensitive display or a non-touch sensitive display. The display device 150 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, active matrix organic light emitting diode (AMOLED), flexible display, 3D display, a plasma display panel (PDP), a cathode ray tube (CRT) display, and the like, for example. However, the disclosure is not limited to these example display devices and may include other types of display devices.

[0088] The wearable computing device 100 may include an output device 160 configured to provide an output to the user and may include, for example, one or more of an audio device (e.g., one or more speakers), a haptic device to provide haptic feedback to a user, a tight source (e.g., one or more light sources such as LEDs which provide visual feedback to a user), and the like. In FIG. 1, the output device 160 is further depicted as including an audio device 162 including a transmitter 162a and a receiver 162b. For example, the transmitter 162a may correspond to a speaker and the receiver 162b may correspond to a microphone. How ever, the depiction of the audio device 162 as part of the output device 160 is merely an example, and the audio device 162 could also be part of the input device 140, the one or more sensors 180, or a standalone feature. In addition, it would be understood that the transmitter 162a could be considered as an output device while the receiver 162b could be considered as an input device and/or sensor. Aspects of the audio device 162 in the context of obtaining a BCG signal for determining biometric information associated with a user are described herein.

[0089] The wearable computing device 100 may include one or more cameras 170. For example, the one or more cameras 170 may include an imaging sensor (e.g., a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD)) to capture, detect, or recognize a user's behavior, figure, expression, etc. For example, the one or more cameras 170 may be configured to capture images associated with the user which may be used by the wearable computing device 100 to detect whether a user is sleeping, is stationary or moving, and the like.

[0090] The wearable computing device 100 may include one or more sensors 180. For example, the one or more sensors 180 may include an inertial measurement unit 182 which includes one or more accelerometers 182a and/or one or more gyroscopes 182b. The one or more accelerometers 182a may be used to capture motion information with respect to the wearable computing device 100. The one or more gyroscopes 182b may also be used additionally or alternatively to capture motion information with respect to the wearable computing device 100. For example, the inertial measurement unit 182 may be configured as a six-axis or six-dimensional inertial measurement unit (e.g., a tri-axial accelerometer and a tri-axial gyroscope).

[0091] For example, the one or more sensors 180 may include one or more one or more optical sensors 184 (e.g., one or more photoplethysmography (PPG) sensors) which can also be used to monitor the heart rate of the user. The one or more optical sensors 184 may include one or more emitters (e.g., light-emitting diodes (LEDs)) and one or more detectors (e.g., photodiodes). For example, the one or more optical sensors 184 may be configured to emit light (e.g., green or red) via the one or more emitters, onto the skin of user and to measure variations in the intensity of the reflected or transmitted light via the one or more detectors caused by changes in blood flow. For example, the one or more optical sensors 184 may be configured to capture the pulsatile nature of the blood flow which can be used to estimate various physiological parameters related to the cardiovascular system. For example, the heart rate of the user may be determined based on the frequency of the pulsatile signal. Additionally, the one or more optical sensors 184 may be configured to provide information about heart rate variability (HRV), blood oxygen saturation (SpO2) levels, and the like. Furthermore, in some examples described herein a pulse transit time (PTT) may be determined based on a measurement of the time delay between a representative peak of the BCG signal (e.g., a J-wave peak representing certain movement activity of the heart) obtained via the audio device 162 and a representative peak of the PPG signal (e.g., corresponding to a systolic point) obtained via the one or more optical sensors 184. For example, the one or more optical sensors 184 may be disposed at a side of the wearable computing device 100 (e.g., a rear side) such that the one or more optical sensors 184 are in contact with a body part (e.g., wrist or forearm) of the user. For example, the one or more optical sensors 184 may be disposed such that a heart rate of the user may be passively monitored and measured without a user actively or consciously engaging the one or more optical sensors 184.

[0092] For example, the one or more sensors 180 may include one or more proximity sensors 186 which can be used to detect a distance to an object. For example, the one or more proximity sensors 186 may be configured to determine a distance between the wearable computing device 100 and the chest of the user. For example, the one or more proximity sensors 186 may be disposed on the wearable computing device 100 at one or more locations to ensure a transmitted waveform (e.g., an infrared waveform, ultrasonic waveform, etc.) is likely to reflect off of a user’s chest (e.g., while sleeping, sitting, standing, etc.) so that the one or more proximity sensors 186 receives the reflected waveform to determine the distance between the wearable computing device 100 and the chest of the user.

[0093] The one or more sensors 180 may also include other sensors such as a magnetometer, GPS sensor, ECG sensors, and the like.

[0094] Example system 1000 may include a user biometric information data store 350 and a machine-learned model(s) and training data store 360.

[0095] In some implementations, user biometric information data store 350 can represent a single database. In some implementations, the user biometric information data store 350 represents a plurality of different databases accessible to the wearable computing device 100, server computing system 300, and external computing device 400. In some examples, the user biometric information data store 350 can include biometric information of a user or a plurality of users (e.g., a hospital or medical facility database). In some examples, the user biometric information data store 350 can include information regarding one or more user profiles, including a variety of user data such as user preference data, user demographic data, user calendar data, user social network data, user historical health data, and the like. For example, the user biometric information data store 350 can include any biometric information or information associated with the biometric information (e.g., time information associated with the collection of the biometric information, location information associated with the collection of the biometric information, etc.). The user biometric information and associated information may be associated with a user account. In some implementations described herein, heart rate information of a user may be stored in user biometric information data store 350, including blood pressure information, pulse transit time information, PPG signals, BCG signals, and the like.

[0096] The user biometric information data store 350 is provided to illustrate potential data that could be analyzed or stored, in some embodiments, by the wearable computing device 100 and/or server computing system 300 to maintain a record of biometric information associated with a user, for example. However, such user data may not be collected, used, or analyzed unless the user has consented after being informed of what data is collected and how such data is used. Further, in some embodiments, the user can be provided with a tool (e.g., in a biometric measurement application or via a user account) to revoke or modify the scope of permissions. In addition, certain information or data can be treated in one or more ways before it is stored or used, so that personally identifiable information is removed or stored in an encrypted fashion. Thus, particular user information stored in the user biometric information data store 350 may or may not be accessible to the wearable computing device 100 and/or server computing system 300 based on permissions given by the user, or such data may not be stored in the user biometric information data store 350 at all.

[0097] In some implementations, machine-learned model(s) and training data store 360 can represent a single database. In some implementations, the machine-learned model(s) and training data store 360 represents a plurality of different databases accessible to the wearable computing device 100, server computing system 300, and external computing device 400. In some examples, the machine-learned model(s) and training data store 360 can include one or more machine-learned models which can be used by the biometric measurement application 130 to analyze a raw PPG signal, a raw audio device signal (which corresponds to a BCG signal), and the like. In some examples, the machine-learned model(s) and training data store 3 0 can store training data which is used to train the one or more machine-learned models (e.g., a custom neural network, an unsupervised neural network, etc.).

[0098] For example, the training data may include data which is collected via an audio device receiver of a wearable computing device which is collected from users while they are sleeping or while they are in a stationary position (e.g., standing, sitting) and classified as such. For example, the training data may include a ground truth BCG signal data collected from a BCG measurement device such as a piezoelectric sensor (e.g., a piezo band) which can be placed around the head or chest of the user while they are sleeping, while they are in a stationary position, etc. In some implementations, other BCG measurement devices may be used to obtain a ground truth BCG signal such as a strain gauge, fiberoptic sensor, etc.

[0099] For example, the one or more machme-leamed models stored in the machine- learned model(s) and training data store 360 may include one or more machine-learned models (e.g., a neural network such as a custom convolutional neural network) trained on a an inertial measurement unit (e.g., including an accelerometer and/or gyroscope) of the wearable computing device 100 to perform a binary motion detection classification which may exclude or omit portions (e.g., certain durations of time) of the PPG signal and/or BCG signal collected over time due to motion artifacts, from being used for determining the biometric information.

[0100] For example, the training data may include motion data which is collected via an inertial measurement unit (e.g., including an accelerometer and/or gyroscope) of a wearable computing device. The motion data may be collected from users while they are sleeping or while they are in a stationary position (e.g., standing, sitting) and classified as such.

[0101] For example, the one or more machine-learned models stored in the machine- learned model(s) and training data store 360 may include one or more machine-learned models (e g., a neural network such as a custom convolutional neural network) trained on pulse transit time data or pulse transit time histogram data to predict or determine a blood pressure value associated with a user. For example, the one or more machine-learned models may be configured to take the PTT histogram data as an input and map the input PTT histogram data to corresponding blood pressure values based on various internal parameters to minimize a loss function that reflects the difference between a predicted blood pressure value and an actual blood pressure value.

[0102] For example, the training data may include pulse transit time data which is used to train the one or more machine-learned models for predicting a blood pressure value.

[0103] For example, the one or more machine-learned models stored in the machine- learned model(s) and training data store 360 may include one or more machine-learned models (e.g., a neural network such as a custom convolutional neural network) trained to determine whether a reflected sound wave (e.g., an ultrasonic sound wave) is indicative of a ballistocardiogram (BCG) signal associated with a particular body part of a user (e.g., the chest).

[0104] For example, the training data may include ground truth data associated with an obtained BCG signal (e.g., by placing a sonar device near the chest of the user (e.g., less than 20 cm) and measuring data).

[0105] In some implementations, one or more machine-learned models stored in the machine-learned model(s) and training data store 360 may learn through one or more various machine learning techniques (e g., by training a neural network or other machine-learned model) to determine (e.g., with a specified confidence level, with a probability above a threshold level, etc.), whether a BCG signal (corresponding to the reflected waveform received by the receiver 162b of the audio device 162 is of a sufficient quality for biometric information to be measured based on the BCG signal, or is deficient and should be discarded or ignored. For example, data descriptive of BCG signals which are “clean” signals and satisfy various predetermined conditions can be stored and used as training data to train (e.g., via supervised or unsupervised training techniques) one or more machine-learned models to, after training, generate predictions which assist in determining whether the BCG signal is of sufficient quality for biometric information to be measured based on the BCG signal, or is deficient and should be discarded or ignored. In such a way, system performance is improved with more accurate and reliable biometric information measurements.

Furthermore, processing, memory, and network resources of a computing system (e.g., a wearable computing device, external computing device, or combinations thereof) are conserved by not measuring biometric information of BCG signals which are deficient. [0106] In some implementations, some or all of the machine-learned models and training data from the machine-learned model(s) and training data store 360 may be stored at any of the wearable computing device 100, server computing system 300, external computing device 400.

[0107] Referring to FIG. 3, an example block diagram of a biometric measurement application is shown, according to one or more examples of the disclosure. FIG. 3 illustrates that the biometric measurement application 130 includes a first heart rate information determination application 132, a second heart rate information determination application 134, a biometric information determiner 136, a motion determiner 138, and a distance determiner 139. However, the biometric measurement application 130 may include fewer or more features than that shown in FIG. 3. For example, any of the features or operations of the components of biometric measurement application 130 may be provided separately from the biometric measurement application 130. For example, some operations (such as the determination of the biometric information by the biometric information determiner 136) may instead be performed by the server computing system 300 (e.g., via biometric measurement application 330).

[0108] Operations of the biometric measurement application 130 will now be described in more detail with reference to FIGS. 3 through 7.

[0109] FIG. 4 is an example illustration of a process flow for determining a ballistocardiogram signal, according to one or more examples of the disclosure. FIG. 5 is an example graph illustrating data collected from a photoplethysmogram measurement and a ballistocardiogram measurement, according to one or more examples of the disclosure. FIG. 6 is an example graph illustrating data collected via an audio device, according to one or more examples of the disclosure. FIG. 7 illustrates an example flow diagram of a nonlimiting computer-implemented method for detemiining biometric information of a user, according to one or more examples of the disclosure.

[0110] The flow diagram of FIG. 7 illustrates a method 7000 for determining biometric information of a user. Although shown in a particular sequence or order, unless otherwise specified, the order of the processes can be modified. Thus, the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processes can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

[0111] In some implementations, the wearable computing device 100 may perform a gating function prior to obtaining heart rate information associated with a user. FIG. 7 illustrates a method 7000 in which two example gating functions at operation 7100 and operation 7200 may be used for determining whether to proceed with taking a biometric measurement of a user (or whether to activate certain sensors or devices), which may help to conserve battery life of the wearable computing device 100 as well as conserve the use of the sensors and devices, thereby extending their expected life cycle. However, method 7000 need not include operation 7100 and operation 7200, or only one of operation 7100 and operation 7200 may be performed. Further, in some implementations operation 7200 may be performed before operation 7100. As indicated in FIG. 7, operations 7100 and 7200 are optional operations. [0112] As described herein, biometric measurement application 130 may be configured to automatically perform or execute some or all of the aspects of the method 7000, in a passive manner without user interaction or conscious effort on the part of the user, for example, during a predetermined activity. The predetermined activity may include the user sleeping or the user remaining stationary for a predetermined duration of time (e.g., while sitting or standing). For example, the biometric measurement application 130 may be configured to automatically activate the one or more optical sensors (one or more optical sensors 184) to obtain the first heart rate information at operation 7300 and/or to automatically activate the audio device to obtain the second heart rate information at operation 7400, in response to the biometric measurement application 130 detecting the user is engaged in the predetermined activity. For example, the biometric measurement application 130 may automatically execute some or all of the aspects of the method 7000 in response to determining the user is asleep (via sensor information associated with the user), in response to determining the user has been stationary for a predetermined duration of time (e.g., 30 seconds), or in response to detecting another circumstance where the wearable computing device 100 is likely able to obtain PPG signals and BCG signals of sufficient quality and duration for measuring biometric information (e.g., a pulse transit time and/or blood pressure value). For example, the biometric measurement application 130 may automatically execute some or all of the aspects of the method 7000 at a predetermined time which may be set as a default time, or at a time determined by the wearable computing device 100 as a time which is probabilistically likely to be successful for PPG signals and BCG signals of sufficient quality and duration for measuring biometric information (e.g., a pulse transit time and/or blood pressure value).

[0113] At operation 7100 the method 7000 can include determining whether motion information associated with the wearable computing device is less than a threshold motion value. For example, the motion determiner 138 may be configured to determine, via one or more motion sensors (e g., via an input received from the inertial measurement unit 182), motion information associated with the wearable computing device 100.

[0114] In some implementations, the motion determiner 138 may be configured to determine whether the motion information associated with the wearable computing device 100 exceeds a threshold motion value. The threshold motion value may correspond to a degree or amount of motion which can cause excess noise or motion artifacts to occur with respect to the PPG signal and/or BCG signal, such that biometric information cannot be reliably obtained from the PPG signal and/or BCG signal. For example, when the motion determiner 138 determines the motion information exceeds the threshold motion value, the wearable computing device 100 (e.g., the one or more processors 110) may be configured to disable the one or more optical sensors 184 (e.g., one or more PPG sensors) from obtaining the first heart rate information (e.g., the PPG signal) and/or may be configured to disable the audio device 162 from obtaining the second heart rate information (e.g., the BCG signal). The motion determiner 138 may also be configured to continue monitoring or detecting motion data associated with the wearable computing device 100 when the motion determiner 138 determines the motion information exceeds the threshold motion value (e.g., until the motion information does not exceed the threshold motion value and operation 7200, operation 7300, or operation 7400 can then be performed).

[0115] For example, when the motion determiner 138 determines the motion information does not exceed the threshold motion value, the wearable computing device 100 (e.g., the one or more processors 110) may be configured to enable the one or more optical sensors 184 (e.g., one or more PPG sensors) to obtain the first heart rate information (e.g., the PPG signal) and/or may be configured to enable the audio device 162 to obtain the second heart rate information (e.g., the BCG signal). As described herein, in some implementations, the first heart information may be obtained before the second heart rate information is obtained. In some implementations, the second heart information may be obtained before the first heart rate information is obtained. In still other implementations, the first heart information may be obtained together with the second heart rate information at the same time. Obtaining the second heart rate information (e.g., the BCG signal) for some predetermined duration of time (e.g., five seconds, ten seconds, etc.) before activating the one or more optical sensors 184 to obtain the first heart information (e.g., the PPG signal) may conserve computing resources including processing and battery resources as it may be more difficult to obtain the second heart rate information. That is, the wearable computing device 100 may be configured to ensure that the second heart rate information is being obtained (e.g., reliably obtained) via the audio device 162 before the one or more optical sensors 184 are activated to obtain the first heart rate information so as to conserve computing resources.

[0116] At operation 7200 the method 7000 can include determining whether a distance between the wearable computing device and a body part associated with the second heart rate information is less than a threshold distance value. For example, the distance determiner 139 may be configured to determine, via one or more distance sensors (e g., via information obtained via the one or more proximity sensors 186, via information obtained from the audio device 162, etc.), distance information associated with a distance between a body part (e.g., the chest) of the user and the wearable computing device 100. For example, in some implementations, the one or more proximity sensors 186 may be disposed on the wearable computing device 100 at one or more locations to ensure a transmitted waveform (e.g., an infrared waveform, ultrasonic waveform, etc.) is likely to reflect off of the user’s chest (e.g., while sleeping, sitting, standing, etc.) so that the one or more proximity sensors 186 receives the reflected waveform to determine the distance between the wearable computing device 100 and the chest of the user. For example, in some implementations, the audio device 162 may be configured to emit a waveform (e.g., an ultrasonic waveform) which reflects off of the user’s chest (e.g., while sleeping, sitting, standing, etc.) so that the audio device 162 receives the reflected waveform. The time of flight of the pulse normalized by the speed of sound includes information about the distance between the wearable computing device 100 and the chest of the user. The distance determiner 139 may be configured to determine the distance based on the time of flight information obtained from the times associated with the transmitted waveform and the reflected waveform.

[0117] In some implementations, the distance determiner 139 may be configured to determine whether a distance between the wearable computing device 100 and a body part associated with the second heart rate information exceeds a threshold distance value. The threshold distance value may correspond to a distance (e.g., 0.5 meters) at which excess noise or motion artifacts occur with respect to the BCG signal, such that biometric information cannot be reliably obtained from the BCG signal. For example, when the distance determiner 139 determines the distance exceeds the threshold distance value, the wearable computing device 100 (e.g., the one or more processors 110) may be configured to disable the one or more optical sensors 184 (e.g., one or more PPG sensors) from obtaining the first heart rate information (e.g., the PPG signal) and/or may be configured to disable the audio device 162 from obtaining the second heart rate information (e g., the BCG signal). The distance determiner 139 may also be configured to continue monitoring or detecting distance data associated with a distance between a body part (e.g., the chest) of the user and the wearable computing device 100 when the distance determiner 139 determines the distance exceeds the threshold distance value (e.g., until the distance does not exceed the threshold distance value and operation 7100, operation 7300, or operation 7400 can then be performed).

[0118] For example, when the distance determiner 139 determines the distance does not exceed the threshold distance value, the wearable computing device 100 (e.g., the one or more processors 110) may be configured to enable the one or more optical sensors 184 (e.g., one or more PPG sensors) to obtain the first heart rate information (e.g., the PPG signal) and/or may be configured to enable the audio device 162 to obtain the second heart rate information (e.g., the BCG signal). As described herein, in some implementations, the first heart information may be obtained before the second heart rate information is obtained. In some implementations, the second heart information may be obtained before the first heart rate information is obtained. In still other implementations, the first heart information may be obtained together with the second heart rate information at the same time. Obtaining the second heart rate information (e.g., the BCG signal) for some predetermined duration of time (e.g., five seconds, ten seconds, etc.) before activating the one or more optical sensors 184 to obtain the first heart information (e.g., the PPG signal) may conserve computing resources including processing and battery resources as it may be more difficult to obtain the second heart rate information. That is, the wearable computing device 100 may be configured to ensure that the second heart rate information is being obtained (e.g., reliably obtained) via the audio device 162 before the one or more optical sensors 184 are activated to obtain the first heart rate information so as to conserve computing resources.

[0119] In some implementations, operation 7300 and/or operation 7400 may be performed in response to the motion determiner 138 determining the motion information does not exceed the threshold motion value and the distance determiner 139 determining the distance between the wearable computing device 100 and the body part associated with the second heart rate information is less than the threshold distance value. However, the disclosure is not limited to this example and other modifications or variations are possible.

[0120] At operation 7300 the method 7000 includes obtaining, via one or more optical sensors of a computing device, first heart rate information associated with a user. For example, the first heart rate information determination application 132 can obtain first heart rate information via the one or more optical sensors 184 (e.g., one or more PPG sensors). For example, the first heart rate information can include a heart rate or a PPG signal which is determined by the PPG signal analyzer 132a. FIG. 5 is an example graph illustrating data collected from a photoplethysmogram measurement, according to one or more examples of the disclosure. For example, the one or more optical sensors 184 may be configured to include a light source (e.g., one or more emitters including one or more light emitting diode) configured to emit light (e g., green or red) via the light source, onto a first body part (e g., a wist, a forearm, etc.) of a user. For example, the one or more optical sensors 184 may be configured to include a photodetector (e.g., a photodiode) configured to receive light reflected from the first body part and measure variations in the intensity of the reflected caused by changes in blood flow to obtain the PPG signal. As shown in FIG. 5, the graph 5000 includes a PPG signal 5100. For example, various known peak detection methods (e.g., peak amplitude thresholding) may be implemented by the PPG signal analyzer 132a to identify peaks (e.g., peaks 5400, 5500) in the PPG signal 5100. The PPG signal analyzer 132a may be configured to determine a time interval (e g., interbeat interval - IBI int) between consecutive peaks which is inversely related to the heart rate. For example, the PPG signal analyzer 132a may be configured to determine the heart rate by taking the reciprocal of the time interval and multiplying it by 60.

[0121] At operation 7400 the method 7000 includes obtaining, via the audio device, second heart rate information associated with the user. For example, the audio device 162 may obtain second heart rate information (e.g., a heart rate, a BCG signal or a signal that corresponds to a BCG signal) associated with the user.

[0122] For example, the second heart rate information determination application 134 can obtain second heart rate information via the audio device 162. For example, the second heart rate information can include a heart rate or BCG signal which is determined by the BCG signal analyzer 134a and/or the one or machine-learned models 134b. FIG. 5 is an example graph illustrating data collected from a BCG measurement, according to one or more examples of the disclosure. As shown in FIG. 5, the graph 5000 includes a BCG signal 5200. The BCG signal 5200 may include a plurality of peaks, including peak 5300 which corresponds to a J-wave peak of a BCG signal. For example, various know n peak detection methods (e g., peak amplitude thresholding) may be implemented by the BCG signal analyzer 134a to identify the peaks in the BCG signal 5200.

[0123] The BCG signal analyzer 134a and/or the one or machine-learned models 134b may be configured to obtain the BCG signal 5200 by processing a reflected waveform received by the receiver 162b of the audio device 162 and input to the second heart rate information determination application 134. FIG. 4 is an example illustration of a process flow for determining a ballistocardiogram signal based on the reflected waveform received by the receiver 162b, according to one or more examples of the disclosure.

[0124] By way of illustration, FIG. 4 depicts a user 4010 sleeping while wearing a wearable computing device 4020 (corresponding to and interchangeably referred to as wearable computing device 100) at a first body part 4030 of the user 4010 (e.g., the wrist or forearm). The transmitter 162a is configured to emit a sound wave (e.g., an ultrasonic waveform) toward a second body part 4040 of the user 4010 (e.g., the chest or back or areas where movements of the heart are expected to be most prominent). For example, the audio device 162 may utilize an ultrasonic band of the transmitter 162a (e.g., a speaker) to transmit inaudible sound waves toward or onto the second body part 4040. As an example, for a 96 kHz speaker digitization rate, the audio device 162 may emit an ultrasonic transmission in the band of 40 kHz to 45 kHz. For example, the transmission frequency may correspond to a frequency that is not audible to a human and which is upper bounded by the Nyquist rate associated with the speaker. An example transmitted ultrasonic sound wave is represented at operation 4100 which depicts a transmitted waveform representing ultrasonic chirping that may be output by the transmitter 162a.

[0125] The receiver 162b is configured to receive a reflected sound wave (e.g., a reflected ultrasonic waveform) that corresponds to the sound wave having reflected off the second body part 4040 of the user 4010.

[0126] At operation 4200 the BCG signal analyzer 134a may be configured to perform a pulse matched filtering method to compare the received reflected waveform with the transmitted waveform (which may be stored in the one or more memory devices 120 of the wearable computing device 100 such as a flash memory) to mitigate other sound sources in the environment and avoid false positives which may be caused by the presence of noise, interference, or other distortions. In some implementations, the BCG signal analyzer 134a may be configured to perform the comparison through a correlation approach by which the BCG signal analyzer 134a makes a binary decision regarding whether the transmitted waveform has reflected off the user and is now recorded by the receiver 162b (e.g., a microphone). For example, if the correlation score is above a threshold correlation value, the BCG signal analyzer 134a may be configured to determine that the transmitted waveform has reflected off the user 4010 and is now recorded by the receiver 162b, and if the correlation score is below the threshold correlation value, the BCG signal analyzer 134a may be configured to determine that the transmitted w aveform has not reflected off the user 4010 or that the reflected waveform has been distorted too greatly to be useful for determining an accurate BCG signal.

[0127] The audio device 162 may be configured to perform the above-described transmit and receive process over many periods to create an ultrasonic pulse train over time. For each period, the BCG signal analyzer 134a may be configured to determine a correlation and can stack up a plurality of detections over time at a common maximum point. This refers to finding the maximum correlation value or peak that occurs consistently across the plurality of periods and indicates a significant reflection that persists over time. Operation 4400 corresponds to the waterfall concatenation process performed by the BCG signal analyzer 134a which involves aligning and stacking (or combining) the plurality of detections at the common maximum point over time to create a concatenated or merged representation of the reflected waveforms. This technique may enhance the detection and analysis of persistent reflections and reduce the impact of noise or unwanted signals.

[0128] The BCG signal analyzer 134a may be configured to analyze the waterfall to perform operation 4500 which involves a range-vitals mapping operation and operation 4600 which involves a range-vitals gating operation. For example, the BCG signal analyzer 134a may be configured to determine information about a distance to the user 4010 as the time of flight of the pulse (e.g., based on stored transmit and receive timing information) normalized by the speed of sound carries information about the distance between the wearable computing device 100 and the second body part 4040 (e.g., the chest) and the subtle ultrasonic phase shifts carried at that distance value. For example, distance determiner 139 may also determine the distance between the wearable computing device 100 and the second body part 4040. The BCG signal analyzer 134a may map a determined distance to a particular measurement at operation 4500. The BCG signal analyzer 134a may further perform a gating operation by excluding or marking measurements which are taken when a distance between the wearable computing device 100 and the second body part 4040 is greater than a threshold distance value, at operation 4600.

[0129] At operation 4700 the BCG signal analyzer 134a may be configured to estimate vitals (biometric information) associated with the user 4010. For example, the subtle ultrasonic phase shifts carry information about the distance between the wearable computing device 100 and the second body part 4040 and reflect distance information about movement of the second body part 4040. For example, the subtle ultrasonic phase shifts may carry micrometer level information about chest movement of the user 4010. For example, a very subtle periodic change in the amplitude of the received reflected waveform may represent or correspond to a BCG signal.

[0130] FIG. 6 is an example graph illustrating data collected from an audio device, according to one or more examples of the disclosure. The graph 6000 shown in FIG. 6 includes amplitude information indicating distance displacement tracking of the second body part 4040 of the user 4010 over time. For example, at point 6100 the distance to the user’s chest is greater than the distance to the user’s chest at point 6200 and a difference between the amplitudes represented by double-arrow 6400 may be used to determine the distance displacement of the user’s chest over an interval of time 6300. Such distance information may be representative of a human BCG signal which reflects the movements of the body resulting from cardiac activity and provides information about the forces generated by the contractions of the heart. The BCG signal analyzer 134a may be configured to perform a phase reconstruction at operation 4800 to output a BCG signal at operation 4900, based on the distance displacement information obtained over time via the reflected sound wave obtained by the receiver 162b.

[0131] The BCG signal analyzer 134a may be configured to perform vanous postprocessing operations such as range gating and/or motion gating (e.g., via motion determiner 138) to enhance BCG signal / phase fidelity.

[0132] For example, one or more machine-learned models 134b may be implemented by the BCG signal analyzer 134a to verify that the output BCG signal is actually representative or indicative of an expected or actual BCG signal associated with the second body part 4040 of the user 4010. For example, when a confidence level associated with an output of the one or more machine-learned models 134b is less than a threshold confidence level, the BCG signal analyzer 134a may be configured to exclude the output BCG signal as a measurement associated with the second heart rate information for determining biometric information associated with the user 4010. When the confidence level associated with the output of the one or more machine-learned models 134b is greater than the threshold confidence level, the BCG signal analyzer 134a may be configured to include the output BCG signal as a measurement associated with the second heart rate information for determining biometric information associated with the user 4010. For example, ground truth data associated with a BCG signal may be stored in a database (e.g., one or more machine-learned models and training data store 360) and referenced for training the one or more machined-leamed models 134b to recognize or assess whether a reflected waveform corresponds to a BCG signal. For example, ground truth BCG signal data may be collected by simulating the oscillatory BCG response for known heart rates (e.g., by placing a sonar device near the chest of the user (e.g., less than 20 cm) and measuring data). [0133] In some implementations, the first heart rate information determination application 132 can obtain first heart rate information (e.g., a PPG signal) via the one or more optical sensors 184 simultaneously while the second heart rate information determination application 134 obtains second heart rate information (e.g., a BCG signal) via the audio device 162. For example, the wearable computing device 100 may be configured to time- synchronize the PPG signal and the BCG signal to determine biometric information based on a combination of data from the PPG signal and the BCG signal.

[0134] At operation 7500 the method 7000 includes determining biometric information based on the first heart rate information and the second heart rate information associated with the user. For example, the biometric information determiner 136 may be configured to determine a pulse transit time (PTT) metric value based on the first heart rate information and the second heart rate information which are (simultaneously) obtained. For example, the biometric information determiner 136 may be configured to determine the PTT metric value, for example, based on a time interval 5600 between a peak of the PPG signal (e.g., peak 5500) and a peak of the BCG signal (e.g., the peak associated with the J-wave of the BCG signal as indicated by peak 5300). The PTT metric value may be a useful prognostic indicator for overall cardiovascular risk and/or for determining a blood pressure value. PTT corresponds to the time delay for a pressure wave to travel between two sites (e.g., two arterial sites) and can be estimated based on the relative timing between proximal and distal arterial waveforms. PTT is generally inversely related to blood pressure and may be used to determine (e.g., indirectly) a blood pressure value according to known methods and algorithms (e.g., using linear regression models, neural network models, support vector machine regression algorithms, etc.) which define a relationship between PTT and blood pressure. Known models include a non-linear model developed by Moens and Korteweg and a non-linear model by Gesche. However, the disclosure is not limited to these methods and other models may be applied by the biometric information determiner 136 to determine a blood pressure value from the PTT.

[0135] The biometric information determiner 136 may be configured to determine a plurality of PTT values to determine a histogram feature and the biometric information determiner 136 may be configured to map the histogram data to a blood pressure value (e.g., via one or more machine-learned models 136a including a neural network).

[0136] The biometric information determiner 136 may be configured to store the plurality of PTT values and/or corresponding blood pressure values (as well as any other biometric information including a PPG signal. BCG signal, etc.) in the one or more memory devices 120, at the server computing system 300, at external computing device 400, at user biometric information data store 350, etc. Storage of the biometric information may provide ahistory of heart health for an individual over time.

[0137] In the context of performing the operations of method 7000, the biometric measurement application 130 may also be configured to consider motion information associated with a user while biometric measurements are being taken in addition to, or instead of, prior to such biometric measurements being taken. For example, biometric measurements may be negatively impacted by excessive motion of a user while a biometric measurement is being recorded by the one or more optical sensors 184 and/or the audio device 162. Motion determiner 138 may be configured to provide information about motion information associated with a user to any of the first heart rate information determination application 132, second heart rate information determination application 134, biometric information determiner 136, and distance determiner 139.

[0138] In some implementations, the motion determiner 138 may be configured to obtain motion information of the wearable computing device 100 (which is associated with the user) based on data output by the inertial measurement unit 182 including the one or more accelerometers 182a and one or more gy roscopes 182b. The motion information may be obtained by the motion determiner 138 while the first heart rate information is obtained by the first heart rate information determination application 132 via the one or more optical sensors 184 and/or while the second heart rate information is obtained by the second heart rate information determination application 134 via the audio device 162. The motion determiner 138 may be configured to determine whether the obtained motion information exceeds a threshold motion value. For example, for a first duration of time in which the motion information associated with the wearable computing device 100 exceeds a threshold motion value (as determined by the motion determiner 138) while the second heart rate information associated w ith the user is obtained, the biometric information determiner 136 may be configured to exclude the second heart rate information corresponding to the first duration of time as second heart rate information for determining the biometric information (e.g., the PTT, heart rate, blood pressure). For example, the second heart rate information determination application 134, biometric information determiner 136, or motion determiner 138 may mark second heart rate information associated with the first duration of time as being unacceptable or to be omitted when determining a heart rate value, pulse transit time, blood pressure value, or other parameter based on the BCG signal obtained via the audio device 162. For example, for a second duration of time in which the motion information (as determined by the motion determiner 138) associated with the wearable computing device 100 does not exceed the threshold motion value while the second heart rate information associated with the user is obtained, the biometric information determiner 136 may be configured to include the second heart rate information corresponding to the second duration of time as second heart rate information for determining the biometric information (e.g., the PTT, heart rate, blood pressure).

[0139] Likewise, for a first duration of time in which the motion information associated with the wearable computing device 100 exceeds a threshold motion value (as determined by the motion determiner 138) while the first heart rate information associated with the user is obtained, the PPG signal analyzer 132a may be configured to exclude the first heart rate information corresponding to the first duration of time as first heart rate information for determining the biometric information. For example, the first heart rate information determination application 132, biometric information determiner 136, or motion determiner 138 may mark first heart rate information associated with the first duration of time as being unacceptable or to be omitted when determining a heart rate value, pulse transit time, blood pressure value, or other parameter based on the PPG signal obtained via the one or more optical sensors 184. For example, for a second duration of time in which the motion information (as determined by the motion determiner 138) associated with the wearable computing device 100 does not exceed the threshold motion value while the first heart rate information associated with the user is obtained, the PPG signal analyzer 132a may be configured to include the first heart rate information corresponding to the second duration of time as first heart rate information for determining the biometric information.

[0140] In the context of performing the operations of method 7000, the biometric measurement application 130 may also be configured to consider distance information pertaining to the distance between the wearable computing device 100 and the body part of the user associated with measuring the second heart information (e.g., the chest), while biometric measurements are being taken in addition to, or instead of, prior to such biometric measurements being taken. For example, biometric measurements may be negatively impacted by excessive distance betw een the chest of the user and the wearable computing device 100 while a biometric measurement is being recorded by the one or more optical sensors 184 and/or the audio device 162. Distance determiner 139 may be configured to provide information about distance information associated with a user and the wearable computing device 100 to any of the first heart rate information determination application 132, second heart rate information determination application 134, biometric information determiner 136, and motion determiner 138.

[0141] In some implementations, the distance determiner 139 may be configured to obtain distance information of the wearable computing device 100 (which is associated with the user) relating to a distance between the wearable computing device 100 and the body part associated with measuring the second heart rate information (e.g., the chest). The distance may be determined based on information provided by the one or more proximity sensors 186, the audio device 162, etc., as described herein. The distance may be obtained by the distance determiner 139 while the first heart rate information is obtained by the first heart rate information determination application 132 via the one or more optical sensors 184 and/or while the second heart rate information is obtained by the second heart rate information determination application 134 via the audio device 162. The distance determiner 139 may be configured to determine whether the obtained distance exceeds a threshold distance value. For example, for a first duration of time in which the distance exceeds a threshold distance value (as determined by the distance determiner 139) while the second heart rate information associated with the user is obtained, the biometric information determiner 136 may be configured to exclude the second heart rate information corresponding to the distance as second heart rate information for determining the biometric information (e.g., the PTT, heart rate, blood pressure). For example, the second heart rate information determination application 134, biometric information determiner 136, or distance determiner 139 may mark second heart rate information associated with the first duration of time as being unacceptable or to be omitted when determining a heart rate value, pulse transit time, blood pressure value, or other parameter based on the BCG signal obtained via the audio device 162. For example, for a second duration of time in which the distance (as determined by the distance determiner 139) does not exceed the threshold distance value while the second heart rate information associated with the user is obtained, the biometric information determiner 136 may be configured to include the second heart rate information corresponding to the second duration of time as second heart rate information for determining the biometric information (e.g., the PTT, heart rate, blood pressure).

[0142] Likewise, for a first duration of time in which the distance exceeds a threshold distance value (as determined by the distance determiner 139) while the first heart rate information associated with the user is obtained, the PPG signal analyzer 132a may be configured to exclude the first heart rate information corresponding to the first duration of time as first heart rate information for determining the biometric information. For example, the first heart rate information determination application 132, biometric information determiner 136, or distance determiner 139 may mark first heart rate information associated with the first duration of time as being unacceptable or to be omitted when determining a heart rate value, pulse transit time, blood pressure value, or other parameter based on the PPG signal obtained via the one or more optical sensors 184. For example, for a second duration of time in which the distance (as determined by the distance determiner 139) does not exceed the threshold distance value while the first heart rate information associated with the user is obtained, the PPG signal analyzer 132a may be configured to include the first heart rate information corresponding to the second duration of time as first heart rate information for determining the biometric information.

[0143] In some implementations, in a circumstance in which the user is aware that a biometric measurement is being recorded, the user may be alerted or notified via the output device 160 to remain stationary in response to the motion determiner 138 determining that the motion information associated with the wearable computing device 100 exceeds the threshold motion value. For example, a graphical user interface may be presented via the display device 150 which provides instructions to a user to remain stationary. If the user remains stationary, the biometric measurement application 130 may be configured to continue measuring the biometric information (e.g., via the one or more optical sensors 184 and audio device 162). In some implementations, in response to the motion determiner 138 determining that the motion information associated with the wearable computing device 100 exceeds the threshold motion value for longer than a threshold period of time (e.g., 3 seconds, 5 seconds, etc.), the biometric measurement application 130 may discontinue or cancel the biometric measurement (e.g., of the PPG signal and/or BCG signal) and provide an output to the user indicating the biometric measurement was discontinued. In addition, the wearable computing device 100 may be configured to disable or turn off the one or more optical sensors 184 and/or the audio device 162 to conserve energy and computing resources. If the motion information associated with the wearable computing device 100 exceeds the threshold motion value for less than the threshold period of time, the biometric measurement application 130 may continue with the biometric measurements but may disregard biometric information collected during the corresponding period of time as described earlier. [0144] In some implementations, in a circumstance in which the user is aware that a biometric measurement is being recorded, the user may be alerted or notified via the output device 160 to move the wearable computing device 100 closer to the relevant body part of the user for measuring the second heart rate information (e.g., the chest) in response to the distance determiner 139 determining that the distance exceeds the threshold distance value. For example, a graphical user interface may be presented via the display device 150 which provides instructions to a user to move their arm closer to their chest. If the distance between the wearable computing device 100 and the body part is subsequently determined to be less than the threshold distance value, the biometric measurement application 130 may be configured to continue measuring the biometric information (e.g., via the one or more optical sensors 184 and audio device 162). In some implementations, in response to the distance determiner 139 determining that the distance exceeds the threshold distance value for longer than a threshold period of time (e.g., 3 seconds, 5 seconds, etc.), the biometric measurement application 130 may discontinue or cancel the biometric measurement (e.g., of the PPG signal and/or BCG signal) and provide an output to the user indicating the biometric measurement was discontinued. In addition, the wearable computing device 100 may be configured to disable or turn off the one or more optical sensors 184 and/or the audio device 162 to conserve energy and computing resources. If the distance associated with the wearable computing device 100 exceeds the threshold distance value for less than the threshold period of time, the biometric measurement application 130 may continue with the biometric measurements but may disregard biometric information collected during the corresponding period of time as described earlier.

[0145] As mentioned above, aspects of the disclosure have been described in view of the biometric measurement application 130 provided in the wearable computing device 100 with respect to FIGS. 3 through 7. However, each of those aspects can also be applied to the biometric measurement application 330 provided in the server computing system 300, and thus some or all of the functions and operations of the biometric measurement application 130 may also be applied and carried out by the biometric measurement application 330 in a similar fashion but will not be described again for the sake of brevity.

[0146] Aspects of the above-described example embodiments may be recorded in non- transitory computer-readable media including program instructions to implement various operations embodied by a computer. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. Examples of non- transitory computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM disks, Blue-Ray disks, and DVDs; magneto-optical media such as optical discs; and other hardware devices that are specially configured to store and perform program instructions, such as semiconductor memory, readonly memory (ROM), random access memory (RAM), flash memory, USB memory, and the like. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. The program instructions may be executed by one or more processors. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments, or vice versa. In addition, a non-transitory computer-readable storage medium may be distributed among computer systems connected through a network and computer-readable codes or program instructions may be stored and executed in a decentralized manner. In addition, the non- transitory computer-readable storage media may also be embodied in at least one application specific integrated circuit (ASIC) or Field Programmable Gate Array (FPGA).

[0147] Each block of the flowchart illustrations may represent a unit, module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of order. For example, two blocks shown in succession may in fact be executed substantially concurrently (simultaneously) or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

[0148] While the disclosure has been described with respect to various example embodiments, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the disclosure does not preclude inclusion of such modifications, variations and/or additions to the disclosed subject matter as would be readily apparent to one of ordinary skill in the art. For example, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the disclosure covers such alterations, variations, and equivalents.

Claims

WHAT IS CLAIMED IS:

1. A wearable computing device, comprising: one or more optical sensors; an audio device; one or more memories configured to store one or more instructions; and one or more processors configured to execute the one or more instructions stored in the one or more memories to: obtain, via the one or more optical sensors, first heart rate information associated with a user wearing the wearable computing device, obtain, via the audio device, second heart rate information associated with the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information.

2. The wearable computing device of claim 1, the audio device including a transmitter configured to emit a sound wave and a receiver configured to receive a reflected sound wave that corresponds to a sound wave reflected off a first location on the user.

3. The wearable computing device of claim 2, the transmitter emitting an ultrasonic sound wave.

4. The wearable computing device according to claim 2, the one or more optical sensors including one or more photoplethysmography (PPG) sensors, the one or more PPG sensors including a light source configured to emit a light and a photodetector configured to receive reflected light reflected off a body part of the user.

5. The wearable computing device of claim 4, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: generate a PPG signal to obtain the first heart rate information based on the reflected light, generate a ballistocardiogram (BCG) signal to obtain the second heart rate information based on the reflected sound wave, and determine the biometric information associated with the user based on a first peak associated with the PPG signal and a second peak associated with the BCG signal.

6. The wearable computing device of claim 5, the biometric information including at least one of a pulse transit time, a blood pressure value, or a heart rate.

7. The wearable computing device of claim 5, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: implement one or more machine-learned models to determine whether the BCG signal generated based on the reflected sound wave is indicative of an expected BCG signal associated with the first location on the user, when a confidence level associated with an output of the one or more machine- learned models is less than a threshold confidence level, exclude the BCG signal as a measurement associated with the second heart rate information for determining the biometric information, and when the confidence level associated with the output of the one or more machine- learned models is greater than the threshold confidence level, include the BCG signal as a measurement associated with the second heart rate information for determining the biometric information.

8. The wearable computing device of claim 1 , the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: determine a distance between the wearable computing device and a body part associated with measuring the second heart rate information, when the distance exceeds a threshold distance value, disable the audio device from obtaining the second heart rate information, and when the distance does not exceed the threshold distance value, enable the audio device to obtain the second heart rate information.

9. The wearable computing device of claim 8, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: when the distance exceeds the threshold distance value, disable the one or more optical sensors from obtaining the first heart rate information, and when the distance does not exceed the threshold distance value, enable the one or more optical sensors to obtain the first heart rate information.

10. The wearable computing device of claim 1, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: determine a distance between the wearable computing device and a body part associated with measuring the second heart rate information, when the distance exceeds a threshold distance value, exclude the measurement associated with the second heart rate information for determining the biometric information, and when the distance does not exceed the threshold distance value, include the measurement associated with the second heart rate information for determining the biometric information.

11. The wearable computing device of claim 10, further comprising a motion sensor, the one or more processors being configured to: determine motion information associated with the wearable computing device via the motion sensor, for a first duration of time in which the motion information associated with the wearable computing device exceeds a threshold motion value while the second heart rate information associated with the user is obtained, exclude the second heart rate information corresponding to the first duration of time as second heart rate information for determining the biometric information, and for a second duration of time in which the motion information associated with the wearable computing device does not exceed the threshold motion value while the second heart rate information associated with the user is obtained, include the second heart rate information corresponding to the second duration of time as second heart rate information for determining the biometric information.

12. The wearable computing device of claim 11, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: determine whether the motion information associated with the wearable computing device exceeds a threshold motion value, when the motion information exceeds the threshold motion value, disable the audio device from obtaining the second heart rate information, and when the motion information does not exceed the threshold motion value, enable the audio device to obtain the second heart rate information.

13. The wearable computing device of claim 1, the one or more processors being configured to obtain, via the one or more optical sensors, the first heart rate information while the second heart rate information is being obtained via the audio device.

14. The wearable computing device of claim 1, the one or more processors being configured to activate the audio device to obtain the second heart rate information in response to the one or more processors detecting the user is engaged in a predetermined activity.

15. The wearable computing device of claim 14, the one or more processors being configured to detect the predetermined activity of the user sleeping.

16. The wearable computing device of claim 1, the wearable computing device including a smartwatch or a tracker.

17. A computer-implemented method, comprising: obtaining, via one or more optical sensor of a wearable computing device, first heart rate information associated with a user wearing the wearable computing device; obtaining, via an audio device of the wearable computing device, second heart rate information associated with the user; and determining biometric information associated with the user based on the first heart rate information and the second heart rate information.

18. The computer-implemented method of claim 17, the first heart rate information being passively obtained via the one or more optical sensors, and the second heart rate information being passively obtained via the audio device.

19. The computer-implemented method of claim 18, obtaining, via the one or more optical sensors of the w earable computing device, the first heart rate information, includes emitting light and detecting reflected light, obtaining, via the audio device of the wearable computing device, the second heart rate information, includes transmitting an audio signal and receiving an audio reflection, determining the biometric information includes: generating a photoplethysmography (PPG) signal based on the reflected light, generating a ballistocardiogram (BCG) signal based on the audio reflection, and determining an interval of time between a first peak associated with the PPG signal and a second peak associated with the BCG signal to determine a pulse transit time.

20. A non-transitory computer-readable medium which stores instructions that are executable by one or more processors of a computing device, the instructions comprising instructions to cause the one or more processors to: obtain, via one or more optical sensors of a wearable computing device, first heart rate information associated w ith a user wearing the wearable computing device; obtain, via an audio device of the wearable computing device, second heart rate information associated with the user; and determine biometric information associated with the user based on the first heart rate information and the second heart rate information.