JPWO2021111436A5

JPWO2021111436A5 -

Info

Publication number: JPWO2021111436A5
Application number: JP2022532066A
Authority: JP
Publication date: 2023-12-05

Claims

A method for obtaining physiological signals from a subject, the method comprising: using a camera to obtain optical data from the subject's face; and using a computing device in communication with the camera to obtain optical data from the subject's face. analyzing the optical data to select data related to a face ; the computing device detecting optical data from skin of the face; and detecting optical data from the skin of the face until an elapsed time period is reached. determining a time series from the optical data by collecting and then calculating the time series from the collected optical data over the elapsed time period; and calculating the physiological signal from the time series. and detecting the optical data from the skin of the face includes determining a plurality of facial boundaries, determining a score for each facial boundary, and selecting the facial boundary with the highest score. and applying a histogram analysis to video data from the face, the histogram analysis comprising a histogram-based classifier with a soft thresholding mechanism .

2. The method of claim 1, wherein the optical data includes video data, and the obtaining the optical data includes obtaining video data of the face of the subject.

3. The method of claim 2, wherein the obtaining the optical data further comprises obtaining video data from the cell phone camera, such that the camera includes a cell phone camera.

4. The method of claim 3, wherein the computing device comprises a mobile communication device.

5. The method of claim 4, wherein the mobile phone camera includes a front-facing camera.

6. The method of claim 5 , wherein the computing device is physically separate from, but in communication with, the mobile phone camera.

2. The method of claim 1 , wherein the determining the plurality of facial boundaries includes applying a multi-parameter convolutional neural network (CNN) to the video data to determine the facial boundaries.

2. The method of claim 1 , wherein the obtaining the optical data further comprises obtaining video data of the subject's finger skin.

9. The method of claim 8 , wherein the obtaining the video data includes obtaining video data of the skin of the subject's fingertip when the fingertip is placed on the camera.

10. The method of claim 9 , wherein the camera for acquiring the fingertip video data includes a mobile phone camera.

11. The method of claim 10 , wherein the fingertip is placed on the mobile phone camera while activating a flash associated with the mobile phone camera to provide light.

The detecting the optical data from the skin of the face includes determining a plurality of face or fingertip boundaries, determining a score for each face or fingertip boundary, and determining a face or fingertip boundary with the highest score. 2. The method of claim 1 , comprising selecting and applying histogram analysis to video data from the face or fingertip.

13. The method of claim 12 , wherein the determining the plurality of face or fingertip boundaries comprises applying a multi-parameter convolutional neural network (CNN) to the video data to determine the face or fingertip boundaries. Method.

14. The method of claim 13 , further comprising combining analyzed data from the face and fingertip images to determine the physiological measurements.

The determining the physiological measurements further includes combining metadata with measurements from the physiological signals, the metadata including weight, age, height, biological sex of the subject. 15. The method of claim 14 , comprising one or more of , body fat percentage, and body muscle percentage.

16. The method of claim 15 , wherein the physiological signal is selected from the group consisting of stress, blood pressure, respiratory rate, and oxygen saturation ( pSO2 ) .

creating a PPG signal from the detected optical data by calculating an rPPG trace signal using the time series; estimating an average pulse rate from the rPPG trace signal; and calculating an average instantaneous pulse rate. and determining the rPPG signal with an initial signal determined according to a projection matrix (PM) applying adaptive Wiener filtering and according to the instantaneous pulse frequency.

18. The method of claim 17, wherein the average pulse rate is estimated using a matched filter between two different rPPG analysis signals constructed from raw interpolated data.

19. The method of claim 18, wherein a cross-correlation is calculated to determine the average instantaneous pulse rate and a frequency estimate is calculated according to a non-linear least squares (NLS) spectral decomposition.

20. The method of claim 19, wherein the PPG signal is further determined by applying an additional filter in the frequency domain to force signal reconstruction and an exponential filter applied to the instantaneous RR values.

A system for acquiring physiological signals from a subject, the system comprising: a camera for acquiring optical data from the subject's face; and a user computing device for receiving optical data from the camera. , the user computing device includes a processor and a memory for storing a plurality of instructions , the processor detecting optical data from the skin of the face, and detecting optical data from the facial skin until an elapsed time period is reached. determining a time series from the optical data by collecting data; then calculating the time series from the collected optical data over the elapsed time period; The step of executing the instructions for analyzing the optical data to select data related to the subject's face and detecting the optical data from the skin of the face, together with the step of calculating a signal, comprises a plurality of steps of: determining face boundaries; determining a score for each face boundary; selecting the face boundary with the highest score; and applying a histogram analysis to video data from the face, A system wherein the analysis includes a histogram-based classifier with a soft thresholding mechanism .

The memory is configured to store a defined native instruction set of code, and the processor receives corresponding primitive instructions selected from the defined native instruction set of code stored in the memory. the memory configured to perform a defined set of primitive operations in response to the native instruction set for analyzing the optical data to select data related to the face of the subject; a first set of machine code selected from the native instruction set for detecting optical data from the facial skin; and a second set of machine code selected from the native instruction set for detecting optical data from the facial skin; a first instruction selected from the native instruction set for determining a time series from the optical data by collecting optical data; and then calculating the time series from the collected optical data over the elapsed time period. 22. The system of claim 21 , storing three sets of machine code and a fourth set of machine code selected from the native instruction set for calculating the physiological signal from the time series.

The determining the plurality of facial boundaries includes applying a multi-parameter convolutional neural network (CNN) to the video data to determine the facial boundaries, such that the memory determines the facial boundaries. 23. The system of claim 22 , further comprising an eighth set of machine code selected from the native instruction set for applying the multi -parameter convolutional neural network (CNN) to the video data.

24. The system of claim 23 , wherein the camera includes a mobile phone camera and the optical data is obtained as video data from the mobile phone camera.

25. The system of claim 24 , wherein the user computing device includes a mobile communication device.

26. The system of claim 25 , wherein the mobile phone camera includes a rear camera, and the subject's fingertip is placed on the camera to obtain the video data.

27. The system of claim 26 , further comprising a flash associated with the mobile phone camera to provide light for acquiring the optical data.

The memory includes a ninth set of machine codes selected from the native instruction set for determining a plurality of face or fingertip boundaries and from the native instruction set for selecting the face or fingertip boundary having the highest score . 10. The method of claim 1, further comprising a selected tenth set of machine code and an eleventh set of machine code selected from the native instruction set for applying histogram analysis to video data from the face or fingertip. 27. The system described in 27 .

29. The system of claim 28 , further comprising combining analyzed data from the face and fingertip images to determine the physiological measurements in accordance with the instructions executed by the processor.

The system further comprises a display for displaying the physiological measurements and/or signals , and the user computing device further includes the display, and the user computing device further includes a display for displaying the physiological measurements and/or signals. 30. The system of claim 29 , further comprising a transmitter for transmitting .

A system for acquiring physiological signals from a subject, the system comprising: a rear camera for acquiring optical data from a finger of the subject; and a user computing device for receiving optical data from the camera. wherein the user computing device includes a processor and a memory for storing a plurality of instructions , the processor detecting optical data from the skin of the finger; determining a time series from the optical data by collecting optical data; then calculating the time series from the collected optical data over the elapsed time period; executing the instructions for analyzing the optical data to select data related to the finger of the subject and detecting the optical data from the skin of the finger, with the step of calculating a target signal; determining a score for each finger boundary, selecting the finger boundary with the highest score, and applying a histogram analysis to the video data from the finger; A system wherein the histogram analysis includes a histogram-based classifier with a soft thresholding mechanism .