WO2022075970A1

WO2022075970A1 - Transmitting biometric healthcare data

Info

Publication number: WO2022075970A1
Application number: PCT/US2020/054311
Authority: WO
Inventors: Robert Campbell; Christopher STEVEN
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2020-10-05
Filing date: 2020-10-05
Publication date: 2022-04-14

Abstract

In one example, a method of transmitting biometric healthcare data is disclosed. A patient computing system is used to determine biometric healthcare data of a patient during a telemedicine video conference. The telemedicine video conference having a data stream to communicate telemedicine audio/visual data between the patient computing system and a remote healthcare provider computing system. A biometric healthcare video data stream containing the biometric healthcare data of the patient is generated. The biometric healthcare video data stream is composited with the data stream of the telemedicine video conference to create a composite data stream. The composite data stream is transmitted from the patient computing system to the remote healthcare provider computing system.

Description

TRANSMITTING BIOMETRIC HEALTHCARE DATA

BACKGROUND

[0001]Telemedicine video conferences are replacing in-office healthcare consultations for a significant portion of healthcare including check-ups and follow-on care. Telemedicine video conferencing does not allow for the collection of biometric healthcare data such as temperature, blood pressure, and heart rate, etc. that is recorded during an in-office visit. Awareness and consideration of such healthcare data is a part of rendering safe and effective healthcare. Self-measurement and reporting lacks the accuracy, uniformity, and objectivity of standardized measurement devices and procedures.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Examples of the disclosure will be rendered by reference to specific examples thereof which are illustrated in the appended drawings. The drawings illustrate only particular examples of the disclosure and therefore are not to be considered to be limiting of its scope. The principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[0003] FIG. 1 illustrates a process by which a patient transmits healthcare data to a healthcare provider during a telemedicine video conference according to examples of the present disclosure.

[0004] FIG. 2 illustrates example instructions stored on an example non-transitory computer-readable storage medium to implement the determination and transmission of biometric healthcare data during a telemedicine video conference according to examples of the present disclosure.

[0005] FIG. 3A illustrates replacement of the background of a video image of a telemedicine video conference with healthcare data in accordance with examples of the present disclosure.

[0006] FIG. 3B illustrates replacement of the background of a video image of a telemedicine video conference with healthcare data in accordance with examples of the present disclosure.

[0007] FIG. 4 illustrates an example display of a telemedicine video conference image providing biometric healthcare data according to examples of the present disclosure.

[0008] FIG. 5 illustrates an example video conferencing system to implement the transmission of biometric healthcare data during a telemedicine video conference according to some examples of the present disclosure.

DETAILED DESCRIPTION

[0009] As noted above, telemedicine video conferences are replacing in-office healthcare consultations for a significant portion of healthcare needs including checkups and follow-on care. Healthcare may be provided remotely via a telemedicine video conference for convenience and safety. However, telemedicine video conferencing limits the ability of the healthcare provider to obtain the patient’s biometric healthcare data in real time which is helpful in evaluating the patient and providing medical care.

[0010] While it may be possible for a patient to obtain their present healthcare data by using measurement devices such as, for example, a thermometer, a heart rate monitor, or a blood pressure monitor, many patients will not have access to sufficiently reliable medical equipment and may not be able to obtain accurate measurements. Further, a patient may have difficulty transmitting biometric healthcare data to the healthcare provider in a timely manner during the telemedicine video conference.

Moreover, requesting and transmitting biometric healthcare data may be interruptive.

[0011] Examples of the present disclosure allow a patient to transmit their biometric healthcare data to a healthcare provider (e.g. doctor, nurse, physical therapist, etc.) during a telemedicine video conference. The biometric healthcare data may include measurements of the characteristics of the human body such as heartbeat, blood pressure, and body temperature.

[0012] The biometric healthcare data may be historical healthcare data, present healthcare data, and real-time healthcare data.

[0013] Historical healthcare data is healthcare data that has been recorded and stored prior to the telemedicine video conference. Historical healthcare data may include logged data from previous telemedicine video conferences as well as stored data from other sources.

[0014] Present healthcare data is healthcare data that is recorded concurrently with, or immediately prior to, the telemedicine video conference. Present healthcare data may include healthcare data that a healthcare provider may want to access, but that may not be considered vital healthcare data and may not require real time updates during a telemedicine video conference. Present healthcare data may include such biometric healthcare data as weight and body mass index (BMI).

[0015] Real-time healthcare data is healthcare data that is recorded during the telemedicine video conference and may be updated periodically during the telemedicine video conference. Real-time healthcare data may include patient vital signs such as body temperature, heart rate, blood pressure, and respiration.

[0016] In accordance with examples of the present disclosure, a patient may obtain their present healthcare data during the telemedicine video conference by using available measurement devices such as, for example, a thermometer, a heart rate monitor, or a blood pressure monitor. For some examples of the present disclosure, the patient’s real time biometric healthcare data, such as medically accurate vital signs, may be captured (obtained) during the telemedicine video conference using the primary camera (e.g., RGB camera) or infrared (IR) camera implemented in a patient’s laptop or other processing device (computing system). In accordance with examples of the present disclosure, the RGB camera or IR camera may be implemented to function as a healthcare data acquisition virtual camera. For some examples of the present disclosure, a patient’s vital signs may be measured and captured throughout the telemedicine video conference using a healthcare data acquisition virtual camera in order to transmit real-time biometric healthcare data to the healthcare provider.

[0017] In accordance with examples of the present disclosure, a patient may transmit historical healthcare data during the telemedicine video conference. Historical healthcare data may include logged biometric healthcare data from previous telemedicine video conferences. In another example, the historical healthcare data may be fitness logs and medical history data, including prescriptions, previous diseases or test results, and drug allergies, among others. Such historical healthcare data may be stored on the patient’s video conference computing system, on health-related monitoring devices, or in remote storage systems (e.g., cloud storage systems). A patient may transmit historical healthcare data directly or may transmit a link (e.g., internet link) to the stored data.

[0018] For some examples of the present disclosure, the present or historical healthcare data that the patient desires to transmit to the healthcare professional is encoded with the real time biometric healthcare data as a video stream and transmitted to the healthcare provider’s computing system. For some examples, the healthcare data is encoded in a human-readable format and/or a machine-readable format such as a matrix barcode (e.g., Quick Response (QR) code).

[0019] For some examples, the encoded video stream containing the healthcare data of the patient is composited with the webcam video stream of the telemedicine video conference. The composited video stream is transmitted to the healthcare provider’s computing system for display during the telemedicine video conference. Some examples of the present disclosure may be implemented as a plugin for a video teleconferencing system (e.g., Zoom™). The healthcare data may be displayed during the telemedicine video conference as a virtual background of the telemedicine video conference or as a picture-in-picture format. For some examples, the healthcare data video stream may be transmitted and presented independently of the telemedicine video conference video stream, and the healthcare data may be stored to a healthcare records system of the healthcare provider.

[0020] FIG. 1 illustrates an example process by which a patient transmits biometric healthcare data to a healthcare provider during a telemedicine video conference according to the present disclosure. Process 100, shown in FIG. 1 , begins with operation 102 in which biometric healthcare data of a patient is determined. For some examples, the biometric healthcare data is determined by use of an infrared camera of the patient’s computing system. The infrared camera detects infrared energy emitted from the user to provide a heat signature measured by the thermal imager of the infrared camera. Pixels in a sensor array then provide corresponding electronic signals.

[0021] At operation 104 a biometric healthcare data video stream is generated that contains the biometric healthcare data of the patient for transmission during the telemedicine video conference. Present and historical healthcare data may also be included. As noted, the healthcare data video stream may contain the healthcare data in a machine-readable format, a human-readable format, or a combination of machine- readable and human-readable formats. For one example, the healthcare data is encoded using video encoding techniques that allow a wide variety of devices to interpret the encoded stream. A live stream may be sent from the patient’s webcam to a server and may be segmented using a streaming protocol such as HLS (HTTP live streaming). The segmented video and healthcare data may then be encoded using an encoding standard (e.g., H.264) so that the content can be appropriately decoded at the remote location. [0022] At operation 106 the biometric data video stream is composited with the video stream of the telemedicine video conference. The telemedicine video conference may have a data stream to communicate telemedicine audio/visual data between the patient’s computing system and a remote healthcare provider computing system. The compositing may layer two video streams together by rendering a color range of the foreground of an image of a first video stream transparent. This may allow a second video stream or static image to be inserted into the image of the first video stream. As noted, for some examples of the present disclosure, the healthcare data video stream may be transmitted and presented independently of the telemedicine video conference video stream and may be directly stored to a healthcare records system of the healthcare provider.

[0023] At operation 108 the composite video stream is transmitted from the patient computing system to the remote healthcare provider computing system. For some examples, the composite video stream may display the biometric healthcare data as an overlay of the telemedicine video conference through use of background removal. For some examples, the biometric healthcare data is displayed as a picture-in-picture format.

[0024] FIG. 2 illustrates example instructions stored on an example non-transitory computer-readable storage medium 200 to transmit biometric healthcare data. More specifically, non-transitory computer-readable storage medium 200 may implement the determination and transmission of biometric healthcare data during a telemedicine video conference according to examples of the present disclosure. FIG. 5 illustrates an example video conferencing system 500 that may be used to implement the transmission of biometric healthcare data during a telemedicine video conference according to some examples of the present disclosure as described in further detail below.

[0025]As shown in FIG. 2, the non-transitory computer-readable storage medium 200 includes instruction 202 that may cause a processor (e.g., CPU 525 of FIG. 5) to transmit such biometric healthcare data.

[0026] The non-transitory computer-readable storage medium 200 may be any electronic, magnetic, optical, or other physical storage device that stores executable instructions. For example, the non-transitory computer-readable storage medium 200 may be random access memory (RAM), an electrically-erasable programmable readonly memory (EEPROM), a storage drive, an optical disc, or the like. The non-transitory computer-readable storage medium 200 may be encoded to store executable instructions that cause a processor to perform operations according to examples of the disclosure.

[0027] Instruction 202 may include instruction 204, instruction 206 and instruction 208 to implement the determination and transmission of biometric healthcare data.

[0028] Instruction 204 may cause a processor to determine biometric healthcare data of a patient during a telemedicine video conference (using, for example, video conferencing system 500 (FIG. 5)). The telemedicine video conference may have a data stream to communicate the telemedicine audio/visual data between a patient computing system (e.g., computing system 505a of FIG. 5) and a remote healthcare provider computing system (e.g., computing system 505b of FIG. 5).

[0029] For some examples, real-time biometric healthcare data may be determined through use of a primary camera (e.g., RGB camera 556 of FIG. 5) or an IR camera (e.g., IR camera 557 of FIG. 5) of the patient’s computing system. For some examples, real-time or present biometric healthcare data may be determined through the use of medical measuring devices (e.g., heart rate and blood pressure monitors) during the telemedicine video conference. For some examples, the medical measuring devices may be directly connected to the patient computing system to facilitate the capturing of real-time or present biometric healthcare data.

[0030] Instruction 206 may cause the processor to encode the biometric healthcare data as a video stream. For some examples, the biometric healthcare video data stream may contain historical healthcare data of the patient. As noted, such historical healthcare data may include many types of previously obtained healthcare data including logged biometric healthcare data from previous telemedicine video conferences. Non-biometrical medical history data may also be transmitted. Historical healthcare data (e.g., historical healthcare data 560 of FIG. 5), may be stored on the patient’s video conference computing system, on health-related monitoring devices, or in remote storage systems (e.g., cloud storage systems).

[0031] Instruction 208 may cause the processor to transmit the biometric healthcare data video stream from the patient computing system to the remote healthcare provider computing system. In some examples, the biometric healthcare data video stream may be composited with the video stream of the telemedicine video conference to create a composite video stream.

[0032] The composite video stream may then be transmitted from the patient computing system to the remote healthcare provider computing system. For some examples, the composite video stream may display the biometric healthcare data as an overlay of the telemedicine video conference through the use of background removal. For some examples, the biometric healthcare data is displayed as a picture-in-picture format where one video stream may be displayed full-screen and other video streams may be displayed in inset windows. In one example, the picture-in-picture is via application programming interfaces (APIs) where the one video stream is opened up in an overlay.

[0033] FIG. 3A and FIG. 3B illustrate replacement of the background of a video image of a telemedicine video conference with healthcare data in accordance with examples of the present disclosure. As shown in FIG. 3A, telemedicine video conferencing image 310a includes a patient image 312 and a background 314a. Background 314a includes background objects, shown for example as background objects 316a - 318a. Examples of the present disclosure use video editing to replace some or all of the background of the telemedicine video conference with biometric healthcare data of the patient. As shown in FIG. 3B, a composited video stream image 310b includes the patient image 312 and a background 314b. Background 314b includes background objects 319b and 320b. Background object 319b includes the biometric healthcare data of the patient in a human-readable format, shown for example as text. Background object 320b includes the biometric healthcare data of the patient in a machine-readable format, shown for example as a QR code. For some examples of the present disclosure, the machine-readable format may be a static QR code that includes patient healthcare data or a link to such data. For some examples of the present disclosure, the machine-readable format may be a dynamic QR code that includes real time biometric healthcare data of the patient determined during the telemedicine video conference and updated periodically throughout the telemedicine video conference.

[0034] FIG. 4 illustrates an image of a live video feed of a telemedicine video conference displayed on a medical provider computing system in accordance with examples of the present disclosure. Image 400, shown in FIG. 4 includes an image of a patient 410 and a human-readable display of biometric healthcare data 419. As shown, biometric healthcare data may include real time vital healthcare data such as heart rate and blood pressure and may include present biometric healthcare data such as weight and BMI. Here, the biometric healthcare data 419 associated with patient 410 is displayed concurrently with the patient 410. In this example, the patient 410 has a heartrate of 50 bpm, a body temperature of 98.6F, a weight of 162 lb and a BMI of 23. Image 400 also includes QR code 420 providing a machine-readable display of biometric healthcare data. In some examples, QR code 420 may be implemented as a dynamic QR code and may contain real time biometric healthcare data. In other examples, QR code 420 may be implemented as a static QR code and may contain historical healthcare care data. [0035] As described above in reference to FIG. 4, some examples of the present disclosure provide biometric healthcare data as part of live video feed of a telemedicine video conference. For some examples, biometric healthcare data may be provided as a patient avatar/icon together with the real time biometric healthcare data of the patient during an audio or voice teleconference.

[0036] FIG. 5 illustrates an example video conferencing system 500 that may be used to implement the transmission of biometric healthcare data during a telemedicine video conference according to some examples of the present disclosure. The video conferencing system 500, shown in FIG. 5, includes patient computing system 505a and healthcare provider computing system 505b. The computing systems 505a and 505b are communicatively coupled to each other by way of a network 510 (e.g., the Internet, an intranet, etc.). According to examples of the present disclosure, a video conferencing system may have any number of computing systems and the computing systems may be communicatively coupled to each other in various ways. Computing systems may include a desktop computer, a mobile computer, a terminal, a network device, or any other suitable computing device.

[0037] As shown in FIG. 5, each of the computing systems 505a and 505b may include computing component 520. Computing component 520 may include a control ler/CPU 525 that may be, for example, a central processing unit (CPU), a chip or any suitable computing or computational device. Controller/CPU 525 may carry out methods described herein and/or execute various modules. Computing component 520 may include an operating system 530, a memory 540, executable code 545 and a storage system 550. Although executable code 545 is shown loaded into memory 540, executable code 545 may be stored in storage system 550. Here, storage system 550 may include input/output (I/O) devices 555.

[0038] Operating system 530 may be, or may include, any code designed to perform tasks involving controlling or otherwise managing operation of computing component 520. This may include scheduling execution of software programs or enabling software programs or other modules or units to communicate. As an example, operating system 530 may be a commercial operating system. For some examples of the disclosure, the computing component 520 may include a computing device that does not use an operating system (e.g., a microcontroller, ASIC, FPGA, or SOC).

[0039] Memory 540 may be implemented in various forms including random access memory (RAM), read-only memory (ROM), volatile or non-volatile memory, a cache memory, or other suitable memory units or storage units. Memory 540 may be a computer-readable non-transitory storage medium.

[0040] Executable code 545 may be any executable code, e.g., an application, a program, or a process. Executable code 545 may be executed by controller/CPU 525 possibly under control of operating system 530. Examples of the present disclosure may include a plurality of executable code that may be loaded into memory 540 and cause controller/CPU 525 to carry out methods described herein.

[0041] Storage system 550 may be or may include, for example, a hard disk drive, flash memory, a micro controller-embedded memory, or removable storage. Content may be stored in storage system 550 and may be loaded from storage system 550 into memory 540 where it may be processed by controller/CPU 525. Although shown as a separate component, storage system 550 may be embedded or included in memory 540.

[0042] I/O devices 555 may include any suitable input devices such as a keyboard/keypad, mouse and any suitable output devices such as displays or monitors. As shown in FIG. 5, the I/O devices 555 of video conferencing system 500 may include a primary color camera, RGB camera 556, and an infrared camera 557. A universal serial bus (USB) device or external hard drive may be included in I/O devices 555. Any applicable I/O devices may be connected to computing component 520 by, for example, a wired or wireless network interface.

[0043]As discussed above, examples of the present disclosure may include a computer-readable medium, which when executed by a processor may cause the processor to perform operations disclosed herein. According to examples of the present disclosure, executable code 545 includes executable code implementing a biometric healthcare data determination module 546, a biometric healthcare data video stream generation module 547, and a video stream compositing module 548. Although shown as distinct modules, the biometric healthcare data determination module 546, the biometric healthcare data video stream generation module 547, and the video stream compositing module 548 may be implemented as a single processing module or multiple processing modules that incorporate and effect other processes.

[0044] According to examples of the disclosure, the biometric healthcare data determination module 546 of the patient computing system 505a may determine real time biometric healthcare data of the patient during a telemedicine video conference. The biometric healthcare data may be determined periodically or continuously throughout the telemedicine video conference. For example, the biometric healthcare data determination module 546 may use images from IR camera 557 in conjunction with an artificial intelligence-based machine-learning application to determine real-time biometric healthcare data such as patient vital signs (e.g., body temperature, heart rate, blood pressure, and respiration).

[0045] According to examples of the disclosure, the biometric healthcare data video stream generation module 547 of the patient computing system 505a may encode the biometric healthcare data determined by the biometric healthcare data determination module 546 to produce a biometric healthcare data video stream containing the biometric healthcare data of the patient. The biometric healthcare data video stream may contain real time biometric healthcare data determined during the telemedicine video conference as well as present and historical healthcare data. The healthcare data may be encoded in a human-readable format and/or a machine- readable format such as a matrix barcode (e.g., QR code). The human-readable format may include numerical or graphical data including the healthcare data. The machine- readable format may be a static code that may include the healthcare data or may include a link or direction to stored healthcare data. The machine-readable format may be a dynamic code that includes the real time biometric healthcare data, and which may be updated periodically or continuously as the real time biometric healthcare data is determined.

[0046] According to examples of the disclosure, the video stream compositing module 548 may composite the biometric healthcare data video stream generated by module 547 with a video stream of the telemedicine video conference to produce a composite video stream. As discussed above, historical healthcare data 560 may also be composited with a video stream of the telemedicine video conference. The composite video stream may provide for the healthcare data encoded therein to be displayed as an overlay of the telemedicine video conference through use of background removal or as a picture-in-picture format.

[0047] Referring again to Figure 5, according to examples of the disclosure, the composite video stream may be transmitted from the patient computing system 505a to the healthcare provider computing system 505b. For some examples, the composite video stream may be received and displayed at the healthcare provider computing system 505b. The biometric healthcare data may be displayed as an overlay of the telemedicine video conference as described above or may be displayed as a picture-in- picture format or through other display methods. For some examples, the healthcare data may be saved to a healthcare records system 570 of the healthcare provider, which may include local or cloud-based storage. Note that healthcare data may be displayed on a different or dedicated display of healthcare provider computing system 505b separate from the display that is showing the video conference.

[0048] Methods according to the above-described examples may be implemented using computer-executable instructions that are stored or otherwise available from computer readable media. Such instructions may include, for example, instructions and data which cause or otherwise configure a special purpose computer, or special purpose processing device to perform a certain function or group of functions. In some examples, some or all of the systems and/or modules may be implemented partially in firmware and/or hardware such as application-specific integrated circuits (ASICs), standard integrated circuits, controllers (e.g., microcontrollers), field- programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), etc. Some or all of the modules, systems and data structures may also be stored (e.g., as software instructions or structured data) on a non-transitory computer-readable storage medium, such as a hard disk or flash drive or other non-volatile storage device.

[0049] Although a variety of examples and other information was used to explain aspects within the scope of the appended claims, no limitation of the claims should be implied based on particular features or arrangements in such examples.

[0050] While the above description is a complete description of specific examples of the disclosure, additional examples are also possible. Thus, the above description should not be taken as limiting the scope of the disclosure which is defined by the appended claims along with their full scope of equivalents.

Claims

CLAIMS:

1 . A method of transmitting biometric healthcare data, the method comprising: determining biometric healthcare data of a patient during a telemedicine video conference, the telemedicine video conference having a data stream to communicate telemedicine audio/visual data between the patient computing system and a remote healthcare provider computing system; generating a biometric healthcare video data stream containing the biometric healthcare data of the patient; compositing the biometric healthcare video data stream with the data stream of the telemedicine video conference to create a composite data stream; and transmitting the composite data stream from the patient computing system to the remote healthcare provider computing system.

2. The method of claim 1 wherein compositing the healthcare video data stream with the data stream of the telemedicine video conference to create a composite data stream includes: replacing the background of the data stream of the telemedicine video conference with the biometric healthcare video data stream.

3. The method of claim 1 wherein the composite data stream is displayable at the remote healthcare provider computing system as a picture-in-picture format of a presentation of the telemedicine video conference.

4. The method of claim 1 wherein the biometric healthcare data of the patient includes real time biometric healthcare data captured by an infrared camera during the telemedicine video conference.

5. The method of claim 4 wherein the biometric healthcare data of the patient is encoded as a dynamic machine-readable code.

6. The method of claim 1 wherein the biometric healthcare data of the patient includes historical biometric healthcare data and the healthcare video data stream includes a machine-readable link to the historical biometric healthcare data.

7. A system for transmitting biometric healthcare data comprising: an RGB camera to capture images of a patient as a telemedicine video conference data stream; an infrared camera to determine real time biometric healthcare data of the patient during the telemedicine video conference; a biometric healthcare data video stream generation module to generate a biometric healthcare video data stream containing the biometric healthcare data of the patient; a video stream compositing module to composite the biometric healthcare data video stream and a telemedicine video conferencing video stream to generate a composite video stream; and a communication link to transmit the biometric healthcare data video stream to a remote healthcare provider computing system.

8. The system of claim 7 wherein the video stream compositing module is to composite the biometric healthcare data video stream with the data stream of the telemedicine video conference to create a composite data stream by replacing the background of the data stream of the telemedicine video conference with the biometric healthcare video data stream.

9. The system of claim 7 wherein the biometric healthcare data of the patient is encoded as a machine-readable code.

10. The system of claim 7 wherein the biometric healthcare data of the patient is encoded as a dynamic machine-readable code.

11 . The system of claim 7 wherein the biometric healthcare data of the patient includes historical biometric healthcare data and the biometric healthcare data stream includes a machine-readable link to the historical biometric healthcare data.

12. A non-transitory, computer-readable storage medium having stored thereon instructions for transmitting biometric healthcare data which, when the instructions are executed by a processor, cause the processor to: determine biometric healthcare data of a patient during a telemedicine video conference, the telemedicine video conference having a data stream to communicate telemedicine audio/visual data between the patient computing system and a remote healthcare provider computing system encode the biometric healthcare data as a video stream; and transmit the biometric healthcare data video stream from the patient computing system to the remote healthcare provider computing system.

13. The non-transitory, computer-readable storage medium of claim 12 further comprising instructions to: composite the biometric healthcare data video stream and a telemedicine video conferencing video stream to generate a composite video stream.

14. The non-transitory, computer-readable storage medium of claim 13 wherein the biometric healthcare data of the patient includes real time biometric healthcare data captured by an infrared camera during the telemedicine video conference and the real time biometric healthcare data is encoded as a dynamic machine-readable code.

15. The non-transitory, computer-readable storage medium of claim 13 further comprising instructions to display the composite data stream at the remote healthcare provider computing system as a background of a presentation of the telemedicine video conference.

17