WO2022076776A1

WO2022076776A1 - Techniques for determining characteristics of dialysis access sites using image information

Info

Publication number: WO2022076776A1
Application number: PCT/US2021/054106
Authority: WO
Inventors: Fansan Zhu; Peter Kotanko
Original assignee: Fresenius Medical Care Holdings Inc
Current assignee: Fresenius Medical Care Holdings Inc
Priority date: 2020-10-08
Filing date: 2021-10-08
Publication date: 2022-04-14
Anticipated expiration: 2023-04-08

Abstract

A dialysis access site monitoring system may generate a treatment recommendation for treating a condition of an access site based on a video of the access site. The dialysis access site monitoring system may operate to generate video information based on a video of a dialysis access site of a patient, determine change in the number of pixels (CNP) information of the video information, the CNP information associated with movement of a skin surface of the patient due to blood flow through the dialysis access site, determine frequency domain information of the CNP information, determine a maximum-to-median power (M2) value of the frequency domain information, and determine at least one access site characteristic based on the M2 value. The at least one access site characteristic may include access blood flow (ABF) and/or a prediction or estimation of stenosis. Other embodiments are described.

Description

TECHNIQUES FOR DETERMINING CHARACTERISTICS OF DIALYSIS ACCESS SITES USING IMAGE INFORMATION

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit to U.S. Provisional Application No. 63/089,2116, filed October 8, 2020, the entire contents of which are incorporated herein by reference in their entirety.

FIELD

[0002] The disclosure generally relates to processes for examining physical characteristics of a portion of patient based on one or more images of the portion, and, more particularly, to techniques for using video information to assess a characteristic of a dialysis access site of a patient.

BACKGROUND

[0003] Dialysis treatment requires access to the patient circulatory system via a dialysis access site in order to process patient blood using a dialysis treatment unit. For peritoneal dialysis (PD), the dialysis access site may be via a catheter. Hemodialysis (HD) treatment requires access to blood circulation in an extracorporeal circuit connected to the main cardiovascular circuit of the patient through a vascular or arteriovenous (AV) access. Typical HD access types may include arteriovenous fistula (AVF) and arteriovenous graft (AVG). During an HD treatment, blood is removed from the vascular access by an arterial needle fluidly connected to the extracorporeal circuit and provided to an HD treatment unit. After processing via the HD treatment unit, the blood is sent back to the vascular access through a venous needle and back into the patient cardiovascular circuit.

[0004] Accordingly, the health of the access site of a patient is of primary importance to the efficacy of the dialysis treatment. For example, a vascular access should be capable of providing a sufficient access blood flow (ABF) for HD treatment and should be free of serious complications, such as severe pain and/or swelling, stenosis, thrombosis, aneurysms, and/or the like. Thermal dilution is generally considered to be the most accurate ABF measurement technique. However, this is an invasive method that cannot typically be applied in a clinical routine. Doppler ultrasound (DU) is a non-invasive method that can be used to identify various access site characteristics, such as stenosis and thrombosis. However, DU requires a specifically trained operator. In addition, DU is time consuming and costly, which has prevented its widespread routine use. Accordingly, conventional access site monitoring techniques require specialized equipment and/or processes performed by trained, knowledgeable healthcare professionals, making them inefficient and cost-prohibitive.

[0005] It is with respect to these and other considerations that the present improvements may be useful.

SUMMARY

[0006] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to necessarily identify key features or essential features of the claimed subject matter, nor is it intended as an aid in determining the scope of the claimed subject matter.

[0007] In one embodiment, an apparatus may include at least one processor and a memory coupled to the at least one processor. The memory may include instructions that, when executed by the at least one processor, may cause the at least one processor to generate video information based on a video of a dialysis access site of a patient, determine change in the number of pixels (CNP) information of the video information, the CNP information associated with movement of a skin surface of the patient due to blood flow through the dialysis access site, determine frequency domain information of the CNP information, and determine FMAX as a maximum amplitude in the frequency domain and FMIN as a minimum amplitude in the frequency domain, wherein FMAX is associated with a frequency of a signal of a driving force of main blood flow in a regional area and FMIN is associated with turbulent blood flow.

[0008] In some embodiments of the apparatus, the instructions, when executed by the at least one processor, may cause the at least one processor to determine at least one access site characteristic based on FMAX or FMIN. In various embodiments of the apparatus, the at least one access site characteristic may include a stenosis estimation.

[0009] In some embodiments of the apparatus, wherein Fmax may approach a frequency of heart rate (FHR =1/HR) with no turbulent flow. In various embodiments of the apparatus, the instructions, when executed by the at least one processor, may cause the at least one processor to determine AF as FMin - FMax. In exemplary embodiments of the apparatus, the instructions, when executed by the at least one processor, may cause the at least one processor to determine a stenosis estimation based on AF.

[0010] In some embodiments of the apparatus, the instructions, when executed by the at least one processor, may cause the at least one processor to split frequency domain information into a plurality of subgroups based on patient characteristics. In various embodiments of the apparatus, the patient characteristics may include at least one of age, gender, BMI, and skin tone.

[0011] In exemplary embodiments of the apparatus, the instructions, when executed by the at least one processor, may cause the at least one processor to perform a regression analysis on each of the plurality of groups to determine an access fluid rate (Qa) for each group.

[0012] In some embodiments of the apparatus, the instructions, when executed by the at least one processor, may cause the at least one processor to determine a maximum-to-median power (M2) value of the frequency domain information, and determine at least one access site characteristic based on the M2 value.

[0013] In one embodiment, a method of determining a stenosis estimation may include generating video information based on a video of a dialysis access site of a patient, determining change in the number of pixels (CNP) information of the video information, the CNP information associated with movement of a skin surface of the patient due to blood flow through the dialysis access site, determining frequency domain information of the CNP information, and determining FMAX as a maximum amplitude in the frequency domain and FMIN as a minimum amplitude in the frequency domain, wherein FMAX is associated with a frequency of a signal of a driving force of main blood flow in a regional area and FMIN is associated with turbulent blood flow.

[0014] In some embodiments of the method, the method may include determining at least one access site characteristic based on FMAX or FMIN. In various embodiments of the method, the at least one access site characteristic may include a stenosis estimation.

[0015] In some embodiments of the method, wherein Fmax approaches a frequency of heart rate (FHR =1/HR) with no turbulent flow. In various embodiments of the method, the method may include determining AF as FMin - FMHX. In exemplary embodiments of the method, the method may include determining a stenosis estimation based on AF. [0016] In some embodiments of the method, the method may include splitting frequency domain information into a plurality of subgroups based on patient characteristics. In various embodiments of the method, the patient characteristics may include at least one of age, gender, BMI, and skin tone.

[0017] In exemplary embodiments of the method, the method may include performing a regression analysis on each of the plurality of groups to determine an access fluid rate (Qa) for each group. In various embodiments of the method, the method may include determining a maximum-to-median power (M2) value of the frequency domain information, and determine at least one access site characteristic based on the M2 value.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By way of example, specific embodiments will now be described, with reference to the accompanying drawings, in which:

[0019] FIG. 1 illustrates a first exemplary operating environment in accordance with the present disclosure;

[0020] FIG. 2 illustrates a second exemplary operating environment in accordance with the present disclosure;

[0021] FIG. 3 illustrates images of an access site in accordance with the present disclosure;

[0022] FIG. 4 illustrates a graph of frequency domain information in accordance with the present disclosure;

[0023] FIG. 5 illustrates a first logic flow in accordance with the present disclosure;

[0024] FIG. 6 illustrates a second logic flow in accordance with the present disclosure;

[0025] FIG. 7 depicts the capture of a video of a target area in accordance with the present disclosure;

[0026] FIG. 8 illustrates a first graph of predicted access site flow rates versus measured access flow rates in accordance with the present disclosure;

[0027] FIG. 9 illustrates a second graph of predicted access site flow rates versus measured access flow rates in accordance with the present disclosure; and

[0028] FIG. 10 illustrates a third graph of predicted access site flow rates versus measured access flow rates in accordance with the present disclosure; [0029] FIG. 11 illustrates a graph of measured access site flow rates versus the change in the number of pixels (CNP) in accordance with the present disclosure;

[0030] FIG. 12 illustrates a decision tree in accordance with the present disclosure consisting of classification and regression processes;

[0031] FIG. 13 illustrates a flow diagram of a flow-based process in accordance with the present disclosure consisting of using amplification code to determine access blood flow (ABF);

[0032] FIG. 14 illustrates a first process for splitting patients into groups in accordance with the present disclosure;

[0033] FIG. 15 illustrates first results of multiple regression analysis in accordance with the present disclosure;

[0034] FIG. 16 illustrates a first correlation between measurement of Qa by clinical reference and Qa calculated in accordance with the present disclosure;

[0035] FIG. 17 illustrates a flow diagram of a flow-based process in accordance with the present disclosure for determining ABF;

[0036] FIG. 18 illustrates a second process for splitting patients into groups in accordance with the present disclosure;

[0037] FIG. 19 illustrates second results of multiple regression analysis in accordance with the present disclosure;

[0038] FIG. 20 illustrates a second correlation between measurement of Qa by clinical reference and Qa calculated in accordance with the present disclosure;

[0039] FIG. 21 shows a relation between percent stenosis and a change in AF between pre- and post-intervention;

[0040] FIG. 22 shows a difference between Fmax and FHR associated with % stenosis preintervention;

[0041] FIG. 23 shows a degree of AVF stenosis related with a change in Fmin;

[0042] FIGS. 24 and 25 depict AF associated with Qa pre- and post-intervention, respectively; and

[0043] FIG. 26 illustrates an embodiment of a computing architecture in accordance with the present disclosure. DETAILED DESCRIPTION

[0044] The present embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which several exemplary embodiments are shown. The subject matter of the present disclosure, however, may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and willfully convey the scope of the subject matter to those skilled in the art. In the drawings, like numbers refer to like elements throughout.

[0045] Dialysis treatment requires at least one dialysis access site for accessing the circulatory system of a patient. For example, hemodialysis (HD) may use an access site that includes an arteriovenous (AV) fistula (AVF), AV graft (AVG), or an HD catheter. An AVF is an artery surgically connected to a vein, while an AVG is a surgically placed conduit of synthetic material connecting an artery to a vein. For HD, AVF is the preferred modality.

[0046] In order to deliver enough blood through the extracorporeal circuit, the AVF must deliver an access blood flow (ABF) above a threshold rate, for example, several hundred milliliters (mL) per minute. In certain patients, such as chronic kidney disease (CKD) and/or end-stage renal disease (ESRD) patients, a sufficient ABF is often hindered by the development of access site complications, including AVF stenosis and thrombosis. Such complications may lead to inadequate ABF and/or AVF failure. Patients who develop AVF failure may be required to undergo various types of vascular intervention, which adds burden and costs to the patient, as well as the potential to introduce new health risks.

[0047] ABF monitoring and pre-emptive repair of sub-clinical AVF stenosis has been shown to reduce the rate of thrombosis and other complications and prolong the functional life of mature AVFs. For example, the measurement of ABF may be used to assess the development of AVF stenosis and other complications. As described above, thermal dilution is generally considered to be the most accurate ABF measurement technique. However, thermal dilution is an invasive method that cannot typically be applied in a typical HD clinical setting. Doppler ultrasound (DU) is a non-invasive method that can be used to identify various access site characteristics, such as stenosis and thrombosis. However, DU requires specialized equipment and a trained, knowledgeable operator in order to be effective. In addition, DU is time consuming and costly, which has prevented its wide-spread routine use. Accordingly, conventional access site monitoring techniques require specialized equipment and/or processes performed by trained, knowledgeable healthcare professionals, making them inefficient, inconvenient, and cost- prohibitive.

[0048] Accordingly, some embodiments provide an access site analysis process operative to perform a non-invasive, contact-free, and easy-to-use technique that may allow the clinical staff, the patient, and/or a caregiver to assess access site characteristics, including AVF ABF, stenosis, thrombosis, and/or the like. In some embodiments, the access site analysis process may be or may include a video imaging process (VIP) operative to access a plurality of images (for instance, a video) of an access site of the patient. For example, a patient or caregiver may take a video of the AVF region of the patient using a smartphone or other computing/imaging device and provide the video to an access site analysis system. A region of interest (ROI) in the video may be analyzed to determine changes in the number of pixels (CNP) of elements or portions of the access site over the course of time, such as over successive frames of a video. The CNP information may indicate access site motion, for instance, AVF motion due to ABF through AVF. For example, the greater the change in CNP, the greater the motion of the AVF and, therefore, the greater the ABF, and vice versa. The VIP process may be used to estimate ABF and/or predict stenosis.

[0049] The access site analysis process may convert the CNP information into a time series that may be transferred to the frequency domain, for instance, via a Fourier transfer process. The frequency domain information may be analyzed to determine a maximum waveform at a specific frequency in the frequency domain (MAX) and the median of all frequency ranges (MED). The MAX and MED may be used to determine the maximum-to-median power (M2) via calculating the square of the ratio of MAX to MED: M2 = (MAX/MED)². In some embodiments, access site information may include historical or model information indicating access site characteristics associated with various M2 values or ranges of values. The access site information may be patient-specific and/or population-specific. The M2 determined for a patient may be compared with the model information to determine at least one access site characteristic. For example, for Patient A, an M2 value of X may correlate to an estimated ABF of Y (for example, determined directly or via further processing, such as a regression analysis). In another example, for Patient B, an M2 value below a threshold may indicate complication Z (for instance, thrombosis). Embodiments are not limited in this context. [0050] In some embodiments, the access site analysis process may be used to remotely monitor, analyze, trend, and/or the like a patient’s access site by using a combination of digital imaging, trending, intervention and outcome information to improve the longevity and care of a patent and their access site. In some embodiments, the access site analysis process may be an internetbased, Software-as-a-Service (SaaS), and/or cloud-based platform that may be used by a patient or a healthcare team to monitor patients clinical care and can be used to provide expert third- party assessments, for example, as a subscription or other type of service to healthcare providers. [0051] For example, the access site analysis process may operate in combination with a “patient portal” or other type of platform that a patient and healthcare team may use to exchange information. For instance, dialysis treatment centers mange in-home patients who receive treatment in their own home and in-center patients who receive treatment at a treatment center. In-home patients may take a video of their access site, such as an AVF site, AVG site, and/or the like, using a smartphone or other personal computing device on a periodic basis (for instance, daily, weekly, monthly, and/or the like) or as necessary (for instance, based on the appearance and/or change in the condition of the access site). A video of an access site may include skin in the area adjacent, proximal, in contact with, or otherwise associated with the access site. The video may be uploaded to a patient portal or other platform and routed to a dialysis access site analysis system operative to perform the access site analysis process according to some embodiments. Similarly, videos of the access sites of in-center patients may be taken by the patient and/or clinical staff and uploaded to the patient portal for access by the access site analysis system.

[0052] In some embodiments, patient videos may be stored in a repository or other database, including, without limitation, a healthcare information system (HIS), electronic medical record (EMR) system, and/or the like. Videos in the repository may be catalogued and indexed by patient including key clinical information, demographics, medical history, and/or the like to be processed by the access site analysis system at a patient level and/or a population level. Use of patient image information, for example, at a population-level, may require de-identification of protected health information (PHI) and/or other information capable of identifying a patient according to required regulations, protocols, and/or the like, such as Health Insurance Portability and Accountability Act of 1996 (HIPAA). [0053] In some embodiments, the access site analysis system may provide an assessment or diagnosis and/or one or more treatment recommendations based on the M2 value determined based on the access site video, which may be provided to a healthcare team. The healthcare team may then review the recommendations and either accept, decline, or revise the intervention for the patient. Healthcare team interventions may be documented and stored in the repository on both a patient-level and a population-level so that they can be followed to monitor success rates and outcomes to provide further training data to computational models used according to some embodiments.

[0054] Therefore, processes, techniques, methods, and/or apparatuses according to some embodiments, including dialysis access site analysis processes and/or flow-based analysis processes according to some embodiments, may provide multiple technological advantages and technical features over conventional systems, including improvements to computing technology. One non-limiting example of a technological advantage may include examining access sites using an automated video imaging process (VIP). In some embodiments, a VIP may include processing of digital videos employing, for example, artificial intelligence (Al) and/or machine learning (ML) processes. Another non-limiting example of a technological advantage may include allowing remote analysis of a patient access site without requiring an in-person visual inspection and/or specialized test (for instance, thermal dilution, DU, and/or the like) by a healthcare professional, reducing or even eliminating the need for a visit to/from the healthcare professional by/to the patient. In a further non-limiting example of a technological advantage, access site analysis processes according to some embodiments may operate to generate an access site diagnosis (for instance, healthy, thrombosis, stenosis, access blood flow (ABF) information, and/or the like) determined based on video information of a patient access site. For example, computing technology may be improved by being programmed or operating according to some embodiments to allow a mobile computing form factor (such as a smart phone) to capture access site information (for instance, video of a patient access site) and determine, alone or in combination with another computer or software, access site information, such as ABF and/or stenosis estimations. Other technological advantages would be understood by a person of skill in the art in view of the teachings of the present disclosure.

[0055] The technological advantages of some embodiments may include improvements to a computing device and/or system. In one non-limiting example, a computing device configured according to some embodiments may be capable of performing image-based analysis of patient access sites, which is not possible using conventional computing devices. The technological advantages of some embodiments may include improvements within a technical field. In one non-limiting example, processes configured according to some embodiments may provide improvements in the technical fields of, for instance, dialysis treatment and/or healthcare delivery by providing technology-based efficient and accurate methods for analysis of patient access sites that do not require in-patient visits. Embodiments are not limited in this context. [0056] In addition, some embodiments may include processes, such as a dialysis access site analysis processes and/or a flow-based analysis processes, that are integrated into at least one practical application. For example, a dialysis access site analysis process and/or a flow-based analysis process may be integrated into the practical application of determining a patient diagnosis, particularly a diagnosis of a patient access site. In another example, a dialysis access site analysis process and/or a flow-based analysis process may be integrated into the practical application of generating a treatment recommendation or plan to a patient, for example, based on the patient diagnosis. In a further example, a dialysis access site analysis process and/or a flowbased analysis process may be integrated into the practical application of providing medical treatment to a patient, for example, based on the patient diagnosis, such as treating an access site condition. Other integrations into practical applications are provided in the present disclosure. Embodiments are not limited in this context.

[0057] FIG. 1 illustrates an example of an operating environment 100 that may be representative of some embodiments. As shown in FIG. 1, operating environment 100 may include a dialysis access site analysis system 105. In various embodiments, dialysis access site analysis system 105 may include a computing device 110 communicatively coupled to network 170 via a transceiver 160. In some embodiments, computing device 110 may be a server computer or other type of computing device.

[0058] Computing device 110 may be configured to manage, among other things, operational aspects of an access site analysis process according to some embodiments. Although only one computing device 110 is depicted in FIG. 1, embodiments are not so limited. In various embodiments, the functions, operations, configurations, data storage functions, applications, logic, and/or the like described with respect to computing device 110 may be performed by and/or stored in one or more other computing devices (not shown), for example, coupled to computing device 110 via network 170 (for instance, one or more of client devices 174a-n). A single computing device 110 is depicted for illustrative purposes only to simplify the figure. Embodiments are not limited in this context.

[0059] Computing device 110 may include a processor circuitry that may include and/or may access various logics for performing processes according to some embodiments. For instance, processor circuitry 120 may include and/or may access an access site analysis logic 122 and/or image processing logic 124. Processing circuitry 120, access site analysis logic 122, image processing logic 124, and/or portions thereof may be implemented in hardware, software, or a combination thereof. As used in this application, the terms “logic,” “component,” “layer,” “system,” “circuitry,” “decoder,” “encoder,” “control loop,” and/or “module” are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution, examples of which are provided by the exemplary computing architecture 1100. For example, a logic, circuitry, or a module may be and/or may include, but are not limited to, a process running on a processor, a processor, a hard disk drive, multiple storage drives (of optical and/or magnetic storage medium), an object, an executable, a thread of execution, a program, a computer, hardware circuitry, integrated circuits, application specific integrated circuits (ASIC), programmable logic devices (PLD), digital signal processors (DSP), field programmable gate array (FPGA), a system-on-a-chip (SoC), memory units, logic gates, registers, semiconductor device, chips, microchips, chip sets, software components, programs, applications, firmware, software modules, computer code, a control loop, a computational model or application, an Al model or application, an ML model or application, a proportional-integral- derivative (PID) controller, variations thereof, combinations of any of the foregoing, and/or the like.

[0060] Although access site analysis logic 122 is depicted in FIG. 1 as being within processor circuitry 120, embodiments are not so limited. For example, access site analysis logic 122, image processing logic 124, and/or any component thereof may be located within an accelerator, a processor core, an interface, an individual processor die, implemented entirely as a software application (for instance, an access site analysis application 150) and/or the like.

[0061] Memory unit 130 may include various types of computer-readable storage media and/or systems in the form of one or more higher speed memory units, such as read-only memory (ROM), random-access memory (RAM), dynamic RAM (DRAM), Double-Data-Rate DRAM (DDRAM), synchronous DRAM (SDRAM), static RAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, polymer memory such as ferroelectric polymer memory, ovonic memory, phase change or ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS) memory, magnetic or optical cards, an array of devices such as Redundant Array of Independent Disks (RAID) drives, solid state memory devices (e.g., USB memory, solid state drives (SSD) and any other type of storage media suitable for storing information. In addition, memory unit 130 may include various types of computer-readable storage media in the form of one or more lower speed memory units, including an internal (or external) hard disk drive (HDD), a magnetic floppy disk drive (FDD), and an optical disk drive to read from or write to a removable optical disk (e.g., a CD-ROM or DVD), a solid state drive (SSD), and/or the like.

[0062] Memory unit 130 may store various types of information and/or applications for an access site analysis process according to some embodiments. For example, memory unit 130 may store access site images 132, image information 134, patient information 136, access site characteristic information 138, access site model information 140, and/or an access site analysis application 150. In some embodiments, some or all of access site images 132, image information 134, patient information 136, access site characteristic information 138, access site model information 140, and/or an access site analysis application 150 may be stored in one or more data stores 172a-n accessible to computing device 110 via network 170. For example, one or more of data stores 172a-n may be or may include a HIS, an EMR system, a dialysis information system (DIS), a picture archiving and communication system (PACS), a Centers for Medicare and Medicaid Services (CMS) database, U.S. Renal Data System (USRDS), a proprietary database, and/or the like.

[0063] In some embodiments, access site analysis logic 122, for example, via image processing logic 124 and/or access site analysis application 150, may operate to analyze access site images 132 of access sites to determine access site characteristics, such as ABF and/or the presence of access site conditions, such as thrombosis. Access site images 132 may be or may include an electronic file that includes a picture and/or video of an access site and/or other portions of a patient. The images may be stored as image or video files such as *.mp3, *.mp4, *.avi,*.jpg, *.png, *.bmp, *.tif, and/or the like. In some embodiments, the access site images 132 may include a video of an access site taken over an image time period (for instance, about 60 seconds to about 120 seconds). In other embodiments, the access site images 132 may include a series of successive images taken over an image time period at successive intervals, such as about every 5 seconds to about every 5 minutes.

[0064] In various embodiments, image processing logic 124, for example, via access site analysis application 150, may operate to process images to generate image information 134. In reference to a video, image information 134 may be or may include video information. In general, image information 134 may include any information that may be obtained or determined from access site images 132. Non-limiting examples of image information may include color information, intensity information, motion information, and/or the like. For example, image information may include information associated with changes between frames of a video, such as variation in color, intensity, CNP, and/or the like.

[0065] In addition, image information 134 may include information resulting from processing of an image, video, frame, and/or other image information 134 by image processing logic 124. For example, all or some of the motion exhibited by an access site, such as an AVF, may be difficult if not impossible to see with the human eye from raw image data. In some embodiments, access site motions may be amplified to increase the scale of the motion to simplify the delineation between noise and movement in the video. The motion amplification may be achieved by, for example, performing Eulerian video magnification on the captured video. In some embodiments, Eulerian video magnification may be performed according to one or more of the methods and/or variations thereof described in Wu et al., “Eulerian Video Magnification for Revealing Subtle Changes in the World,” ACM Trans. Graph. Vol. 31, pp.1-8, 2012; U.S. Patent Application Publication No. 2014/0072190, entitled “Linear-Based Eulerian Motion Modulation,” and/or U.S. Patent Application Publication No. 2017/0119258, entitled “Method and Apparatus of Assessment of Access Flow in Hemodialysis Patients by Video Imaging Processing,” all of which are incorporated by reference as if fully set forth herein.

[0066] In another example, image information 134 may include transformations of data generated based on access site images 132, including, without limitation, linear transforms, time series or domain information, frequency domain information/graphs (see, for example, FIG. 4), CNP, M2 information, regression analysis information, and/or the like. Embodiments are not limited in this context. [0067] A patient, healthcare provider, caretaker, or other individual may capture access site images (or videos) 132 using any capable device, such as a smartphone, tablet computing device, laptop computing device, personal computer (PC), camera, digital camera, video camera, and/or the like. A user, such as the patient and/or healthcare professional, may send, transmit, upload, or otherwise provide access site videos 132 to access site analysis system 105 via a client device 174a communicatively coupled to computing device 110 via network 170. For example, access site analysis application 150 may be or may include a website, internet interface, portal, or other network-based application that may facilitate uploading digital access site videos 132 for storage in memory unit 130 and/or data stores 172a-n. In some embodiments, a patient client device 174a-n may operate a client application (for instance, a mobile application or “app”) operative to communicate with access site analysis application 150 for providing access site videos 132. In some embodiments, a patient may upload digital access site videos 132 via a patient portal of a dialysis clinic or other healthcare provider. Access site analysis application 150 may be communicatively coupled to the patient portal to receive images therefrom. Embodiments are not limited in this context.

[0068] In addition, a patient or healthcare provider may provide image information 134 describing characteristics of the access site. In general, image information 134 may include any type of textual, audio, visual, and/or the like data outside of an access site video 132 that may indicate characteristics of the access site. For example, image information 134 may include descriptions regarding day/time of video capture, pain, swelling, color, size, ABF information, duration of a condition or characteristic, age of access site, type of access site, patient vitals, and/or the like. In various embodiments, image information 134 may be associated with one or more access site videos 132, for example, as metadata, related within one or more medical record entries, and/or the like. For instance, access site analysis application 150 may create a record for an access site video 132 that includes or refers to associated image information 134. In this manner, access site analysis application 150 may access information describing and/or providing context to an access site video 132 (for instance, a record for Patient Video 1 for Patient A, indicating that the patient has had the AVF for X months and has had thrombosis for Y weeks). [0069] In some embodiments, patient information 136 may include information associated with a patient corresponding to an access site video 132. Non-limiting examples of patient information 136 may include name and address information, patient identifiers, dialysis information, prescription information, medical conditions, healthcare provider information, access site information, and/or the like. In some embodiments, patient information 136 may include access site images 132 associated with the patient, for instance, as part of a medical record. Embodiments are not limited in this context.

[0070] Access site analysis application 150 may analyze access site images 132 and/or image information 134 to determine access site characteristic information 138. In general, access site characteristic information 138 may include a diagnosis, classification, categorization, access site features, or other analysis result determined from analyzing an access site image 132 and/or image information 134. For example, access site characteristic information 138 may include access site features of an access site image 132, including, without limitation, blood flow information (for instance, ABF), color, size, shape, access site elements (for instance, scabbing, bleeding, and/or the like), and/or other information that may be discerned from analyzing an access site image 132. In some embodiments, access site characteristic information 138 may include an estimated ABF value or rate (for instance, X ml/min). In various embodiments, access site characteristic information 138 may include a diagnosis or other classification of an access site, such as a healthy diagnosis, a grade or other classification level, indication of the presence and/or severity of an abnormality or failure (for instance, thrombosis), and/or the like. [0071] In some embodiments, access site analysis application 150 may use access site model information 140 to determine access site characteristic information 138. In general, access site model information 140 may be or may include historical information of the patient and/or a patient population that may be used to determine access site characteristic information 138. For example, access site model information 140 may include M2 values of a population of patients. As described in more detail herein, access site analysis application 150 may determine an M2 value for the patient via analyzing an access site video 132. Access site analysis application 150 may lookup the M2 value in a lookup table, computational model, database, and/or the like to determine an ABF rate corresponding to the M2 value. For example, Video 1 of Patient A may be determined to have an M2 value of X, which corresponds to an ABF rate of Y ml/min. In another example, Video 2 of Patient B may be determined to have an M2 value of T, which corresponds to an ABF rate of U ml/min based on M2 values of a patient population having characteristics that correspond to Patient B. [0072] In some embodiments, access site model information 140 may include a computational model for determining access site characteristic information 138 (for instance, ABF rates) based on image information 134 (for instance, M2 values) and/or patient information 136. For example, an Al and/or ML model may be used to determine an ABF rate for a patient corresponding to a determined M2 value and patient characteristics. In various embodiments, access site model information 140 may include models specifically for analyzing images. For example, an Al and/or ML model may be trained to determine image information 134, such as CNP, via analyzing an access site video 132. Non-limiting examples of computational models may include an ML model, an Al model, a neural network (NN), an artificial neural network (ANN), a convolutional neural network (CNN), a deep learning (DL) network, a deep neural network (DNN), a recurrent neural network (RNNs), combinations thereof, variations thereof, and/or the like. Embodiments are not limited in this context.

[0073] FIG. 2 illustrates an example of an operating environment 200 that may be representative of some embodiments. As shown in FIG. 2, operating environment 200 may include a patient computing device 274, such as a smartphone, a tablet computing device, a portable computing device, and/or the like. Computing device 274 may execute an access site application 250. In some embodiments, access site application 250 may be or may include a mobile application, client application, web-based application, email application and/or the like for interacting (for instance, directly or via a patient portal) with a dialysis access site analysis system 205 configured according to various embodiments.

[0074] A user may capture or otherwise access a video 232 of an access site or other portion of the patient. For example, a user may take one or more pictures and/or a video of the access site via camera 202 (see, for instance, FIG. 7). A user may capture video 232 of an area 216 of a portion 212 of patient 210, such as an arm 212 portion having a target area 214. In some embodiments, target area 214 may be an area targeted for videoing that may indicate blood flow according to some embodiments. In various embodiments, target area 214 may include all or some of an access site. In other embodiments, target area 214 may include an area completely or partially outside of the access site, such as an area above the access site, below the access site, to the side of the access site, or otherwise adjacent to the access site. Accordingly, reference to a video or still image of an access site herein may refer to include video images of all, some, or none of an access site and/or all, some, or none of an area (for instance, skin) above, below, besides, or otherwise adjacent to an access site. Embodiments are not limited in this context. [0075] Area 216 on patient 210 may have various dimensions. For example, patient 210 may capture video 232 of an area about 10 cm in length and 5 cm wide. In various embodiments, area 216 may have a length and/or width of about 1 cm, about 2 cm, about 5 cm, about 10 cm, about 15 cm, about 20 cm, and any value or range between any two of these values (including endpoints). Camera 202 may capture video 232 at an image distance from portion 212, such as about 1 cm, about 2 cm, about 5 cm, about 10 cm, about 15 cm, about 20 cm, about 25 cm, about 30 cm, 100 cm, any value or range between any two of these values (including endpoints), and/or any other distance. Video 232 may be captured for an image or video time period. In various embodiments, video time period may be about 5 seconds, about 10 seconds, about 30 seconds, about 1 minute, about 2 minutes, about 5 minutes, about 10 minutes, about 30 minutes, about 1 hour, any value or range between any two of these values (including endpoints), and/or any other duration. In some embodiments, video 232 may be captured at a resolution of below about 720 p, about 720p, below about 1080p, about 1080p, above about 1080p, and any value or range between any two of these values (including endpoints). In exemplary embodiments, video 232 may be captured at various frame speeds, such as about 10 frames-per-second (fps), about 20 fps, about 30 fps, about 50 fps, and any value or range between any two of these values (including endpoints). In general, the size of area 216, the image distance, the image time period, the resolution, the frame speed, and/or any other video characteristics may be of sufficient values to capture access site 214 for analysis according to some embodiments.

[0076] In some embodiments, access site application 250 may allow a user to enter patient information 237 describing video 232 and/or other personal characteristics. In some embodiments, access site application 250 may provide text boxes, check boxes (for example, to indicate the presence of a condition), selection objects, or other graphical user interface (GUI) objects for entering patient information 236, including access site description information. In some embodiments, access site application 250 may facilitate the capture of video 232. For example, a user may open access site application 250 and access site application 250 may provide an image capture function (for instance, using camera 202 and/or display screen of computing device 274), such as providing a target area or bounding box to be placed around access site 214 for proper video capture of access site 214. [0077] Video 232 may be uploaded or otherwise made available to access site system 205 according to some embodiments, along with other information, such as patient information 236, image information (for instance, patient description of the image, camera information, resolution information, raw image data, and/or the like). In various embodiments, video 232 may be transmitted to access site analysis system 205 associated with patient information 236, image information, and/or the like, for instance, as a record or other data structure. In various embodiments, access site analysis system 205 may process video 232 according to some embodiments to generate access site characteristics 238 such as, for example, an ABF estimate. [0078] FIG. 3 illustrates images of an access site in accordance with the present disclosure. For example, image 322a may be a first frame (Fi) and image 322n may be a subsequent frame (Fn) of a video. Image 322n may be a certain amount of time after image 322n, depending on the frame speed (i.e., fps) and/or the amount of time between image analyses by access site analysis process (for example, about 10 milliseconds, about 50 milliseconds, about 1 second, about 2 seconds, about 5 seconds, about 10 seconds, about 30 seconds, about 1 minute, about 2 minutes, about 5 minutes, about 30 minutes, and any range or value between any two of these values (including endpoints)). Images 322a-n may be images of an area 216 of a portion 212 of a patient, for example, including an access site 214 and skin surrounding or adjacent to access site. In some embodiments, area 216 may be a region of interest (ROI), a bounding box, or other visual element. In other embodiments, area 216 may not be a part of image. In some embodiments, successive frames may be analyzed. In other embodiments, frames at various intervals (for instance, every X frames) may be analyzed.

[0079] Access site analysis process may determine access site 214 in images 322a. In various embodiments, access site analysis process, for example, via image processing logic, may determine change in the number of pixels (CNP) of access site 214 between images. In general, CNP may determine a change in the number of pixels with respect to a characteristic of the pixels, such as a visual property of the pixels. Non-limiting examples of a visual property may include color, brightness, contrast, brilliance, saturation, sharpness, tint, intensity, and/or the like. For example, determining CNP may include determining, pixel-by-pixel, a plurality of intensity differences between each pixel (or each pixel in an area of interest) in consecutive frames.

[0080] As shown in FIG. 3, the size of an area associated with access site 214 may increase between image 322a and image 322n. The increase in the area may be due to various characteristics associated with the function of access site 214, for example, due to fluid dynamics beneath the skin of access site 214 and/or adjacent to access site 214. For example, blood flow may change the color of the skin of access site 214 and/or adjacent to access site 214; blood flow may change the size of access site 214 and/or an opening thereof; blood flow may cause motion of access site 214 and/or skin adjacent to access site (i.e., access site 214 and/or skin adjacent to access site 214 may be displaced, flex, or pulsate based on blood flow); combinations thereof, and/or the like. The changes in the physical characteristics of access site 214 and/or physical characteristics skin that is a part of or adjacent to access site 214 may be detectable via determining changes in pixels of images of access site and/or skin that is a part of or adjacent to access site. For example, the sequence of frames Fl-Fn of images 232a-n may capture motion by the skin in the area of interest as illustrated by AV fistula motion 316. In another example, the sequence of frames Fl-Fn of images 232a-n may capture changes in the color of or other changes in a characteristic of the skin in the area of interest as illustrated by AV fistula motion 316 [0081] Referring to image 322n, therein is depicted access site 214 that has increased in size or a characteristic (for instance, a skin color) that has changed in size (the dotted portion within access site 214 of image 322n is access site 214 in image 322a to show the increase in size). For example, with a pulse of blood flow, the color of skin of access site 214 and/or adjacent to access site 214 may change in color and/or intensity of color (which may not be perceptible to the human eye). The rate of blood flow may be related to the amount of change of pixels in images 322a-n. For example, (the patterned area of) access site 214 in image 322a may be, for example, a certain intensity of a color or color of interest (red, for example, when amplified). An illustrative set of pixels 314 in image 322a are not colored (or are below a threshold intensity). In image 322n, the red color and/or intensity (or other characteristic) thereof has increased in area, such that 4 of the set of pixels 314 are now red. Accordingly, between images 322a and 322n, a CNP of 4 has occurred. The CNP may be determined based on various characteristics, such as pixels changing from one color (i.e., normal skin color of the patient) to another (i.e., a color generated when blood flows through access site) and/or changes in intensity from a first intensity (below a threshold) to a second intensity (above a threshold), affected by blood flow through access site. As described in more detail herein, the CNP may be related to the rate of blood flow (for instance, ABF rate). For example, the greater the blood flow rate, the greater CNP, and vice versa. In another example, an amount or pattern of CNP may indicate an access site failure.

[0082] The motion exhibited by the skin based on various fluid dynamics beneath the skin due to blood flow may be very small. Amplifying the small motion exhibited by the skin according to some embodiments better enables delineation between noise and motion in the video that is attributable to the movement of the skin. Any of a variety of methods may be employed to amplify the motion in the captured video, for example, as described in U.S. Patent Application Publication No. 2014/0072190, entitled “Linear-Based Eulerian Motion Modulation,” and/or U.S. Patent Application Publication No. 2017/0119258, entitled “Method and Apparatus of Assessment of Access Flow in Hemodialysis Patients by Video Imaging Processing.” Embodiments are not limited in this context. For example, other image processing techniques may be employed to obtain video information from images 322a-n.

[0083] Accordingly, in some embodiments, movement of the skin surfaces of access site and/or adjacent to access site induced by ABF may be determined by quantitation of the frame-to-frame changes in CNP. In some embodiments, CNP from a sequence of image frames in a digital video may be transferred to a time series, for example, as described in Nakajima et al. “Detection of Apparent Skin Motion Using Optical Flow Analysis: Blood Pulsation Signal Obtained from Optical Flow Sequence,” Rev. Sci. Instrum. Vol. 68, pp. 1331-1336, 1997 and/or Zhu et al., “Estimation of Arterio-Venous Access Blood Flow in Hemodialysis Patients Using Video Image Processing Technique,” Conf Proc IEEE Eng Med Biol Soc. pp:207-210, 2016. This time series can then be transferred into the frequency domain for example, via applying fast Fourier transform (FFT) to the time series. In this signal, sample frequency (Fs) is the frame per second (fps) and time is total number of frames divided by Fs. The variation of baseline of signal in time series may be associated with patient physical characteristics, such as skin color.

Accordingly, adjustments may be performed to adjust the baseline based on patient characteristics, such as skin color, skin tone, race, and/or the like.

[0084] FIG. 4 illustrates a graph of frequency domain information in accordance with the present disclosure. As shown in FIG. 4, video information from a video of an access site may be transformed via image processing into a frequency domain. Graph 405 depicts frequency 415 vs. magnitude 410. In various embodiments, the change in the amplitude in main components or its ratio in the frequency domain may correlate with the change in ABF. For example, the frequency domain information may be analyzed to determine a maximum waveform at a specific frequency in the frequency domain (MAX) and the median of all frequency ranges (MED). The MAX and MED may be used to determine the maximum-to-median power (M2) via calculating the square of the ratio of MAX to MED: M2 = (MAX/MED)². In various embodiments, M2 may be used as a predictor of ABF in regression models. For example, regression models with ABF as the dependent variable may include, without limitation, as independent variables one or more of M2, age, race, and weight.

[0085] Included herein are one or more logic flows representative of exemplary methodologies for performing novel aspects of the disclosed architecture. While, for purposes of simplicity of explanation, the one or more methodologies shown herein are shown and described as a series of acts, those skilled in the art will understand and appreciate that the methodologies are not limited by the order of acts. Some acts may, in accordance therewith, occur in a different order and/or concurrently with other acts from that shown and described herein. For example, those skilled in the art will understand and appreciate that a methodology could alternatively be represented as a series of interrelated states or events, such as in a state diagram. Moreover, not all acts illustrated in a methodology may be required for a novel implementation. Blocks designated with dotted lines may be optional blocks of a logic flow.

[0086] A logic flow may be implemented in software, firmware, hardware, or any combination thereof. In software and firmware embodiments, a logic flow may be implemented by computer executable instructions stored on a non-transitory computer readable medium or machine readable medium. The embodiments are not limited in this context.

[0087] FIG. 5 illustrates an embodiment of a logic flow 500. Logic flow 500 may be representative of some or all of the operations executed by one or more embodiments described herein, such as computing device 110. In some embodiments, logic flow 500 may be representative of some or all of the operations of an access site analysis process according to some embodiments.

[0088] At block 502, logic flow 500 may receive a video of a patient access site. For example, an access site analysis system may receive a video of an access site (and surrounding or adjacent skin) of a patient. Logic flow 500 may generate video information at block 504. For example, an image processing logic may analyze characteristics of pixels in frames of the video to determine information such as color information, intensity information, and/or the like for pixels within the frames. At block 506, logic flow 500 may generate CNP information. For example, CNP may be determined based on the video information indicating a change in a characteristic of pixels. For example, CNP may include a count or other measurement of pixels experiencing a change from a first color to a second color, a change from a first intensity to a second intensity, falling below an intensity threshold, going above an intensity threshold, and/or the like. Embodiments are not limited in this context. In general, the pixel factors for determining CNP may include factors that are associated with determining access site blood flow characteristics. [0089] Logic flow 500 may generate frequency domain information at block 508. For example, the CNP information may be converted to a time series. The time series may be converted to the frequency domain, for example, via a Fourier analysis, such as FFT. Graph 405 of FIG. 4 illustrates an example of frequency domain information according to some embodiments. At block 510, logic flow 500 may determine an M2 value. For example, the frequency domain information may be analyzed to determine a maximum waveform at a specific frequency in the frequency domain (MAX) and the median of all frequency ranges (MED). The MAX and MED may be used to determine the maximum-to-median power (M2) via calculating the square of the ratio of MAX to MED: M2 = (MAX/MED)².

[0090] At block 512, logic flow 500 may determine an access site characteristic. For example, M2 may be used to look up an ABF rate in a lookup table and/or provided to a computational model to determine an ABF rate that corresponds to the M2 value (and any associated patient information). In various embodiments, M2 may be used as a predictor of ABF in regression models. For example, regression models with ABF as the dependent variable may include, without limitation, as independent variables one or more of M2, age, race, and weight.

[0091] In some embodiments, determining an access site characteristic may include a diagnosis of an access site, such as an access site failure (for instance, stenosis). For example, the ABF may be used as an indicator and/or predictor of an access site failure (alone or in combination with other factors, such as the physical condition of the access site).

[0092] FIG. 6 illustrates an embodiment of a logic flow 600. Logic flow 600 may be representative of some or all of the operations executed by one or more embodiments described herein, such as computing device 110. In some embodiments, logic flow 600 may be representative of some or all of the operations of an access site analysis process according to some embodiments. [0093] At block 602, logic flow 600 may receive video input. For example, a patient or healthcare professional may capture a video of a portion of the patient including an access site and surrounding skin. The video may be uploaded as a digital file to an access site analysis system configured according to some embodiments. Logic flow 600 may perform preprocessing at block 604. For example, the video may be processed to improve the signal-to- noise ratio, magnify or amplify relevant features (for instance, via Eulerian video magnification), and/or the like.

[0094] Logic flow 600 may convert the video to a time series at block 606. For example, an access site analysis process may convert the video to video information that includes a signal including a time series based on CNP between consecutive frames. For example, CNP for pixels in the video may be determined for a pixel characteristic, such as a color value or intensity. A time series may be determined for the CNP for the pixel characteristic over time as determined by analyzing frames of the video. At block 608, logic flow 600 may apply a band-pass filter to the time series video information. For example, the motion of interest may have a limited possible range of frequency values. Accordingly, signal components with frequency values above/below the range of interest are likely noise and may be excluded. In the example of a video of an AVF, the range of interest may be between a low threshold and a high threshold and a bandpass filter may be constructed that attenuates signal components, for example, below about 0.02 Hz (low threshold) and above about 2 Hz (high threshold). In some embodiments, the low threshold may be about 0.01 Hz to about 1 Hz and the high threshold may be about 2 Hz to about 10 Hz.

[0095] Logic flow 600 may take different paths to obtain an ABF value at block 660 or other determinations (for instance, an access site condition, such as an access site failure). A first path may include a “frequency domain” path starting at block 610. A second path may include a “time domain” or “amplitude” path starting at block 622. In some embodiments, an access site failure may be determined or predicted by a combination of time domain and frequency domain analysis. For example, regression analysis may be performed by two indepenent variables, the M2 in frequency domain and the amplitude in time domain to fit the dependent varible of degree of stenosis or other access problem.

[0096] At block 610, logic flow 600 may convert the time series to the frequency domain. For example, FFT may be applied to the time series at a specified interval to produce the frequency domain signals. For example, the specified interval may be about every 10 seconds. In other examples, the specified time interval may be about every 1 second, about every 2 seconds, about every 5 seconds, about every 10 seconds, about every 30 seconds, about every 60 seconds, and any value or range between any two of these values (including endpoints). Logic flow 600 may determine M2 from the frequency domain information at block 612. For example, frequency domain information may be video information generated from a video as plotted in graph 405 of FIG. 4. The frequency domain information may be analyzed to determine a maximum waveform at a specific frequency in the frequency domain (MAX) and the median of all frequency ranges (MED). The MAX and MED may be used to determine the maximum-to-median power (M2) via calculating the square of the ratio of MAX to MED: M2 = (MAX/MED)².

[0097] At block 614, logic flow 600 may perform a regression analysis. For example, one or more regression analyses based on one or more of the independent variables of M2, age, gender, and/or race and dependent variable of ABF may be performed. In some embodiments, ABF dependent variable information may be determined via a standardized, conventional process, such as a thermal dilution method (for instance, performed using a HVT100 endovascular flowmeter provided by Transonic® of Ithaca, New York, United States). Logic flow 600 may determine an estimated ABF rate based on the regression analysis at block 660.

[0098] At block 622, logic flow 600 may perform an amplitude analysis. For example, the time domain signal may include a waveform of CNP over time (or frames). The analysis of the amplitude of the waveform may provide changes in volume and pressure of the blood associated with the access site, such as access vessels. Logic flow 600 may perform a moving average filter at block 624. For example, a moving average filter may be applied to reduce the intervention from turbulent flow in the access site. At block 626, logic flow 600 may perform patient characteristics adjustments. For example, the variation of baseline of signal in time series may be associated with patient physical characteristics, such as skin color. Accordingly, adjustments may be performed to adjust the baseline based on patient characteristics, such as skin color, skin tone, race, and/or the like.

[0099] At block 628, logic flow 600 may determine a peak point. For example, a peak point may be detected which is related to the moving amplitude of blood volume. At block 630, logic flow 600 may perform a regression analysis. For example, line regression analysis may be performed to determine a relationship between blood flow and the peak point of CNP in time series signal. In some embodiments, moving average and peak detection may be used to identify the maximun displacement of the skin above the access site (for instance, AVF) from the time series signals.

CLINICAL STUDY: HD PATIENTS WITH AFV FAILURES

[0100] A Clinical Study of HD patients was performed using an access site analysis process according to some embodiments. The HD patients were identified as having AVF failures (stenosis) and were targeted for an endovascular intervention involving the following steps: 1) pre-intervention assessment; 2) balloon angioplasty; 3) post-intervention assessment. The ABF was measured pre- and post-intervention by thermodilution using an HVT100 device. In addition, X-ray images were done to identify the location of the stenosis. Using a smartphone, videos were captured of AVF and the skin above the AVF with patients in a sitting position. FIG. 7 depicts the capture of a video of a target area according to some embodiments. The patients’ race, gender, body mass, height, body mass index (BMI), and blood pressure were recorded.

[0101] The video-taped area had a length of about 10 cm and a width of about 5 cm. The distance between the smartphone and the AVF was approximately 25 cm. The video was recorded for 60 seconds with a 1080p resolution and 30 frames per second (fps).

[0102] The video images were processed according to the steps of logic flow 600 of FIG. 6. For example, the video was preprocessing to make sure the video was quantified for the next steps of the analysis. Determination of the motion amplitude was made by converting CNP of pixels between two frames in the video sequence into a time series waveform. A band pass filter (0.02 Hz - 2 kHz) was used to reduce unrelated signals. Every 10 seconds of time series data were transformed to frequency domain by fast Fourier transform (FFT). The average of the squared ratio of maximum-to-median power (M2) was then calculated (see, for example, FIG. 4) and used as an independent variable in a multiple regression model that related ABF (dependent variable) to the following independent variables: age, race, and BMI of the patient.

[0103] Referring to FIG. 4, therein is depicted an example of frequency domain analysis using video information captured from a patient participating in the Clinical Study. As shown in graph 405, the maximum (MAX) spectral power 420 of 7.14 x 10³ pixel (line 430) appears at a frequency of 2.2 Hz and the median (MED) spectral power 422 of 1.41 x 10³ pixel is indicated at line 432. The square of ratio of MAX to MED (M2) is computed as (7.14 / 1.41)² = 25.64 and used as a predictor of the ABF rate.

[0104] FIGS. 8 and 9 show the correlations between measured and predicted ABF in patients from the derivation cohort with M2<=100 (graph 805 of FIG. 8) and M2>100 (graph 905 of FIG. 9), respectively. In this clinical study, the ABF was measured by an HVT100 thermal dilution device (i.e., a “gold standard” of ABF measurement). Graph 905 depicts a curve fitting in the group with M2>100. Access flow was measured with the HVT100 device and predictions were made based on M2 calculations.

[0105] FIG. 10 depicts graph 1005 of the correlation between measured and predicted ABF in patients from a validation cohort. In particular, the multiple regression model developed in the derivation cohort was then evaluated in a validation cohort for patients with M2<= 100. FIG. 11 depicts graph 1105 of the relationship between CNP in a time domain analysis and ABF determined by routine online conductivity measurement (ABFOLC) in 15 patients.

[0106] Some embodiments may provide an access site analysis process based on flow type (a flow-based analysis process), such as a blood flow driving force (e.g., the driving force come from pulse pressure beat by beat to force blood volume from heart to flow to reginal capillaries) and turbulent flow.

[0107] As described above, CNP may include factors that are associated with determining access site blood flow characteristics. In some embodiments, the time series signal from a segment of vascular access may include at least two parts or signals: blood flow and turbulent flow. For example, the amplitude of the signal collected by video may be influenced or mixed by the turbulent flow factor. In general, the components of turbulent flow may be derived from one or more aspects, including, for example, the area vascular conjunction of artery and vein (fistula) and/or the point of stenosis in the blood vessels of both arteries and veins. In some embodiments, the factor of the point of stenosis in the blood vessels of both arteries and veins may be more complex in the time series analysis because the degree of influence depends on the location and the level of stenosis.

[0108] In some embodiments, the flow-based analysis process may include using MAX in the frequency domain as reflecting a driving force in the hemodynamic system in a vascular access segment. Physiologically, the driving force may come from pulse pressure beat-by-beat to force blood volume from the heart to flow to regional capillaries. Since a variable degree of turbulent flow exists in this system, components of a signal with variable frequency and amplitude may be observed in the frequency domain. Accordingly, the flow-based analysis process may use MED of all frequency components in as representing a level of turbulent flow. Since the value of MAX may be influenced by turbulent flow, the ratio of Max to Med (Max/Med) may be defined as a factor of representing reginal blood flow rate. Moreover, the term of the maximum-to- median power (M2) via calculating the square of the ratio of MAX to MED (M2 = (MAX/MED)²) may be used to present the main nature of quantitative change of blood flow in a system.

[0109] As described in the present disclosure, the relationship between M2 and ABF may be affected by many factors, such as skin tone, age, BMI, pulse pressure, heart rate, and/or compliance (see, for example, FIG. 4).

[0110] The flow-based analysis process may be based on or used to identify a relationship between blood flow and parameters based on the classification with similar characteristics of one or more patients. One assumption of the flow-based analysis process is that the patients with one or more similar (or identical or substantially identical) characteristics may have a similar relationship of blood flow to the function of M2 and other relate metrics. Multiple regression analyses may be used to establish the function of independent variables of M2 and related metrics to the dependent variable of blood flow. In various embodiments, the number of parameters depends on the total number of patients.

[OHl] The concept of classification may be based on the knowledge that the signal by motion of blood flow appears together with other noises by physiological or non-physiological activities defined as characteristics such that, for example, patients with the same characteristics may be separated. For example, a process may split all data into subgroups according to the metrics including components of frequency domain from the signals of video image analysis, anthropometric and physiological measurements.

[0112] FIG. 12 illustrates a decision tree 1205 in accordance with the present disclosure consisting of classification and regression processes. According to some embodiments, decision tree 1205 may be used for classification of patients to calculate ABF. The metrics of A, B, C ... may represent target variables and a, b, c, ... may represent thresholds for each target, respectively. Regression analysis or modelling may be used to find relationships between blood flow and the metrics in each subgroup, for example, for training process for Al and/or ML. In addition, in some embodiments, the same or similar models may be applied in patients for determining access flow by video signal.

[0113] A flow-based analysis process may analyze or otherwise process the components and functions in the frequency domain to determine information about patient characteristics, such as access stenosis. In some embodiments, the frequency (Fmax) at maximum amplitude in frequency domain may reflect the frequency of a signal of driving force which represents main blood flow in the regional area. The frequency (Fmin) of minimum amplitude may reflect the frequency of “noise” which represents degree of turbulent flow in the system. Since the main cause of turbulent flow is the stenosis in a vascular access structure, the degree of stenosis may be associated with the difference between maximum and minimum frequency. In addition, in some embodiments, Fmax may represent the frequency of signal in motion with main blood flow so that Fmax may approach the frequency of heart rate (FHR =1/HR) if there is no or less turbulent flow. Therefore, a flow-based analysis process may use the difference between Fmax and FHR as a metric for determining stenosis and/or the level of stenosis.

FLOW-BASED ANALYSIS PROCESS - EXAMPLE 1 : Estimation of Access Flow with Amplification Code

[0114] A study of 90 HD patients with AVF dysfunction requiring intervention with balloon angioplasty and/or stenting was conducted. More specifically, a patient sits on a chair and a one- minute video recording of the skin above the AVF and access fluid rate (Qa) measurements were conducted before and after the endovascular intervention. Videos were recorded with an iPhone 6S (Apple Inc., Cupertino, CA, USA). The reference of Qa was measured by a standard ultrasound device HVT100 endovascular flowmeter (Transonic Systems Inc., Ithaca, NY, USA). [0115] FIG. 13 illustrates a flow diagram of a flow-based process 1305 in accordance with the present disclosure consisting of using amplification code to determine ABF. As depicted in FIG. 13, a one-minute video was taken with 2k resolution and 30 fps. The video was processed by an amplification program to improve the ratio of signal-to-noise. After the video was converted to a time series, a digital filter was applied, for example, to keep the signal in range of 0.02-2 Hz. The signal of time series was transported to frequency domain by FFT. In the frequency domain, the main components, such as Max, Min, Mean, Med, Max/Med and M2, were determined based on results of statistical analysis. The flow-based process 1305 operated to split all data into subgroups based on characteristics of patients (see, for example, FIG. 14). Finally, multiple regression analysis were used for each group to obtain Qa of calculation by individual multiple regression models for each group.

[0116] FIG. 14 illustrates a process 1405 for splitting patients into groups (for example, four groups G1-G4). FIG. 15 illustrates the results of multiple regression analysis in each group (Gl, G2, G3 and G4) for the models or equations of FIG. 14. FIG. 16 illustrates graphs 1605 and 1610 of the correlation between measurement of Qa by clinical reference and Qa calculated by the model in patients with post intervention according to flow-based processes of some embodiments, via correlation between measurements (1605) and Bland- Altman (1610).

FLOW-BASED ANALYSIS PROCESS - EXAMPLE 2: Estimation of Access Flow without Amplification Code

[0117] As with Example 1, a study of 90 HD patients with AVF dysfunction requiring intervention with balloon angioplasty and/or stenting was conducted. More specifically, a patient sits on a chair and a one-minute video recording of the skin above the AVF and access fluid rate (Qa) measurements were conducted before and after the endovascular intervention. Videos were recorded with an iPhone 6S (Apple Inc., Cupertino, CA, USA). The reference of Qa was measured by a standard ultrasound device HVT100 endovascular flowmeter (Transonic Systems Inc., Ithaca, NY, USA).

[0118] FIG. 17 illustrates a flow diagram of a flow-based process 1705 in accordance with the present disclosure for determining ABF. In general, process 1705 is essentially the same as process 1305, however, without the code amplification step. As depicted in FIG. 17, a one- minute video was taken with 2k resolution and 30 fps. The video was processed by an amplification program to improve the ratio of signal-to-noise. After the video was converted to a time series, a digital filter was applied, for example, to keep the signal in range of 0.02-2 Hz.

The signal of time series was transported to frequency domain by FFT. In the frequency domain, the main components, such as Max, Min, Mean, Med, Max/Med and M2, were determined based on results of statistical analysis. The flow-based process 1705 operated to split all data into subgroups based on characteristics of patients (see, for example, FIG. 18). Finally, multiple regression analysis were used for each group to obtain Qa of calculation by individual multiple regression models for each group. [0119] FIG. 18 illustrates a process 1805 for splitting patients into groups (for example, four groups G1-G4). FIG. 19 illustrates the results of multiple regression analysis in each group (Gl, G2, G3 and G4) for the models or equations of FIG. 18. FIG. 20 illustrates graphs 2005 and 2010 of the correlation between measurement of Qa by clinical reference and Qa calculated by the model in patients with post intervention according to flow-based processes of some embodiments, via correlation between measurements (2005) and Bland- Altman (2010).

FLOW-BASED ANALYSIS PROCESS - EXAMPLE 3: Prediction of Stenosis

[0120] A VIP analysis process according to some embodiments was employed pre- and post- endovascular interventions in 90 hemodialysis patients (age 63.27 ± 14.3, 41 females, weight 78.6 ± 21.5 kg) and Fmax and Fmin pre- and post-intervention were recorded. AF was defined as FMin - FMax for each video. Qa was measured pre- and post-intervention by HVT100 endovascular flowmeter. The degree of stenosis (%) was quantitated by angiography. Heart rate (HR, beats/min) was expressed as a frequency (FHR, HZ). The pre- and post-intervention differences between Fmax and FHR were 1.13±0.7 Hz and 1 ,38±1.4 Hz, respectively. AF was associated with Qa pre- and post-intervention. AF increased most when Qa increased from preintervention range of 300 to 600 ml/min to post-intervention 600 to 900 ml/min. FIG. 21 shows the relation between % stenosis and the change in AF between pre- and post-intervention. The difference between Fmax and FHR was associated with % stenosis pre-intervention (FIG. 22). Higher degrees of stenosis were associated with greater increases in Qa from before to after the intervention. The degree of AVF stenosis was also positively related with the change in Fmin from before to after the intervention (FIG. 23). FIGS. 24 and 25 depict AF associated with Qa pre- and post-intervention, respectively.

[0121] Accordingly, some embodiments may provide a flow-based analysis process configured to use frequency domain analysis based on signal of video image processing (VIP) technique to estimation of access flow rate and prediction of stenosis for hemodialysis patients. In some embodiments, a method of classification and regression algorithms or processes may be used to make an automatic decision for separating patients into subgroups with the same or similar characteristics of patients in order to improve the accuracy of regression analysis. In various embodiments, a flow-based analysis process may be based on a clustering analysis, for example, using the metrics of components in frequency domain analysis, anthropometric, and/or physiological measurements. In exemplary embodiments, a flow-based analysis process can be performed with or by Al and/or ML to process a large number of patient data in order to obtain more accurate and procession results based on video image processing technique. Predicting degree (%) stenosis can be performed with the metrics AF (Fmin-Fmax), Fmin, and/or Fmax-FuR including, but not limited to, the combination with above frequency metrics.

[0122] The relationship between Fwax and FHR may indicate that the signal of Fj iax represents an important hemodynamic component of Qa. AF may be used, for example, as an index to predict low levels of stenosis. The use of frequency domain analysis from video image data provides a contact-free method to ascertain Qa and to indirectly indicate the degree of stenosis.

[0123] FIG. 26 illustrates an embodiment of an exemplary computing architecture 2600 suitable for implementing various embodiments as previously described. In various embodiments, the computing architecture 2600 may comprise or be implemented as part of an electronic device. In some embodiments, the computing architecture 2600 may be representative, for example, of computing device 110. The embodiments are not limited in this context. As used in this application, the terms “system” and “component” and “module” are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution, examples of which are provided by the exemplary computing architecture 2600. For example, a component can be, but is not limited to being, a process running on a processor, a processor, a hard disk drive, multiple storage drives (of optical and/or magnetic storage medium), an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers. Further, components may be communicatively coupled to each other by various types of communications media to coordinate operations. The coordination may involve the unidirectional or bi-directional exchange of information. For instance, the components may communicate information in the form of signals communicated over the communications media. The information can be implemented as signals allocated to various signal lines. In such allocations, each message is a signal. Further embodiments, however, may alternatively employ data messages. Such data messages may be sent across various connections. Exemplary connections include parallel interfaces, serial interfaces, and bus interfaces.

[0124] The computing architecture 2600 includes various common computing elements, such as one or more processors, multi-core processors, co-processors, memory units, chipsets, controllers, peripherals, interfaces, oscillators, timing devices, video cards, audio cards, multimedia input/output (I/O) components, power supplies, and so forth. The embodiments, however, are not limited to implementation by the computing architecture 2600.

[0125] As shown in FIG. 26, the computing architecture 2600 comprises a processing unit 2604, a system memory 2606 and a system bus 2608. The processing unit 2604 may be a commercially available processor and may include dual microprocessors, multi-core processors, and other multi-processor architectures.

[0126] The system bus 2608 provides an interface for system components including, but not limited to, the system memory 2606 to the processing unit 2604. The system bus 2608 can be any of several types of bus structure that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. Interface adapters may connect to the system bus 2608 via a slot architecture. Example slot architectures may include without limitation Accelerated Graphics Port (AGP), Card Bus, (Extended) Industry Standard Architecture ((E)ISA), Micro Channel Architecture (MCA), NuBus, Peripheral Component Interconnect (Extended) (PCI(X)), PCI Express, Personal Computer Memory Card International Association (PCMCIA), and the like.

[0127] The system memory 2606 may include various types of computer-readable storage media in the form of one or more higher speed memory units, such as read-only memory (ROM), random-access memory (RAM), dynamic RAM (DRAM), Double-Data-Rate DRAM (DDRAM), synchronous DRAM (SDRAM), static RAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, polymer memory such as ferroelectric polymer memory, ovonic memory, phase change or ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS) memory, magnetic or optical cards, an array of devices such as Redundant Array of Independent Disks (RAID) drives, solid state memory devices (e.g., USB memory, solid state drives (SSD) and any other type of storage media suitable for storing information. In the illustrated embodiment shown in FIG. 26, the system memory 2606 can include non-volatile memory 2610 and/or volatile memory 2612. A basic input/output system (BIOS) can be stored in the nonvolatile memory 2610.

[0128] The computer 2602 may include various types of computer-readable storage media in the form of one or more lower speed memory units, including an internal (or external) hard disk drive (HDD) 2614, a magnetic floppy disk drive (FDD) 2616 to read from or write to a removable magnetic disk 2611, and an optical disk drive 2620 to read from or write to a removable optical disk 2622 (e.g., a CD-ROM or DVD). The HDD 2614, FDD 2616 and optical disk drive 2620 can be connected to the system bus 2608 by a HDD interface 2624, an FDD interface 2626 and an optical drive interface 2628, respectively. The HDD interface 2624 for external drive implementations can include at least one or both of Universal Serial Bus (USB) and IEEE 1114 interface technologies.

[0129] The drives and associated computer-readable media provide volatile and/or nonvolatile storage of data, data structures, computer-executable instructions, and so forth. For example, a number of program modules can be stored in the drives and memory units 2610, 2612, including an operating system 2630, one or more application programs 2632, other program modules 2634, and program data 2636. In one embodiment, the one or more application programs 2632, other program modules 2634, and program data 2636 can include, for example, the various applications and/or components of computing device 110.

[0130] A user can enter commands and information into the computer 2602 through one or more wired/wireless input devices, for example, a keyboard 2638 and a pointing device, such as a mouse 2640. These and other input devices are often connected to the processing unit 2604 through an input device interface 2642 that is coupled to the system bus 2608, but can be connected by other interfaces.

[0131] A monitor 2644 or other type of display device is also connected to the system bus 2608 via an interface, such as a video adaptor 2646. The monitor 2644 may be internal or external to the computer 2602. In addition to the monitor 2644, a computer typically includes other peripheral output devices, such as speakers, printers, and so forth.

[0132] The computer 2602 may operate in a networked environment using logical connections via wired and/or wireless communications to one or more remote computers, such as a remote computer 2648. The remote computer 2648 can be a workstation, a server computer, a router, a personal computer, portable computer, microprocessor-based entertainment appliance, a peer device or other common network node, and typically includes many or all of the elements described relative to the computer 2602, although, for purposes of brevity, only a memory/storage device 2650 is illustrated. The logical connections depicted include wired/wireless connectivity to a local area network (LAN) 2652 and/or larger networks, for example, a wide area network (WAN) 2654. Such LAN and WAN networking environments are commonplace in offices and companies, and facilitate enterprise-wide computer networks, such as intranets, all of which may connect to a global communications network, for example, the Internet.

[0133] The computer 2602 is operable to communicate with wired and wireless devices or entities using the IEEE 802 family of standards, such as wireless devices operatively disposed in wireless communication (e.g., IEEE 802.16 over-the-air modulation techniques). This includes at least Wi-Fi (or Wireless Fidelity), WiMax, and Bluetooth™ wireless technologies, among others. Thus, the communication can be a predefined structure as with a conventional network or simply an ad hoc communication between at least two devices. Wi-Fi networks use radio technologies called IEEE 802.1 lx (a, b, g, n, etc.) to provide secure, reliable, fast wireless connectivity. A Wi-Fi network can be used to connect computers to each other, to the Internet, and to wire networks (which use IEEE 802.3-related media and functions).

[0134] Numerous specific details have been set forth herein to provide a thorough understanding of the embodiments. It will be understood by those skilled in the art, however, that the embodiments may be practiced without these specific details. In other instances, well-known operations, components, and circuits have not been described in detail so as not to obscure the embodiments. It can be appreciated that the specific structural and functional details disclosed herein may be representative and do not necessarily limit the scope of the embodiments.

[0135] Some embodiments may be described using the expression "coupled" and "connected" along with their derivatives. These terms are not intended as synonyms for each other. For example, some embodiments may be described using the terms “connected” and/or “coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. The term "coupled,” however, may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other. [0136] Unless specifically stated otherwise, it may be appreciated that terms such as “processing,” “computing,” “calculating,” “determining,” or the like, refer to the action and/or processes of a computer or computing system, or similar electronic computing device, that manipulates and/or transforms data represented as physical quantities (e.g., electronic) within the computing system’s registers and/or memories into other data similarly represented as physical quantities within the computing system’s memories, registers or other such information storage, transmission or display devices. The embodiments are not limited in this context.

[0137] It should be noted that the methods described herein do not have to be executed in the order described, or in any particular order. Moreover, various activities described with respect to the methods identified herein can be executed in serial or parallel fashion.

[0138] Although specific embodiments have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments. It is to be understood that the above description has been made in an illustrative fashion, and not a restrictive one. Combinations of the above embodiments, and other embodiments not specifically described herein will be apparent to those of skill in the art upon reviewing the above description. Thus, the scope of various embodiments includes any other applications in which the above compositions, structures, and methods are used.

[0139] Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0140] As used herein, an element or operation recited in the singular and proceeded with the word “a” or “an” should be understood as not excluding plural elements or operations, unless such exclusion is explicitly recited. Furthermore, references to “one embodiment” of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

[0141] The present disclosure is not to be limited in scope by the specific embodiments described herein. Indeed, other various embodiments of and modifications to the present disclosure, in addition to those described herein, will be apparent to those of ordinary skill in the art from the foregoing description and accompanying drawings. Thus, such other embodiments and modifications are intended to fall within the scope of the present disclosure. Furthermore, although the present disclosure has been described herein in the context of a particular implementation in a particular environment for a particular purpose, those of ordinary skill in the art will recognize that its usefulness is not limited thereto and that the present disclosure may be beneficially implemented in any number of environments for any number of purposes.

Accordingly, the claims set forth below should be construed in view of the full breadth and spirit of the present disclosure as described herein.

Claims

CLAIMS What is claimed is:

1. An apparatus, comprising: at least one processor; and a memory coupled to the at least one processor, the memory comprising instructions that, when executed by the at least one processor, cause the at least one processor to: generate video information based on a video of a dialysis access site of a patient, determine change in the number of pixels (CNP) information of the video information, the CNP information associated with movement of a skin surface of the patient due to blood flow through the dialysis access site, determine frequency domain information of the CNP information, and determine FMAX as a maximum amplitude in the frequency domain and FMIN as a minimum amplitude in the frequency domain, wherein FMAX is associated with a frequency of a signal of a driving force of main blood flow in a regional area and FMIN is associated with turbulent blood flow.

2. The apparatus of claim 1, the instructions, when executed by the at least one processor, to cause the at least one processor to determine at least one access site characteristic based on FMAX or FMIN.

3. The apparatus of claim 2, the at least one access site characteristic comprising a stenosis estimation.

4. The apparatus of claim 1, wherein Fmax approaches a frequency of heart rate (FHR =1/HR) with no turbulent flow.

5. The apparatus of claim 1, the instructions, when executed by the at least one processor, to cause the at least one processor to determine AF as FMin - FM_;1\.

6. The apparatus of claim 1, the instructions, when executed by the at least one processor, to cause the at least one processor to determine a stenosis estimation based on AF.

37

7. The apparatus of claim 1, the instructions, when executed by the at least one processor, to cause the at least one processor to split frequency domain information into a plurality of subgroups based on patient characteristics.

8. The apparatus of claim 7, the patient characteristics comprising at least one of age, gender, BMI, and skin tone.

9. The apparatus of claim 7, the instructions, when executed by the at least one processor, to cause the at least one processor to perform a regression analysis on each of the plurality of groups to determine an access fluid rate (Qa) for each group.

10. The apparatus of claim 1, the instructions, when executed by the at least one processor, to cause the at least one processor to: determine a maximum-to-median power (M2) value of the frequency domain information, and determine at least one access site characteristic based on the M2 value.

11. A method of determining a stenosis estimation, the method comprising: generating video information based on a video of a dialysis access site of a patient; determining change in the number of pixels (CNP) information of the video information, the CNP information associated with movement of a skin surface of the patient due to blood flow through the dialysis access site; determining frequency domain information of the CNP information; and determining FMAX as a maximum amplitude in the frequency domain and FMIN as a minimum amplitude in the frequency domain, wherein FMAX is associated with a frequency of a signal of a driving force of main blood flow in a regional area and FMIN is associated with turbulent blood flow.

38

12. The method of claim 11, comprising determining at least one access site characteristic based on FMAX or FMIN.

13. The method of claim 12, the at least one access site characteristic comprising a stenosis estimation.

14. The method of claim 11, wherein Fmax approaches a frequency of heart rate (FHR =1/HR) with no turbulent flow.

15. The method of claim 11, comprising determining AF as FMin - FMax.

16. The method of claim 11, comprising determining a stenosis estimation based on AF.

17. The method of claim 11, comprising splitting frequency domain information into a plurality of subgroups based on patient characteristics.

18. The method of claim 17, the patient characteristics comprising at least one of age, gender, BMI, and skin tone.

19. The method of claim 17, comprising performing a regression analysis on each of the plurality of groups to determine an access fluid rate (Qa) for each group.

20. The method of claim 11, comprising: determining a maximum-to-median power (M2) value of the frequency domain information, and determining at least one access site characteristic based on the M2 value.