WO2016081994A1 - Gait monitoring system, method and device - Google Patents

Gait monitoring system, method and device Download PDF

Info

Publication number
WO2016081994A1
WO2016081994A1 PCT/AU2015/050738 AU2015050738W WO2016081994A1 WO 2016081994 A1 WO2016081994 A1 WO 2016081994A1 AU 2015050738 W AU2015050738 W AU 2015050738W WO 2016081994 A1 WO2016081994 A1 WO 2016081994A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
gait
subject
device
image
parameters
Prior art date
Application number
PCT/AU2015/050738
Other languages
French (fr)
Inventor
Philip Hermann GOEBEL
Wesley Joseph LOH
Krishna Savant SYREDDY
Original Assignee
Quanticare Technologies Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Detecting, measuring or recording for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1116Determining posture transitions
    • A61B5/1117Fall detection
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Detecting, measuring or recording for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/112Gait analysis
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Detecting, measuring or recording for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices

Abstract

A method for monitoring gait of a subject includes operating an image capture device such as a digital camera, to capture at least one image of a region of interest of the subject during ambulation on a walking surface. Captured images are processed to identify automatically pixels representing a relevant feature of the subject and a virtual marker such as a 3D coordinate marker is assigned to the identified pixels. The virtual marker is processed to calculate a gait parameter for the subject, typically bycomparing a virtual marker assigned to a captured image with a virtual marker assigned to a subsequently captured image, determining movement of the relevant feature of the subject between the two captured images, and calculating the gait parameter according to the determined movement. The method is computer-implemented, with the image capture device mounted on an ambulatory aid such as a wheeled walking frame. A system and devices for monitoring gait are also disclosed.

Description

GAIT MONITORING SYSTEM, METHOD AND DEVICE

TECHNICAL FIELD

[1 ] The present invention generally relates to a device, system, and method for recording and assessing a walking characteristic of a subject. More particularly, the present disclosure relates to a device, system and method to continuously or periodically monitor and assess the gait of a subject.

BACKGROUND

[2] Falls and walking impediments are an enormous challenge facing the healthcare system, particularly in aged care or respite care facilities. Falls are often associated with an acute expression of pathological deterioration that may affect an individual's walking quality and overall life quality. Falls may become particularly problematic with elderly persons, such as persons over the age of 65, as they may result in serious injury or even death in some situations.

[3] In order to improve or sustain an individual's walking quality, post-fall treatments are commonly used. These treatments may include a monitored walk assessment wherein the gait of a subject or walking characteristics of a subject are assessed in a clinical environment. This assessment provides healthcare

professionals and carers with baseline or threshold gait parameters by which to assess and recommend levels of physical therapy.

[4] However, assessment is suboptimal in that it only provides a small amount of time in which the gait of a subject may be monitored and any subsequent assessments to a subject's threshold parameters would require the subject to return to the gait laboratory for further assessment. In addition, the gait parameters obtained in the gait laboratory may be artificially skewed by the walking environment. As such, the current paradigm is one of a reactive treatment rather than an adaptive treatment that monitors the walking quality of a subject over extended periods of time.

[5] Other methods to monitor the gait of a subject in a natural environment include embedded systems within flooring or monitoring devices embedded within a subject's shoes. However, these monitoring devices may be costly and are limited to areas in which flooring has been altered for the embedded systems. Additionally, gait parameters or gait characteristics obtained from these devices commonly contain inaccuracies due to sensor constraints in the flooring or in the subject's shoes and may not take into consideration any pre-existing medical conditions of the subject.

[6] US Patent Application No. 201 1/897722 generally discloses a wearable gait monitoring device wherein the device measures the stride lengths of a patient when the device featuring an accelerometer is worn around the ankle of said patient; and the accelerometer measures the movement of patient's feet.

[7] US Patent Application No. 201 1/556858 discloses a system for monitoring gait of a subject wherein the device is positioned within an insole of a shoe and pressure applied to a pressure sensor, mounted in the insole, by the subject is recorded as gait monitoring data.

[8] The background art in relation to gait monitoring devices and methods has focused on a wearable device in which a pressure sensor or an accelerometer is used to determine a walking characteristic of the subject. These devices are required to be worn on the patient's body and rely heavily on the device experiencing an external force imparted by the movement of the patient's feet on the actual device.

Additionally, these devices do not take into consideration whether the subject is about to experience a fall or if the subject has fallen over. Additionally, these devices are required to be worn by the subject to measure a subject's gait which can be inconvenient and uncomfortable.

[9] Additionally, previous devices may have interoperability or connection problems with other external electronic devices and systems and alarms due to the fact that the device may have been typically mounted on a patient's foot which moves over a range of area.

[10] It would be desirable to provide an improved gait monitoring system that allows elderly persons and persons with disabilities more freedom to move without continual supervision. It may also be desirable to have the ability to alert a carer or healthcare professional in the event of an irregular gait or fall. [1 1 ] The discussion of the background to the invention included herein including reference to documents, acts, materials, devices, articles and the like is included to explain the context of the present invention. This is not to be taken as an admission or a suggestion that any of the material referred to was published, known or part of the common general knowledge in Australia or in any other country as at the priority date of any of the claims.

SUMMARY OF THE INVENTION

[12] A first aspect of the present invention provides a device for monitoring gait of a subject, the device comprising: a gait sensing means for generating a gait parameter, a transducer operatively associated with the gait sensing means, a microprocessor in communication with the transducer, a memory circuit associated with the microprocessor for recording the gait parameter, wherein gait parameter is compared with at least one predetermined threshold parameter to determine whether the gait of the subject has exceeded the at least one predetermined threshold parameter.

[13] The device may comprise a mounting means such that the device is mountable on an ambulatory aid.

[14] In some embodiments, the gait sensing means may be disposed off-centre distal to the mounting means.

[15] The gait sensing means may comprise at least one of: a monocular camera module, a stereoscopic camera module, an infrared sensor, a proximity sensor or a combination thereof.

[16] In some embodiments, an alert may be triggered if a predetermined number of threshold parameters are exceeded.

[17] In some embodiments, the gait sensing means may be adapted to determine spatial or directional variations in the gait of the subject . The gait sensing means may comprise at least one of an accelerometer and/or a gyroscope.

[18] Preferably, the spatial or direction variations are determined by assigning a co-ordinate system to a first recorded image and comparing the co-ordinate system of the first recorded image with a co-ordinate system of at least one second recorded image. This may involve allocating a 3D coordinate vector to a feature in the first and second recorded images.

[19] Preferably, calibration of the gait sensing means determines at least one of: a relative height of the device compared to the walking surface and an angle of inclination of the device.

[20] Preferably, the device comprises a transceiver for sending and receiving data.

[21 ] In some embodiments, the mounting means comprises an omnidirectional pivot for positioning the device.

[22] In some embodiments, the camera module comprises a curvilinear lens or a rectilinear lens.

[23] Preferably, the device has software associated with the microprocessor of the device configured to calculate at least one gait parameter for the subject selected from the group including: a) right step length, b) left step length, c) ambulation time, d) walking velocity, e) walking distance, f) number of steps taken by a subject, g) cadence, h) gait cycle time, i) left single leg stance time, j) right single leg stance time, k) double leg stance time, I) right swing time, m) left swing time, n) left foot clearance, o) right foot clearance, p) double leg stance base of support, and q) stride length.

[24] In some embodiments the device employs a method for monitoring the gait of a subject, the method comprising the steps of: A) recording at least one image, B) virtually segmenting the at least one said image and assigning a co-ordinate marker system thereto, C) comparing the assigned co-ordinate marker system to at least one predetermined threshold parameter, and D) determining whether a number of predetermined threshold parameters have been exceeded based on the comparison.

[25] A second aspect of the present invention provides a process for monitoring gait of a subject, the process comprising the steps of: A) recording at least one image associated with of the gait of the subject, B) determining at least one directional or spatial variation of the gait of the subject based on the at least one said image, C) comparing the at least one directional or spatial variation of the subject with at least one predetermined gait threshold parameter; and D) determining whether a

predetermined number of gait threshold parameters have been exceeded.

[26] The process may comprise the step of determining a relative location of the subject.

[27] In some embodiments, the process includes the step of triggering an alert if the predetermined number of gait threshold parameters have been exceeded.

[28] Preferably, the process includes the step of assigning virtual marker points to the recorded image to determine at least one directional or spatial variation of the subject.

[29] A third aspect of the present invention provides a method for monitoring the gait of a subject, the method comprising the steps of: A) recording at least one image, B) virtually segmenting the at least one said image and assigning a coordinate marker system thereto, C) comparing the assigned co-ordinate marker system to at least one predetermined threshold parameter, and D) determining whether a number of predetermined threshold parameters have been exceeded based on the comparison.

[30] Preferably, the method includes the step of calibrating a gait sensing means to record the at least one image by adjusting the focus of an associated camera module.

[31 ] The method may also comprise the step of calculating a relative location of the gait sensing means relative to the subject.

[32] In some embodiments, an alert is triggered if the calculated parameters exceed the number of predetermined threshold parameters.

[33] A fourth aspect of the present invention provides a system for monitoring gait of a subject, the system including: a digital image capture device configured to capture one or more images of the subject during ambulation on a walking surface; a computer-implemented image processor for processing the one or more captured images, the image processor being configured to process the one or more captured images by automatically identifying in one or more of the captured images one or more relevant features of the subject and allocating a virtual marker to the one or more relevant features; a computer-implemented gait parameter processor for processing virtual marker data from the image processor and generating one or more gait parameters; and a display device configured to display one or more of the gait parameters calculated for the subject; wherein the one or more gait parameters are indicative of the gait of the subject being monitored.

[34] The digital image capture device may be provided in a housing that is attachable to an ambulatory aid having substantially continuous contact with the walking surface during use, or in a housing that is integral with an ambulatory aid having substantially continuous contact with the walking surface during use. Typically, the digital image capture device is selected from a group including: a monocular camera, a stereoscopic camera, and an infrared image sensor and any of the foregoing capable of capturing sequential images.

[35] In some embodiments, the system further includes a calibration module configured to determine automatically an orientation of the digital image capture device relative to the walking surface. The calibration module uses the orientation of the digital image capture device to determine a 3D coordinate reference system for the virtual marker data. Typically, the virtual marker data includes a 3D coordinate vector calculated by the computer-implemented image processor for each virtual marker.

[36] Preferably, the one or more relevant features are features of the left and/or right foot of the subject and ideally, include at least the forward-most point on the subject's left and/or right foot although any feature on the subject's lower limb anatomy may be used by the system.

[37] Preferably, the computer-implemented gait parameter processor calculates one or more gait parameters by comparing virtual marker data associated with a captured image with virtual marker data associated with a previously captured image or multiple previously captured images. The gait parameters calculated by the computer-implemented gait parameter processor may include primary gait

parameters selected from the group including: (a)stride length; (b) right step length; (c) left step length; (d) gait cycle time; (e) right single leg stance time; (f) left single leg stance time; (g) double leg stance time; (h) right leg swing time; (i) left leg swing time; (j) right foot clearance; (k) left foot clearance; and (I) double leg stance base of support.

[38] In some embodiments, the system includes a computer-implemented secondary gait parameter processor configured to calculate from the virtual marker data and the primary gait parameters, one or more secondary gait parameters selected from the group including: (a) ambulation time; (b) walking velocity; (c) walking distance; (d) number of steps taken; (e) cadence; and (f) stride length variability. In some embodiments, a processor of the system is further configured to determine automatically one or more of: start time and finish time of an assessment of a subject's gait; number of gait cycles in the assessment; and total time spent ambulating during the assessment.

[39] In some embodiments, the system includes a computer-implemented gait data processor configured to perform one or more of: (a) calculating automated notifications based on a comparison of virtual marker data for the subject with one or more reference values; (b) aggregating data from one or both of the computer- implemented image processor and the computer-implemented gait parameter processor, and (c) generating assessment report data suitable for display on the display device or transmission to another computer-implemented processing device.

[40] A fifth aspect of the present invention provides a method for monitoring gait of a subject including the steps of: (a) operating an image capture device to capture at least one image of a region of interest of the subject during ambulation on a walking surface; (b) image processing the at least one image to identify automatically a relevant feature of the subject and assigning a virtual marker to the identified feature; and (c) processing the virtual marker to calculate a gait parameter for the subject; wherein the gait parameter is indicative of the gait of the subject being monitored. Preferably the image processing involves identifying automatically pixels in the captured image that represent the relevant feature of the subject and assigning the virtual marker to the identified pixels.

[41 ] Typically, the virtual marker comprises a 3D coordinate vector and calculating the gait parameter includes comparing a virtual marker assigned to a captured image with a virtual marker assigned to a subsequently captured image, determining movement of the relevant feature of the subject between the two captured images, and calculating the gait parameter according to the determined movement. It is to be understood that the method may track movement of features between multiple captured images to determine the gait parameter.

[42] In some embodiments, the method includes calibrating the image capture device by determining automatically the orientation of the image capture device relative to the walking surface. The orientation information obtained during calibration is used to provide a reference system for virtual markers assigned to the captured images.

[43] Ideally, the image capture device is attached to or incorporated in an ambulatory aid having substantially continuous contact with the walking surface during use, so as to maintain a substantially constant orientation of the image capture device relative to the walking surface. When the image capture device is attached to an ambulatory aid, this may be done during initial manufacture, or during a retrofitting process to modify an ambulatory aid that is already in use.

[44] In some embodiments, the method includes processing one or more gait parameters of the subject to produce assessment data. The assessment data is configurable to produce one or more of a dashboard, chart, report or visual representation of the subject's gait to a third party. Alternatively/additionally, the method includes processing one or more gait parameters of the subject to identify an alert condition. This may be achieved by comparing values calculated for one or more gait parameters of the subject with one or more reference values for those gait parameters and confirming an alert condition exists if one or more of the subject's gait parameter values differ from the respective reference values by more than a predetermined amount.

[45] In some embodiments, the method includes generating an alert when a condition is met. Such conditions may include e.g. when (a) values calculated for a particular gait parameter of the subject differ from the respective reference value by more than the pre-determined amount; (b) values calculated for a plurality of gait parameters of the subject differ from their respective reference values by more than a pre-determined amount for each parameter; and (c) values calculated for a plurality of gait parameters of the subject differ from their respective reference values by more than a predetermined sum total amount, to name a few.

[46] A generated alert may be assigned an alert priority selected from a group including: (a) first priority when the alert corresponds to detection of a fall; (b) second priority when the alert corresponds to detection of a fall being likely; (c) third priority when the alert corresponds to detection of a gait parameter indicative of possible injury to the subject occurring or likely to occur. The alert may be generated by any suitable means including one or more of: text message, a noise alert, vibrations, a video message, illumination of a lighting element, and a flagged status update in live monitoring data.

[47] A sixth aspect of the present invention provides a device for monitoring a gait of a subject, the device including: (a) a housing containing a digital image capture device configured to capture one or more images of the subject during ambulation on a walking surface; and (b) a computer-implemented image processor in the housing, for processing the one or more captured images, the image processor being configured to process the one or more captured images by automatically identifying in one or more of the captured images one or more relevant features of the subject and allocating a virtual marker to the one or more relevant features; and (c) a transceiver configured to transmit one or both of the captured image data and the processed image data to a remote processor; wherein the housing is attachable to an

ambulatory aid having substantially continuous contact with the walking surface during use.

[48] A seventh aspect of the present invention provides a device comprising an ambulatory aid for monitoring gait of a subject, the ambulatory aid having: (a) a digital image capture device in a housing forming part of the aid and configured to capture one or more images of the subject during ambulation on a walking surface; and (b) a computer-implemented image processor in the housing, for processing the one or more captured images, the image processor being configured to process the one or more captured images by automatically identifying in one or more of the captured images one or more relevant features of the subject and allocating a virtual marker to the one or more relevant features; and (c) a transceiver configured to transmit one or both of the captured image data and the processed image data to a remote

processor; wherein the ambulatory aid has substantially continuous contact with the walking surface during use.

[49] The device of the sixth and seventh aspects of the invention may be configurable for use with: (a) a computer-implemented gait parameter processor for processing virtual marker data from the image processor and generating one or more gait parameters; and (b) a display device configured to display one or more of the gait parameters calculated for the subject; wherein the one or more gait parameters are indicative of the gait of the subject being monitored.

[50] The digital image capture device may be selected from a group including: a monocular camera, a stereoscopic camera, and an infrared image sensor and any of the foregoing capable of capturing sequential images.

[51 ] The device may further include a calibration module configured to determine automatically an orientation of the digital image capture device relative to the walking surface and using the orientation to provide a reference system for the virtual marker data. The calibration module uses the orientation of the digital image capture device to determine a 3D coordinate reference system for the virtual marker data. Typically, the virtual marker data includes a 3D coordinate vector calculated by the computer-implemented image processor for each virtual marker.

[52] Ideally, the transceiver of the device is configurable to interoperate with the system disclosed in the foregoing.

[53] Where the terms "comprise", "comprises", "comprised" or "comprising" are used in this specification (including the claims) they are to be interpreted as specifying the presence of the stated features, integers, steps or components, but not precluding the presence of one or more other features, integers, steps or components or group thereof.

[54] The invention is to be interpreted with reference to the at least one of the technical problems described or affiliated with the background art. The present aims to solve or ameliorate at least one of the technical problems or to provide a useful alternative and this may result in one or more advantageous effects as defined by this specification and described in detail with reference to the preferred embodiments of the present invention.

BRIEF DESCRIPTION OF THE FIGURES

[55] Features of the invention will now be described with reference to the accompanying figures which illustrate embodiments of the invention. It is to be understood that these embodiments are examples only, and are not to be taken as limiting on the scope of the invention as defined in the claims appended hereto.

[56] Figure 1 illustrates an embodiment of the gait monitoring device of the present disclosure;

[57] Figure 2 illustrates an embodiment of the device of the present disclosure mounted on an ambulatory aid;

[58] Figure 3 is a schematic illustration of components of the device according to an embodiment of the present invention;

[59] Figure 4A illustrates another embodiment of the device of the present disclosure mounted on an ambulatory aid;

[60] Figure 4B illustrates a close up view of the device of Figure 4A;

[61 ] Figure 4C illustrates an alternative embodiment of the device of Figure 4B;

[62] Figure 5 is a flowchart illustrating steps in a method for use with the device of the present disclosure; and

[63] Figure 6 is a flowchart illustrating steps in another method for use with the device of the present disclosure.

[64] Figure 7 illustrates a gait monitoring device incorporated into an

ambulatory aid, according to another embodiment of the invention.

[65] Figure 8 is a schematic illustration of a system for monitoring the gait of a subject according to an embodiment of the invention. [66] Figure 9 illustrates an example of a dashboard presenting aggregated data for use on display device according to embodiments of the invention.

DETAILED DESCRIPTION OF THE INVENTION

[67] Preferred embodiments of the invention will now be described with reference to the accompanying drawings and non-limiting examples.

[68] A first preferred embodiment of the present disclosure generally includes a portable device, system and method for recording and/or assessing gait of a subject. The embodied device records the gait of a subject and assesses the recorded gait of the subject by comparing the observed or recorded gait to predetermined gait parameters. If the recorded gait of the subject is outside of a predetermined number of parameters at least one alert can be triggered by the device to alert at least one of a healthcare professional, a carer or an assigned person, or to the subject

themselves. The alert triggered can be selected from a list of predetermined or customised messages depending e.g. on the predetermined gait parameter that the recorded gait differs from and an extent (in time or quantity) that the recorded gait parameter is outside the predetermined parameter value. The predetermined gait parameter may be defined by a single value, or a range of values, or a function that determines a single value or range of values and may relate to any gait parameter such as for example those identified in Table 1 and Table 2 disclosed herein.

[69] In some cases, the triggering of an alert may be prioritised so that some recorded gait are more likely to trigger an alert than other gait parameters that fall outside the predetermined value. Prioritisation may be global (e.g. where a

comparison of recorded gait against predetermined gait parameters indicate a fall has occurred) or subject or cohort or condition specific (e.g. where the subject has a condition such as right knee arthritis, right leg stance time may be prioritised over other non-fall related parameters). The alert may be delivered in any suitable form such as a text message, a noise alert, vibrations, a video message, a status update or any other means to notify a healthcare professional, carer or assigned person which may include the subject.

[70] In this description, "recorded gait" means the data that represents the gait which is observed by the device and/or its associated sensors. [71 ] Figure 1 of the present disclosure depicts a gait monitoring device 100 having a housing 102 with a gait sensing means 104 on an exterior surface of the housing 102. The gait sensing means 104 is preferably a camera module 104 positioned distal to a mounting point on an ambulatory aid, such that the camera module 104, if positioned correctly, has an unobstructed view to record images of the subject's feet while walking. The camera module 104 records the gait of a subject, preferably as an image or sequence of images/video, and stores the recorded gait in a memory of the device 100. The camera module 104 can be configured to take a plurality of images, preferably between 0 to 72 images a second ('framerate') although faster framerates are contemplated for high definition analysis, and slower framerates are contemplated e.g. for devices where processing and battery limitations are incompatible with higher definition gait recording and processing.

[72] In at least one embodiment, the device 100 may be powered by a rechargeable internal battery, such as a Li-ion battery, Li Polymer, or NiMH battery, and is rechargeable via power input 106. In an alternate embodiment the device 100 has interchangeable batteries. Recorded gait images and video can be transferred from the device 100 via at least one external interface 108 to another device, such as a personal computer, tablet device, mobile phone or the like. In a preferred embodiment the device includes a plurality of external interfaces 108, such as a video output, auxiliary output, audio output, RCA, HDMI, Ethernet port and at least one USB port. Alternatively/additionally, the device 100 can send and receive data via wireless transceiver 1 10. The wireless transceiver 1 10 can use a range of contactless communication technologies, for example, Bluetooth™, low energy Bluetooth™, Wi- Fi™, 3G, 4G, ZigBee or any other suitable wireless local area network (WLAN) or wide area network means.

[73] In another embodiment one or more infrared emitters (not shown) can be disposed on the exterior of the housing 102 and the camera module 104 can be configured to detect infrared (IR) energy from which the subject's gait may be recorded directly or indirectly, or as a supplement to improve the accuracy of the calculated gait parameters of a subject determined using images acquired by the camera module 104. Preferably, the emitters are mounted in a circular fashion around the perimeter of the lens of the camera module 104. In some embodiments, the device 100 further comprises a second gait sensing means 104 which can be used to augment calculated gait parameter values. In yet another embodiment an IR sensor or a high-sensitivity IR sensor can be used to determine at least one of the following: depth information, height of the device 100, relative distance to the subject, and angle of the device relative to a horizontal or vertical axis. An IR sensor may also detect low light conditions during use of the device. Low light information from the IR sensor can be used by the microprocessor of the device 100 during gait analysis.

[74] Typically, the camera module 104 is a monocular camera module which has a lens with a focal length shorter than the short side of the film or sensor, such as a curvilinear 'fisheye' lens or a rectilinear lens. Preferably the camera module 104 has interchangeable lenses such that the device 100 can be adapted for use in a number of different environments and particularly, with different types of ambulatory aids, such as a walking stick, cane, crutches, Zimmer frame, walker, or any other aid used to assist with gait mobility. Preferably such ambulatory aids are of the type that maintain continuous contact with the walking surface when in use, so as to maintain a constant distance between the camera module 104 and the walking surface thereby simplifying the processing of images of the subject's feet acquired by the camera module. In some embodiments the gait monitoring device 100 can be adapted to collect further data such as pressure data from a pressure sensor, or GPS or other data from another sensor or measurement instrument to improve or augment the calculation of at least one gait parameter of the subject.

[75] Wherein IR sensing is used, the IR filters commonly associated with most small digital camera sensors may be omitted to allow the camera to detect energy in the IR wavelength.

[76] Figure 2 of the present disclosure depicts a gait monitoring device 200 removably mounted to an ambulatory aid 210. The device 200 includes a housing 202 with a gait sensing means 204 configured to face towards the legs and/or feet of a subject or patient. The housing 202 is adapted to couple with a mounting means 206 such that it can be removably mounted at a predetermined location 208 on the ambulatory aid 210. The mounting means 206 can be used to pivot and relocate the gait monitoring device 100, 200. Thus, the gait monitoring device 200 may be removed from one ambulatory aid 210 and used on a different aid.

Alternatively/additionally, the gait monitoring device 200 may be moved to a different mounting location on the same ambulatory aid 210 to assess different gait

parameters. The mounting means 206 may include an omnidirectional pivot such that the device 100, 200 can be orientated in a desired direction. Preferably the device 100, 200 is mounted on an ambulatory aid 210 such that the camera module 104, 204 is oriented at a downward angle such that the gait of a subject can be recorded by capturing images of the subject's feet. More preferably the camera module 204 is oriented such that a region of interest, for example, between a subject's knees and a floor surface including the subject's feet can be captured and recorded to assess the gait of the subject.

[77] In yet another embodiment the software associated with the device 1 00, 200 is configurable to differentiate background image content from foreground image content including the subject's feet and to isolate image components corresponding to a lower leg region, particularly including the feet of the subject for analysis.

Preferably the ambulatory aid 210 has wheels or casters 212 or other means for maintaining continuous contact with the walking surface, to reduce unnecessary movement of the gait monitoring device 100, 200 during patient movement or walking. Ideally, the position of the gait sensing means 204 relative to the walking surface remains substantially constant during monitoring so as to not adversely affect image capture or gait analysis to a significant degree. It is to be understood however, that effects on the image of the relative movement between the gait sensing means and the walking surface can be removed during signal processing if necessary.

[78] In one embodiment the device 100, 200 is a single module system in a package (SiP) operated by a single-board computer, such as a Raspberry PiTM.

Figure 3 illustrates schematically an example of a printed circuit board (PCB) 300 comprising a camera serial interface (CSI) 302 configured to receive a signal from gait sensing means 104, 204 (Figures 1 and 2). The PCB 300 receives power from a power source through a power input 304, such as a Micro-USB power input 304 which can charge rechargeable battery 306. A microprocessor 308 is configured to control the transfer of data to from the gait sensing means 104, 204 to a memory storage 324. The microprocessor 308 can be configured to convert analogue signals to digital signals and/or digital signals to analogue utilising either hardware or software modules to achieve this output. Optionally, there are a number of external output/input interfaces associated with the PCB 300 including, USB ports 310, Ethernet port 312, HDMI port 314, video output 316 and audio output 318. In a preferred embodiment a plurality of status lights, such as LEDs, indicate whether the device is active, inactivate, in standby mode, charging, low power, experiencing an error or any other predetermined status effect. In one embodiment, the PCB 300 also has a display serial interface (DSI) 322 such than a screen can be used with the device 100, 200. It should be noted that while Figure 3 depicts a plurality of voltage regulators and power management circuitry, other electronic components, such as oscillators and phase-locked loops, can be configured or interchanged with the components illustrated. In at least one embodiment oscillators and phase-locked loops can be used to regulate recording and assessment of the gait of a subject at predetermined or irregular time intervals.

[79] In some embodiments, the device has at least one of a; HDMI port 314, a USB port 310, an Ethernet port 312, an audio output 318, and a video output 316 for transferring data either to or from the device. Stored data can be transferred to a standalone or other remote device, such as a personal computer or mobile device, to be analysed to determine at least one gait parameter of a subject although

microprocessor 308 may alternatively/additionally analyse stored data in this way. Preferably at least one status indicator 320, such as an LED, can be visible from the external surface of the housing to indicate whether the device is operating as intended or experiencing a suboptimal status. Referring to Figure 4C , a power switch or button 410 can be positioned on the device housing 102, 202, 402 such that a subject can manually activate, deactivate or put the device in a standby or other predetermined mode.

[80] The gait monitoring device 100, 200 can be used to calculate a subject's gait parameters. The device 100, 200 includes: (a) a gait sensing means 104, 204 for sensing directional and spatial variation in a subjects gait, (b) a transducer in communication with the gait sensing means 104, 204, (c) a microprocessor 308 having a memory 324, (d) the microprocessor in communication with the transducer, (e) firmware for controlling the operation of the microprocessor to sample the output of the gait sensing means at a predetermined time interval (t) and to temporarily store the sampled data in the memory 324, (f) a wireless or external interface to allow for transfer of selected stored data to an output device, and (g) software for controlling the operation of the output device to assess a subject's gait parameters. [81 ] In some embodiments the output device is, for example, a display screen, a speaker, a personal computer, a mobile phone, a tablet device or any suitable device capable of issuing an alert message. The output device can be integral to the device 100, 200 or a remote standalone or other device configured to issue updates or alerts to a healthcare professional, carer or assigned person with the subject's current or historically recorded gait parameters to intermittently or continuously monitor the gait of a subject. Updates, information and alerts issued to a healthcare professional or carer can assist with increasing a subject's compliance with a physical therapy regime or subject's prescribed health goals, and allow for a healthcare professional or carer to intervene if a subject has fallen, is likely to experience a fall or other walking related incident.

[82] The software associated with the device 100, 200 and adapted to run on the microprocessor includes a calibration module configured to automatically detect the orientation of the device , such as the relative angle, pitch and the height of the device relative to the horizontal and vertical axes of the ambulatory aid and/or with respect to the subject's walking surface. This spatial positioning information or orientation information can be used to calibrate the recorded gait information or data observed by the camera. In some embodiments an accelerometer, a three-axis accelerometer and/or a gyroscope can be used to determine the relative angle, pitch, spatial and directional variations, and a relative height of the device 100, 200. This allows the device 100, 200 to operate with a relatively high degree of accuracy in the event that the subject inadvertently moves the device and also allows the device to be adapted for use with multiple ambulatory aid types.

[83] The memory associated with the device 100, 200 can store at least one of the following: data from the gait sensing means 104, 204, at least one derived gait parameter value, predetermined gait threshold parameters, visual representations of a gait parameter, visual representation of multiple gait parameters, event triggering and user notification statuses, a pre-recorded alert message, triggering an alarm or an alert in the event of a fall or imminent fall, and prioritisation of case-load based on event triggering to name a few. Stored data can be sent to a remotely located device such as a standalone or other device and used to alert a medical professional or carer if a subject's gait is outside of the predefined threshold parameters, is displaying gait abnormalities, or if the subject has fallen or stopped walking. In at least one embodiment the device 100, 200 is configured to receive signals from one or more additional gait monitoring means 104, such as a stereoscopic camera module, an array of cameras, echo location means, IR sensors, ultrasonography sensors, thermographic image sensors, GPS, a proximity sensor or any other suitable means for detecting gait of a subject to increase accuracy of the detected gait parameters. In at least one embodiment data is recorded remotely e.g. in a data warehouse or in cloud storage to retain gait parameter values for an extended period of time. The recorded data stored in the memory may include information relating to a

predetermined time interval, wherein the time interval relates to a rolling window of data.

[84] The software associated with the device 100, 200 is configured to process recorded image data, virtually segment an image, and assign a virtual point tracking or virtual marker system to a feature of a recorded image. Typically, the feature of the recorded image to which the virtual marker system is assigned is a foot of the subject which is identified by the software during the virtual segmentation process. Preferably, the forward-most part of the foot is the feature that is assigned a virtual marker system, although additional or alternative features of the subject's anatomy could be identified and assigned a virtual marker system according to some embodiments of the invention. A subsequent recorded image can also have a virtual marker system assigned thereto, such that 3-dimensional vectors in co-ordinate space can be calculated for each image and compared to determine the gait of a subject, and to compare the subject's current gait to at least one of the predetermined threshold parameters. Other sensors can be used to augment or otherwise manipulate the recorded data. For example, data from IR sensors can be used to compensate for low light conditions, and pressure data from a pressure sensor located in one or both shoes of the subject may be used to supplement data recorded based on images acquired from the gait sensing means 104,204.

[85] Figures 4A and 4B illustrate yet another embodiment of the present disclosure. A gait monitoring device 400 is removably mounted on an ambulatory aid by a clip or clamp. The device 400 comprises a housing 402, preferably formed from a water resistant and wear resistant material, such as a polymer or metal alloy. The housing may be preferably made of: stainless steel, ABS, PEEK or polyurethane. Preferably, A camera module 404 is positioned off centre on the exterior of the housing 402 facing the towards the patient's feet in such a fashion that the

ambulatory aid support 406 is adapted to not obstruct the visual pathway between the camera module 404 and the region of interest of which images are captured.

Preferably, the housing of the device is generally box shaped and includes a face, a first and second side; the camera module 404 is preferably positioned on the face wherein the camera module 404 and/or lens is disposed proximal or closer to the first side, wherein the second side is proximal to attached portion of the ambulatory aid. The ambulatory aid preferably comprises wheels or casters 408 to provide a more consistent image capture to reduce potential errors arising while monitoring the gait of a subject.

[86] Figure 4C depicts another embodiment of the gait monitoring device 400. The device 400 is mounted on an ambulatory aid and positioned at a downward angle to record the gait of a subject. The device 400 comprises a toggle switch or button 410 which can be used to activate or deactivate the device 400. A power input port 412 allows the device 400 to be recharged at a recharging station or plugged into a power outlet either while mounted to the ambulatory aid or when detached. Transfer means 414 can be adapted for use with a wireless transceiver or other means used to transfer data to e.g. a standalone device, such as a personal computer, mobile phone, tablet device or the like.

[87] In another embodiment, the button 410 can be configured to turn or cycle the device 400 to a standby, periodic capture or another predetermined mode. The periodic capture mode can intermittently switch the device 400 between a recording and assessing mode to a standby mode to conserve battery power or temporarily monitor the gait of a subject during a monitoring period.

[88] In a preferred embodiment the device 100, 200, 400 can be configured to detect and assess at least one of the following gait parameter values of a subject: right leg stride length, left leg stride length, ambulation time, walking velocity, walking distance, number of steps taken by a subject, cadence, gait cycle time, left single leg stance time, right single leg stance time, double leg stance time, right leg swing time, left leg swing time, left foot clearance, right foot clearance, double leg stance base of support. The detected gait parameters can subsequently be compared with a subject's baseline or threshold parameters and an alert can be issued if the subject is nearing at least one predetermined gait threshold parameter or has exceeded at least one predetermined gait threshold parameter.

[89] Figure 5 is a flowchart of an embodiment of a method 500 for calculating the gait parameters of a subject. In use, the camera module 104, 204, 404 can be calibrated 502 for recording at least one gait parameter of the subject. The calibration can be used to determine the intrinsic and extrinsic camera module calibration parameters for determining the relative position of the gait monitoring device relative to the subject and/or relative to the walking surface. Extrinsic camera module calibration parameters include the angle of inclination of the camera module relative to a predetermined axis and, the height of the camera module relative to the walking surface. Intrinsic camera module calibration parameters include the focal length of the lens and the optical centre of the lens. In the event that the camera module 104, 204, 404 is determined to have been calibrated 504, the device reassesses whether further calibration is required 502 after the initial calibration. If the camera module is correctly calibrated, the camera module records at least one image 506 or video of the subject and temporarily stores the image in the memory. After recording at least one image, the image is digitally processed 508 and a predetermined or adaptive number (one or more) of marker points are assigned to the image 510 to record and assess the current gait of the subject. Based on the assigned marker points a series of 3D co-ordinate vector points can be calculated 512 in a 3D co-ordinate space and compared to a second image also with assigned marker points. The co-ordinate space can be, for example, a virtual box or virtual space corresponding to the region of interest in which a subject's legs and particularly their feet, are expected to be observed during ambulation. Ideally, the coordinate space is referenced to the calibration data obtained at 502. The vector points are subsequently used to calculate and assess at least one current gait parameter value 514 of the subject and are used to compare the current gait parameter with at least one predetermined gait threshold parameter of the subject 516, or a previous gait parameter of the subject.

[90] In some embodiments such as in a continuous monitoring application of the invention, in the event that a predetermined number of gait threshold parameters are exceeded 516 an alert can be issued 518. If the predetermined number of gait threshold parameters are not exceeded the device determines whether further monitoring of the subject is required. If further monitoring of the subject is required (e.g. in the case of "continuous monitoring" use of the invention as opposed to limited duration "assessment") the process repeats 520 from the step determining whether a camera calibration is required 502. If further monitoring is not required, the device can be configured to deactivate or enter a standby mode 522 to conserve power. While in an active or standby mode the device can transmit data to another device, such as a personal computer, tablet or other mobile computing device, and can preferably be remotely activated.

[91 ] When the gait of a subject is within the predetermined number of gait threshold parameters this is indicative of a satisfactory subject gait and no alert would be issued. The device may also detect when a subject has fallen, is predicted to fall or experiencing difficulty walking e.g. by identifying anomalous patterns in the recorded gait, and to alert healthcare professionals or carers to respond to the incident. In some embodiments the severity of an incident triggering an alarm (e.g. actual fall versus predicted fall) can place the subject in a response queue such that a healthcare professional or carer can prioritise their response to cases with a high risk of subject injury or a high risk of a subject falling.

[92] Figure 6 illustrates a method 600 for using a preferred embodiment of the device 100, 200, 400 of the present disclosure. In use, a predetermined number of reference or threshold gait parameters 602 are stored in the device 100, 200, 400 memory 324 or an external memory accessible by the device 100, 200, 400. The gait monitoring means 104, 204, 404 can detect and record at least one image of the gait of a subject and analyse the image to assess the current gait parameters of the subject 604. In some embodiments the number of images recorded per second can be, for example, between 0 to 72 images but faster framerates are contemplate for high definition analysis, and slower framerates are contemplated e.g. for devices where processing and battery limitations are incompatible with higher definition gait recording and processing. A comparison between the threshold parameters and the recorded current gait parameters 606 may be used to determine one or more of: the subject experiencing an irregular gait, a regular gait, about to fall, has fallen or is having walking difficulties. If the device 100, 200, 400 assesses and determines whether a subject has exceeded a first predetermined number of threshold

parameters 608, the device can issue a first trigger alert 610. The first trigger alert 610 can be triggered if the subject exceeds a first predetermined number of gait threshold parameters 608 or exceeds a gait threshold parameter by more than a predetermined upper limit. The first trigger alert 610 can assign priority to the subject such that a carer can respond immediately to attempt to prevent a potential fall of the subject.

[93] If the subject has not exceeded the first predetermined number of gait threshold parameters to trigger a first alert, but has exceeded a second

predetermined number of threshold parameters 612, a second alert is triggered 614. In the event that the current gait parameters of the subject has not exceeded the second predetermined number of threshold parameters no alert is issued. If no alert is triggered a check to repeat the monitoring of the subject at a predetermined time (t) interval 616. While it is not illustrated if an alert has been triggered, further

assessment of the gait of the subject can be continued from step 604.

[94] In another embodiment the device 100, 200, 400 continually monitors the subject and does not make a check to determine whether to repeat the monitoring process 616. In this embodiment after the device 100, 200, 400 has determined that the second number of predetermined threshold parameters have not been exceeded the device again detects the current gait of the subject 604. The device 100, 200, 400 can be configured to issue one or more alarms based on any number of threshold values. For example, if a subject's gait parameter exceeds or is outside a reference value, or exceeds or falls short of X predetermined number of threshold parameters, the device can issue an alert. Various alert types may be stored in memory 324 and assigned according to e.g. X threshold parameters being exceeded, or the subject's gait parameters falling short of X threshold parameters, or one or more of the subject's gait parameters having values that do not correspond with a reference value representing a "normal" or acceptable gait. It is to be understood that the reference value may be a range or reference values, or a function for determining the reference value.

[95] The first and second trigger alerts can be communicated using at least one of; a text message, an audio message, an alarm, a noise, a light, a vibration, or any other means to draw the attention of a healthcare professional, carer or assigned person or even the subject him or herself. [96] Figure 7 depicts a gait monitoring device 700 according to another embodiment of the invention, which is incorporated into an ambulatory aid 710 in the form of a wheeled walking frame. Housing 702 includes a transparent front face behind which is contained a digital image capture device, typically a digital camera or the like, configured to capture one or more images of the subject during ambulation on a walking surface. Also in the housing 702 is a computer-implemented image processor (not shown) for processing the one or more captured images. In one embodiment, this is achieved by automatically identifying in one or more of the captured images one or more relevant features of the subject and allocating a virtual marker to the one or more relevant features. The one or more features of the subject can be found, during ambulation, in region of interest 720 on which the camera is focussed during use. These features typically include the subject's feet and more specifically, the forward-most point on the subject's feet.

[97] Gait monitoring device 700 may further include a calibration module configured to determine automatically an orientation of the digital image capture device/camera, relative to the walking surface. Ideally, the calibration module uses the orientation information to provide a reference system for the virtual marker data. Preferably, the reference system is a 3D coordinate reference system, such that the virtual marker data assigned to features in images captured by the camera can be ascribed values according to the reference coordinates defined by reference to the walking surface. Thus, it is also preferred that the virtual marker assigned to the captured images includes a 3D coordinate vector.

[98] A transceiver (not shown) is also provided as part of the ambulatory aid 710 although it need not be in the housing 702. The transceiver transmits one or both of the captured image data and the processed image data to a remote processor. For ease of image processing, in the embodiment shown the ambulatory aid has substantially continuous contact with the walking surface during use, by means of wheels or castors 712. Advantageously, this gait monitoring device including the camera is mobile to the extent that the subject and the ambulatory aid 710 are mobile. This overcomes short comings of prior art gait monitoring systems using image capture which utilise a fixed location camera attached to a wall, floor, treadmill or other fixed structure. In a further advantage, because the orientation of the camera with respect to the walking surface remains substantially constant, image processing is relatively straightforward due to the stability of the captured images.

[99] Ideally gait monitoring device 100, 200, 400, 700 is ideally configurable to interoperate with or form part of a system for monitoring the gait of a subject illustrated schematically in Figure 8, according to an embodiment of the present invention.

[100] Figure 8 shows a digital image capture device (camera) 804 configured to capture one or more images of the subject during ambulation on a walking surface. The image capture device 804 may be provided in a housing that is attachable to an ambulatory aid (Figures 2, 4A, 4B, 4C), or it may be provided in a housing that is integral with an ambulatory aid (Figure 7). Digital image capture device 804 may be e.g. a monocular camera, a stereoscopic camera, an infrared image sensor or such device capable of capturing sequential images. Ideally, for ease of image processing, the ambulatory aid has substantially continuous contact with the walking surface during ambulation.

[101 ] Calibration module 802 is typically provided with the image capture device 804 and is configured to determine automatically an orientation of the image capture device relative to the walking surface. That is, calibration module 802 is configured to determine the height and angular orientation of the image capture device 804 relative to the walking surface. This orientation data provides a reference system, typically a 3D coordinate reference system, for the virtual marker data that is calculated by image processor 806 since location of the camera remains substantially fixed relative to the walking surface during ambulation and capture of the images.

[102] Computer-implemented image processor 806 automatically identifies one or more relevant features of the subject and allocates a virtual marker to those features. As described previously, typically the virtual marker includes a 3D

coordinate vector. A relevant feature may be any distinct pixel in the captured image corresponding to features of the left and/or right foot of the subject, which can be robustly identified. Typically, the image processor 806 automatically identifies, using algorithms known in the art, the forward-most point on the subject's feet to which a virtual marker is then assigned. However it is to be understood that other features of the foot or indeed of the subject's legs may be utilised in the calculation of one or more gait parameters of the subject, according to the embodiments disclosed herein.

[103] In some embodiments, software embodied in one or more of the image processor 806 and the gait processor 808 may be "trained" onto a particular subject, by first capturing an image of the subject's left and/or right foot, and then selecting in the captured image the "features" to which a virtual marker is to be automatically assigned during monitoring. This training may be performed on one image, or on a plurality of images collected over time, as the subject ambulates. This enables development of feature detection specific to a subject which may improve monitoring performance, e.g. where a subject's gait leads to occlusion of the feet.

[104] Computer-implemented gait parameter processor (gait processor 808) processes the virtual marker data and generates one or more gait parameters, typically primary gait parameters. Table 1 indicates a non-exhaustive list of primary gait parameters that can be determined using data from image processor 806.

Secondary gait parameters may be calculated by secondary gait processor 810. Table 2 indicates a non-exhaustive list of secondary gait parameters that are typically derived from primary gait parameters, although they may also be determined directly from data from image processor 806. Ideally the system 800 determines further relevant data such as start time and finish time of an assessment of a subject's gait; number of gait cycles in the assessment; and total time spent ambulating during the assessment.

Data

Metric Description

Permutations

Distance right foot travels during

Right Step Length

swing phase measured from the toe Min, Mean, Max (cm)

of the left foot

Distance left foot travels during swing

Left Step Length (cm) Min, Mean, Max phase measured from toe of right foot

Sum of Right and Left step length

Stride Length (cm) Min, Mean, Max during a gait cycle

Time to complete one full gait cycle

Gait Cycle time (sec) Min, Mean, Max

(right toe off to next right toe off)

Right Single Leg % Time during a gait cycle that only

Min, Mean, Max Stance time (%) the right foot contacts the ground Left Single Leg Stance % Time during a gait cycle that only

Min, Mean, Max time (%) the left foot contacts the ground

Double Leg Stance % Time during a gait cycle that both

Min, Mean, Max time (%) feet contact the ground

% Time during gait cycle that right

Right Swing time (%) Min, Mean, Max foot is not contacting the ground

% Time during gait cycle that left foot

Left Swing time (%) Min, Mean, Max is not contacting the ground

Right Foot clearance Maximum distance between ground

Min, Mean, Max (cm) and right foot during swing phase

Left Foot clearance Maximum distance between ground

Min, Mean, Max (cm) and left foot during swing phase

Double Leg Stance Percentage of gait cycle spent with

Min, Mean, Max base of support (%) both feet on the ground

Table 1

Data

Metric Description

Permutations

At mean walking

Stride Length Variation in stride to stride length at a

velocity, At max Variability (%) given velocity

walking velocity

Ambulation time Total time spent walking in a given

Min, Mean, Max (duration) time period

Walking Velocity (m/s) Stride length over time Min, Mean, Max

Total distance walked in a given time

Walking Distance (m) Min, Mean, Max period

Sum of right and left steps in a given

Number of Steps Min, Mean, Max time period

At mean walking

Cadence (steps/min) Number of steps per minute velocity, At max walking velocity

Table 2

[105] In Figure 8, broken lines form boxes around different components of the system 800 for monitoring the subject's gait. These boxes designate different implementation architecture options as are contemplated by the current disclosure and within the scope of the claimed invention. [106] In one embodiment, the image capture, calibration, image processing and gait parameter processing are all performed in a device according to Architecture A. These components are shown contained within the broken-line box designated "A" and are typically embodied in a physical device which is located on or incorporated into the ambulatory aid. This architecture implementation is "device heavy" since considerable processing power and energy needs must be satisfied by the

responsible device which is typically battery operated and has size and weight limitations due to the mobility of the device.

[107] In another embodiment, the image capture, calibration and image processing are performed according to Architecture B. Certain components are shown contained within the broken-line box designated "B" and are typically embodied in a physical device which is located on or incorporated into the ambulatory aid. This architecture implementation is somewhat "bandwidth heavy" since considerable transmission bandwidth is required to transmit the processed image data to the gait parameter processor 808 which is typically located remotely from the device comprising the components of Architecture B. Either or both of image processor 806 and gait parameter processor 808 may be embodied in software deployed on any suitable processing means. Such means may comprise a remote personal computer as may routinely be used to provide access to display device 820. Alternatively/additionally, the secondary gait processor 810 may be embodied on a remote server accessible via communication network 812 or other means.

[108] In another embodiment, the image capture and calibration are performed according to Architecture C. Certain components are shown contained within the broken-line box designated "C" and are typically embodied in a physical device which is located on or incorporated into the ambulatory aid. This architecture implementation is "bandwidth heavy" since raw captured image data sizes are typically large. These require transmission to the image processor 806 which is typically located remotely from the device comprising the components of Architecture C. Either or both of image processor 806 and gait parameter processor 808 may be embodied in software deployed on any suitable processing means such as a remote personal computer as may routinely be used to provide access to display device 820 or a remote server accessible via communication network 812 or other means. [109] In each implementation architecture, gait parameters of the subject once calculated, are typically communicated, via a wireless communication network 812 to a display device 820 for presentation to a user such as a healthcare professional, carer or the like. Display device 820 may in turn communicate with or incorporate a processor performing the functions of secondary gait processor 810. It is to be understood that this communication is not exclusive of communication of gait related data to the subject through a feedback device located on the ambulatory aid to provide feedback to the subject on their walking behaviour. The feedback device may provide an audible message, an alarm, a noise, a light, a vibration, or any other means to draw the attention of the subject while being monitored.

[1 10] In some embodiments, the secondary gait processor 810 incorporates or is supplemented by a gait data processor that is configured to calculate notifications, aggregate data from the image processor 806 and/or the gait processor 808, generate gait assessment reports for one or more subjects who have been monitored, and perform other data presentation and manipulation tasks which may be

substantially automated or performed manually by a user.

[1 1 1 ] The gait data processor may generate automated notifications e.g. when a comparison of gait parameter values for a subject do not correlate with one or more reference values for that parameter. A reference value may be a fixed reference value, a range of values, or a function for calculating a reference value. Data aggregation may involve collating data from a single subject, or from a number of different subjects who have been monitored using the system 800. In this regard, database 814 and memory service 816 accessible via communications network 812 (typically via the cloud) enable a user to access data acquired by equivalent systems deployed at other sites both locally and at more distant locations.

[1 12] Figure 9 is an example of a dashboard presenting aggregated data for use on display device 820. Overview tab 910 presents an overview of the subject's walking behaviour over time as monitored according to embodiments of the invention, wherein various gait parameters are presented. Summary data including mean, maximum and minimum values as calculated by the gait data processor are also presented to provide an overview and reference points with which future data may be compared to monitor improvements or deterioration in the subject's walking behaviour. Monitor tab 920 provides detailed analysis of data collected by device and allows the user to explore in more detail all data collected and all permutations of the data. Alerts tab 930 presents a list of alerts that have been generated by the system. Numerical indicator 932 indicates the number of unattended alerts, in this example there are 5. Notes tab 940 enables a user to make notes about the subject or the assessment, and to access and in some cases, edit previously made notes. Reporting tab 950 enables the user to create, edit and send assessment reports on the subject's gait parameters. Profile tab 960 is where the subject's personal, health and

demographic information is entered, edited and displayed.

[1 13] The devices and systems described herein are configurable to perform methods for monitoring the gait of a subject in which an image capture device is operated to capture at least one image of a region of interest of the subject during ambulation on a walking surface. The captured images are automatically processed to identify pixels in the images representing a relevant feature of the subject, such as the foot or front-most point on the foot, assigning a virtual marker (typically a 3D coordinate vector) to the identified pixels and processing the virtual marker to calculate a gait parameter for the subject which is indicative of the gait of the subject being monitored. Typically, calculating the gait parameter includes comparing a virtual marker assigned to a captured image with a virtual marker assigned to a

subsequently captured image, determining movement of the relevant feature of the subject between the two captured images, and calculating the gait parameter according to the determined movement.

[1 14] Ideally, the gait monitoring method includes first calibrating the image capture device by determining automatically the orientation of the image capture device relative to the walking surface. The orientation information obtained during calibration is used to provide a reference system for virtual markers assigned to captured images. Ideally, the image capture device is attached to or incorporated in an ambulatory aid having substantially continuous contact with the walking surface during use e.g. using wheels or casters, so as to maintain a substantially constant orientation of the image capture device relative to the walking surface. This is advantageous in that avoids image processing demands arising when the image capture device must constantly recalculate points of reference. [1 15] In some embodiments, the method includes processing one or more gait parameters of the subject to identify an alert condition, by comparing values calculated for one or more gait parameters of the subject with one or more reference values for those gait parameters and confirming an alert condition if one or more of the subject's gait parameter values differ from the respective reference values by more than a pre-determined amount. An alert may be generated e.g. if values calculated for one gait parameter of the subject differ from the respective reference value by more than the pre-determined amount and/or if values calculated for a plurality of gait parameters of the subject differ from the respective reference values by more than a pre-determined amount for each parameter. An alert may be generated by any suitable means including but not limited to text message, a noise alert, vibrations, a video message, illumination of a lighting element, and a flagged status update in live monitoring data.

[1 16] There are numerous advantages associated with use of the present invention. Firstly, use of image processing as the primary sensing methodology provides a direct indication of the subject's movement during ambulation and hence their gait by acquiring images of the subject's foot and lower leg. This is more reliable that other gait monitoring systems that use pressure sensors, accelerometers and the like because they provide only indirect indications of the subject's gait based on data interpretation models that can be unreliable across a spectrum of individuals, particularly when they have very different gait characteristics. Secondly, use of the image processing techniques according to embodiments disclosed herein are robust and efficient since the image capture device is typically mounted on or incorporated into an ambulatory aid that has substantially continuous contact with the walking surface during ambulation. This reduces signal processing overhead, since the acquired images have greater stability and the assigned virtual markers have a fixed reference point, defined by the location of the walking surface relative to the image capture device. This streamlines processing of images acquired over time and the detection of movement patterns of features in those images as identified by the device.

[1 17] Additionally, the inventive monitoring device, system and method provides an objective framework for monitoring the walking characteristics, or gait of a subject. Objective analysis removes inconsistencies in manual assessments undertaken by human observation of how the subject walks. Furthermore, objective assessment reports created according to embodiments of the invention provide reference data for rehabilitation programs and further healthcare assessment including neurological, physiological, cognitive and physical therapy assessments. Some embodiments provide alerts when recorded gait parameters indicate a fall, or irregular gait parameters monitored over time indicate a high likelihood of a fall occurring. These features may be incorporated into an ambulation aid which, when in regular or daily use, provides the subject with more confidence during mobility, since the system provides means for providing feedback when the subject is likely to have a fall, and transmit an alert for assistance in the event that an actual fall is detected.

[1 18] Although the invention has been described with reference to specific examples, it will be appreciated by those skilled in the art that the invention may be embodied in many other forms, in keeping with the broad principles and the spirit of the invention described herein.

[1 19] The present invention and the described preferred embodiments specifically include at least one feature that is industrial applicable.

[120] It is to be understood that various modifications, additions and/or alterations may be made to the parts previously described without departing from the ambit of the present invention as defined in the claims appended hereto.

Claims

1 . A system for monitoring gait of a subject, the system including:
(a) a digital image capture device configured to capture one or more images of the subject during ambulation on a walking surface;
(b) a computer-implemented image processor for processing the one or more captured images, the image processor being configured to process the one or more captured images by automatically identifying in one or more of the captured images one or more relevant features of the subject and allocating a virtual marker to the one or more relevant features;
(c) a computer-implemented gait parameter processor for processing virtual marker data from the image processor and generating one or more gait parameters; and
(d) a display device configured to display one or more of the gait
parameters calculated for the subject;
wherein the one or more gait parameters are indicative of the gait of the subject being monitored.
2. The system according to claim 1 , wherein the digital image capture device is provided in a housing that is attachable to an ambulatory aid having
substantially continuous contact with the walking surface during use.
3. The system according to claim 1 , wherein the digital image capture device is provided in a housing that is integral with an ambulatory aid having
substantially continuous contact with the walking surface during use.
4. The system according to any one of the preceding claims, wherein the digital image capture device is selected from a group including: a monocular camera, a stereoscopic camera, and an infrared image sensor and any of the foregoing capable of capturing sequential images.
5. The system according to any one of the preceding claims, further including a calibration module configured to determine automatically an orientation of the digital image capture device relative to the walking surface and using the orientation to provide a reference system for the virtual marker data.
6. The system according to claim 5, wherein the calibration module uses the orientation of the digital image capture device to determine a 3D coordinate reference system for the virtual marker data.
7. The system according to any one of the preceding claims, wherein the virtual marker data includes a 3D coordinate vector calculated by the computer- implemented image processor for each virtual marker.
8. The system according to any one of the preceding claims, wherein the one or more relevant features are features of the left and/or right foot of the subject.
9. The system according to claim 8, wherein the relevant features include at least the forward-most point on the subject's left and/or right foot.
10. The system according to any one of the preceding claims, wherein the
computer-implemented gait parameter processor calculates one or more gait parameters by comparing virtual marker data associated with a captured image with virtual marker data associated with a previously captured image.
1 1 . The system according to any one of the preceding claims, wherein the gait parameters calculated by the computer-implemented gait parameter processor include primary gait parameters selected from the group including:
(a) stride length;
(b) right step length;
(c) left step length;
(d) gait cycle time;
(e) right single leg stance time;
(f) left single leg stance time;
(g) double leg stance time;
(h) right leg swing time;
(i) left leg swing time;
(j) right foot clearance;
(k) left foot clearance; and
(I) double leg stance base of support
12. The system according to claim 1 1 , further including a computer-implemented secondary gait parameter processor configured to calculate from the virtual marker data and the primary gait parameters, one or more secondary gait parameters selected from the group including:
(a) ambulation time;
(b) walking velocity;
(c) walking distance;
(d) number of steps taken;
(e) cadence; and
(f) stride length variability.
13. The system according to any one of the preceding claims, further configured to determine automatically one or more of: start time and finish time of an assessment of a subject's gait; number of gait cycles in the assessment; and total time spent ambulating during the assessment.
14. The system according to any one of the preceding claims, further including a computer-implemented gait data processor configured to perform one or more of:
(a) calculating automated notifications based on a comparison of virtual marker data for the subject with one or more reference values;
(b) aggregating data from one or both of the computer-implemented image processor and the computer-implemented gait parameter processor, and
(c) generating assessment report data suitable for display on the display device or transmission to another computer-implemented processing device.
15. A method for monitoring gait of a subject including the steps of:
(a) operating an image capture device to capture at least one image of a region of interest of the subject during ambulation on a walking surface;
(b) image processing the at least one image to identify automatically a relevant feature of the subject and assigning a virtual marker to the identified feature; and
(c) processing the virtual marker to calculate a gait parameter for the subject; wherein the gait parameter is indicative of the gait of the subject being monitored.
16. The method according to claim 15, wherein the virtual marker comprises a 3D coordinate vector.
17. The method of claim 15 or claim 16 wherein calculating the gait parameter includes comparing a virtual marker assigned to a captured image with a virtual marker assigned to a subsequently captured image, determining movement of the relevant feature of the subject between the two captured images, and calculating the gait parameter according to the determined movement.
18. The method according to any one of claims 15 to 17, including a calibrating step that includes determining automatically the orientation of the image capture device relative to the walking surface, wherein orientation information obtained during calibration is used to provide a reference system for virtual markers assigned to the captured images.
19. The method according to any one of claims 15 to 18, wherein the image
capture device is attached to or incorporated in an ambulatory aid having substantially continuous contact with the walking surface during use, so as to maintain a substantially constant orientation of the image capture device relative to the walking surface.
20. The method according to any one of claims 15 to 19, further including the step of processing one or more gait parameters of the subject to produce
assessment data, wherein the assessment data is configurable to produce one or more of a dashboard, chart, report or visual representation of the subject's gait to a third party.
21 . The method according to any one of the preceding claims, further including the step of processing one or more gait parameters of the subject to identify an alert condition, by comparing values calculated for one or more gait
parameters of the subject with one or more reference values for those gait parameters and confirming an alert condition exists if one or more of the subject's gait parameter values differ from the respective reference values by more than a pre-determined amount.
22. The method according to claim 21 , including the step of generating an alert when a condition is met including one or more of the following:
(a) values calculated for a particular gait parameter of the subject differ from the respective reference value by more than the pre-determined amount;
(b) values calculated for a plurality of gait parameters of the subject differ from their respective reference values by more than a pre-determined amount for each parameter; and
(c) values calculated for a plurality of gait parameters of the subject differ from their respective reference values by more than a predetermined sum total amount.
23. The method according to claim 22, wherein a generated alert is assigned an alert priority selected from a group including:
(a) first priority when the alert corresponds to detection of a fall;
(b) second priority when the alert corresponds to detection of a fall being likely;
(c) third priority when the alert corresponds to detection of a gait parameter indicative of possible injury to the subject occurring or likely to occur.
24. The method according to claim 22 or claim 23, wherein the alert is generated by means including one or more of: text message, a noise alert, vibrations, a video message, illumination of a lighting element, and a flagged status update in live monitoring data.
25. The method according to any one of claims 15 to 24, wherein the one or more relevant features of the subject are features of the left and/or right foot of the subject
26. The method according to any one of claims 15 to 25, wherein the one or more relevant features include at least the forward-most point on the subject's left and/or right foot.
27. A device for monitoring a gait of a subject, the device including:
(a) a housing containing a digital image capture device configured to capture one or more images of the subject during ambulation on a walking surface; and
(b) a computer-implemented image processor in the housing, for
processing the one or more captured images, the image processor being configured to process the one or more captured images by automatically identifying in one or more of the captured images one or more relevant features of the subject and allocating a virtual marker to the one or more relevant features; and
(c) a transceiver configured to transmit one or both of the captured image data and the processed image data to a remote processor;
wherein the housing is attachable to an ambulatory aid having substantially continuous contact with the walking surface during use.
28. A device comprising an ambulatory aid for monitoring gait of a subject, the
ambulatory aid having:
(a) a digital image capture device in a housing forming part of the aid and configured to capture one or more images of the subject during ambulation on a walking surface; and
(b) a computer-implemented image processor in the housing, for
processing the one or more captured images, the image processor being configured to process the one or more captured images by automatically identifying in one or more of the captured images one or more relevant features of the subject and allocating a virtual marker to the one or more relevant features; and
(c) a transceiver configured to transmit one or both of the captured image data and the processed image data to a remote processor;
wherein the ambulatory aid has substantially continuous contact with the walking surface during use.
29. The device of claim 27 or claim 28, configurable for use with:
(a) a computer-implemented gait parameter processor for processing virtual marker data from the image processor and generating one or more gait parameters; and
(b) a display device configured to display one or more of the gait parameters calculated for the subject;
wherein the one or more gait parameters are indicative of the gait of the subject being monitored.
30. The device according to any one claims 27 to 29, wherein the digital image capture device is selected from a group including: a monocular camera, a stereoscopic camera, and an infrared image sensor and any of the foregoing capable of capturing sequential images.
31 . The device according to any one of claims 27 to 31 , further including a
calibration module configured to determine automatically an orientation of the digital image capture device relative to the walking surface and using the orientation to provide a reference system for the virtual marker data.
32. The device according to claim 31 , wherein the calibration module uses the orientation of the digital image capture device to determine a 3D coordinate reference system for the virtual marker data.
33. The device according to any one of claims 27 to 32, wherein the virtual marker data includes a 3D coordinate vector calculated by the computer-implemented image processor for each virtual marker.
34. The device according to any one of claims 27 to 33, wherein the transceiver is configurable to interoperate with the system according to any one of claims 1 to 14.
35. A device for monitoring gait of a subject, the device comprising:
a gait sensing means for generating a gait parameter;
a transducer operatively associated with the gait sensing means;
a microprocessor in communication with the transducer; and
a memory circuit associated with the microprocessor for recording the gait parameter;
wherein gait parameter is compared with at least one predetermined threshold parameter to determine whether the gait of the subject has exceeded the at least one predetermined threshold parameter.
36. The device as claimed in claim 35, wherein the device comprises a mounting means such that the device is mountable on an ambulatory aid.
37. The device as claimed in claim 36, wherein the gait sensing means is disposed off-centre distal to the mounting means.
38. The device as claimed in any one of claims 35 to 37, wherein the gait sensing means is at least one of: a monocular camera module, a stereoscopic camera module, an infrared sensor, a proximity sensor or a combination thereof.
39. The device as claimed any one of claims 35 to 38, wherein an alert is triggered if a predetermined number of threshold parameters are exceeded.
40. The device as claimed any one of claims 35 to 39, wherein the gait sensing means comprises at least one of an accelerometer and/or a gyroscope adapted to determine spatial or directional variations in the gait of the subject.
41 . The device as claimed in claim 40, wherein the spatial or direction variations are determined by assigning a co-ordinate system to a first recorded image and comparing the co-ordinate system of the first recorded image with a coordinate system of at least one second recorded image.
42. The device as claimed in any one of claims 35 to 41 , wherein calibration of the gait sensing means determines at least one of; a relative height of the device, a relative location of the subject to the device, and an angle of inclination of the device.
43. The device as claimed in any one of claims 35 to 42, wherein the device
comprises a transceiver for sending and receiving data.
44. The device as claimed in claim 36, wherein the mounting means comprises an omnidirectional pivot for positioning the device.
45. The device as claimed in any one of claims 35 to 44, wherein the camera
module comprises a curvilinear lens or a rectilinear lens.
46. The device as claimed in any one of claims 35 to 45, wherein software associated with the microprocessor of the device is configured to calculate at least one gait parameter for the subject selected from the group consisting of: a) right stride length, b) left stride length, c) ambulation time, d) walking velocity, e) walking distance, f) number of steps taken by a subject, g) cadence, h) gait cycle time, i) left single leg stance time, j) right single leg stance time, k) double leg stance time, I) right swing time, m) left swing time, n) left foot clearance, o) right foot clearance, p) double leg stance base of support.
47. The device as claimed in any one of claims 35 to 46, employing a method for monitoring the gait of a subject, the method comprising the steps of:
A) recording at least one image,
B) virtually segmenting the at least one said and assigning a co-ordinate marker system thereto,
C) comparing the assigned co-ordinate marker system to at least one predetermined threshold parameter, and
D) determining whether a number of predetermined threshold parameters have been exceeded based on the comparison.
48. A process for monitoring gait of a subject, the process comprising the steps of:
A) recording at least one image associated with of the gait of the subject,
B) determining at least one directional or spatial variation of the gait of the subject based on the at least one said image,
C) comparing the at least one directional or spatial variation of the subject with at least one predetermined gait threshold parameter; and
D) determining whether a predetermined number of gait threshold parameters have been exceeded.
49. The process as claimed in claim 48, comprising the step of determining a
relative location of the subject.
50. The process as claimed in claim 48 or claim 49, comprising the step of
triggering an alert if the predetermined number of gait threshold parameters have been exceeded.
51 . The process as claimed in any one of claims 48 to 50, comprising the step of assigning virtual marker points to the recorded image to determine at least one directional or spatial variation of the subject.
52. A method for monitoring the gait of a subject, the method comprising the steps of:
A) recording at least one image,
B) virtually segmenting the at least one said image and assigning a coordinate marker system thereto,
C) comparing the assigned co-ordinate marker system to at least one predetermined threshold parameter, and
D) determining whether a number of predetermined threshold parameters have been exceeded based on the comparison.
53. The method as claimed in claim 52, comprising the step of calibrating a gait sensing means to record the at least one image by adjusting the focus of an associated camera module.
54. The method as claimed in claim 53, comprising the step of calculating a
relative location of the gait sensing means relative to the subject.
55. The method as claimed in any one of claims 52 to 54, wherein at least one alert message is triggered if the calculated parameters exceed the number of predetermined threshold parameters.
PCT/AU2015/050738 2014-11-24 2015-11-24 Gait monitoring system, method and device WO2016081994A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
AU2014904741A AU2014904741A0 (en) 2014-11-24 Gait monitoring method and/or device
AU2014904741 2014-11-24

Publications (1)

Publication Number Publication Date
WO2016081994A1 true true WO2016081994A1 (en) 2016-06-02

Family

ID=56073221

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2015/050738 WO2016081994A1 (en) 2014-11-24 2015-11-24 Gait monitoring system, method and device

Country Status (1)

Country Link
WO (1) WO2016081994A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020028003A1 (en) * 2000-03-27 2002-03-07 Krebs David E. Methods and systems for distinguishing individuals utilizing anatomy and gait parameters
US20030228033A1 (en) * 2000-05-18 2003-12-11 David Daniel Method and apparatus for remote medical monitoring incorporating video processing and system of motor tasks
WO2005125210A1 (en) * 2004-06-14 2005-12-29 Arizona Board Of Regents For And On Behalf Of Arizona State University Methods and apparatus for motion capture
US20120075464A1 (en) * 2010-09-23 2012-03-29 Stryker Corporation Video monitoring system
US20140148733A1 (en) * 2012-04-27 2014-05-29 The Curators Of The University Of Missouri Activity analysis, fall detection and risk assessment systems and methods
US20140270402A1 (en) * 2011-07-29 2014-09-18 University Of Ulster Gait recognition methods and systems
WO2014160451A1 (en) * 2013-03-13 2014-10-02 Virtusense Technologies Range of motion system, and method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020028003A1 (en) * 2000-03-27 2002-03-07 Krebs David E. Methods and systems for distinguishing individuals utilizing anatomy and gait parameters
US20030228033A1 (en) * 2000-05-18 2003-12-11 David Daniel Method and apparatus for remote medical monitoring incorporating video processing and system of motor tasks
WO2005125210A1 (en) * 2004-06-14 2005-12-29 Arizona Board Of Regents For And On Behalf Of Arizona State University Methods and apparatus for motion capture
US20120075464A1 (en) * 2010-09-23 2012-03-29 Stryker Corporation Video monitoring system
US20140270402A1 (en) * 2011-07-29 2014-09-18 University Of Ulster Gait recognition methods and systems
US20140148733A1 (en) * 2012-04-27 2014-05-29 The Curators Of The University Of Missouri Activity analysis, fall detection and risk assessment systems and methods
WO2014160451A1 (en) * 2013-03-13 2014-10-02 Virtusense Technologies Range of motion system, and method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
"Kinect: Wikipedia reference", 22 December 2015 (2015-12-22), Retrieved from the Internet <URL:https://en.wikipedia.org/wiki/Kinect> [retrieved on 20160112] *
CHALVATZAKI, G. G. ET AL.: "Towards an Intelligent Robotic Walker for Assisted Living using Multimodal Sensorial Data", PROCEEDINGS OF THE 4TH INTERNATIONAL CONFERENCE ON WIRELESS MOBILE COMMUNICATION & HEALTHCARE, MOBIHEALTH 2014, 3 November 2014 (2014-11-03) *
HU , R. Z-L. ET AL.: "3D Pose Tracking of Walker Users' Lower Limb with a Structured- Light Camera on a Moving Platform", COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, 2011 IEEE COMPUTER SOCIETY CONFERENCE, 20 June 2011 (2011-06-20), pages 29 - 36, XP031926502, DOI: doi:10.1109/CVPRW.2011.5981732 *

Similar Documents

Publication Publication Date Title
Shany et al. Sensors-based wearable systems for monitoring of human movement and falls
Gabel et al. Full body gait analysis with Kinect
Stone et al. Unobtrusive, continuous, in-home gait measurement using the Microsoft Kinect
Luštrek et al. Fall detection and activity recognition with machine learning
US20120229634A1 (en) Method and system for monitoring the activity of a subject within spatial temporal and/or behavioral parameters
US7502498B2 (en) Patient monitoring apparatus
US20080200774A1 (en) Wearable Mini-size Intelligent Healthcare System
US8206325B1 (en) Ambulatory system for measuring and monitoring physical activity and risk of falling and for automatic fall detection
US20080281550A1 (en) Systems and methods for characterizing balance function
US7420472B2 (en) Patient monitoring apparatus
US20010029319A1 (en) System and method of monitoring and modifying human activity-based behavior
Kwolek et al. Human fall detection on embedded platform using depth maps and wireless accelerometer
US20100324437A1 (en) Device and method for assessing physiological parameters
US20090062696A1 (en) Abnormal motion detector and monitor
US5919149A (en) Method and apparatus for angular position and velocity based determination of body sway for the diagnosis and rehabilitation of balance and gait disorders
Wang et al. An enhanced fall detection system for elderly person monitoring using consumer home networks
US20150198454A1 (en) Smart necklace with stereo vision and onboard processing
US20100079291A1 (en) Personalized Activity Monitor and Weight Management System
US20110246123A1 (en) Personal status monitoring
WO2010108287A1 (en) A wearable intelligent healthcare system and method
Yang et al. Real-time gait cycle parameter recognition using a wearable accelerometry system
US20080186189A1 (en) System and method for predicting fall risk for a resident
US20120172681A1 (en) Subject monitor
US20130128051A1 (en) Automatic detection by a wearable camera
US20120025989A1 (en) Method and system for detecting a fallen person using a range imaging device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15863526

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15863526

Country of ref document: EP

Kind code of ref document: A1