GB2554909A

GB2554909A - Dexterity assessment

Info

Publication number: GB2554909A
Application number: GB1617389.0A
Authority: GB
Inventors: Rees Jonathan; Hargrove Caroline; Surmacz Karl; Kirby Georgina; Reynolds William; Gallieri Marco
Original assignee: McLaren Applied Technologies Ltd
Current assignee: McLaren Applied Ltd
Priority date: 2016-10-13
Filing date: 2016-10-13
Publication date: 2018-04-18
Anticipated expiration: 2036-10-13
Also published as: GB201617389D0; GB2554909B; WO2018069725A1

Abstract

An assessment of e.g. surgeons dexterity uses linear 4 and/or angular 5 accelerometers, possibly on an arm band 1, at a subjects arm location e.g. the elbow or above the scrub-line. Acceleration signal analysis at processor 7, 11 identifies discrete movements, for each of which a kinetic energy is estimated via a model stored in memory 6, 12. A measure of dexterity is formed in dependence on the kinetic energies. The dexterity measure may be a count of movement energies above, below or between predetermined thresholds, possibly decided based upon e.g. a surgical task to be performed. The model stores the mass and inertia about a rotation axis of e.g. the forearm, from which the energies are estimated, the accelerometer location being known or input. Discrete movement events may be identified by vector summation of acceleration data, producing velocity data analysed to identify the starts and ends of movements.

Description

(71) Applicant(s):

McLaren Applied Technologies Limited

McLaren Technology Centre, Chertsey Road, Horsell,

Woking, Surrey, GU21 4YH, United Kingdom (72) Inventor(s):

Jonathan Rees Caroline Hargrove Karl Surmacz Georgina Kirby William Reynolds Marco Gallieri (74) Agent and/or Address for Service:

Slingsby Partners LLP

Kingsway, LONDON, WC2B 6AN, United Kingdom (51) INT CL:

A61B 5/11 (2006.01) A61B 5/00 (2006.01) (56) Documents Cited:

WO 2014/197443 A1

The Journal of Bone and Joint Surgery, Inc., Vol. 97A, No. 13, July 2015, Kirby et al., Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors, pages 1119-1127 KR 20160052882 (58) Field of Search:

INT CLA61B, G09B

Other: WPI, EPODOC, TXTA, XPIEE, XPI3E, MEDLINE (54) Title of the Invention: Dexterity assessment

Abstract Title: Measure of dexterity based on kinetic energies (57) An assessment of e.g. surgeons’ dexterity uses linear 4 and/or angular 5 accelerometers, possibly on an arm band 1, at a subject’s arm location e.g. the elbow or above the scrub-line. Acceleration signal analysis at processor 7, 11 identifies discrete movements, for each of which a kinetic energy is estimated via a model stored in memory 6, 12.

A measure of dexterity is formed in dependence on the kinetic energies. The dexterity measure may be a count of movement energies above, below or between predetermined thresholds, possibly decided based upon e.g. a surgical task to be performed. The model stores the mass and inertia about a rotation axis of e.g. the forearm, from which the energies are estimated, the accelerometer location being known or input. Discrete movement events may be identified by vector summation of acceleration data, producing velocity data analysed to identify the starts and ends of movements.

At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy.

/2

11 17

2/2

11 17 φ

ο ο

ω φ

ο £=

DEXTERITY ASSESSMENT

This invention relates to monitoring the performance of people undertaking manual tasks.

Many occupations call for high levels of dexterity in order to perform precise tasks reliably. One example of such an occupation is surgery. At present, surgeons are trained on simulated body parts and cadavers. A trainee surgeon’s level of skill is normally assessed by the extent to which they can carry out the task without error. The trainee’s level of dexterity in handling and positioning surgical instruments is normally either not assessed, or assessed in an essentially subjective way. It would be desirable to have reliable and representative metrics of surgeons’ levels of dexterity. Data of that nature could be used to guide training programs and to assess surgeons’ suitability for particularly delicate procedures such as paediatric surgery or facial reconstruction.

Various studies have attempted to derive a reliable methodology for assessing surgeons’ skill from performance data. For example:

- “Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors” (Kirby et al., J Bone Joint Surg Am, 2015 Jul 01; 97 (13): 1119 -1127) discloses the use of elbow-worn devices which incorporate accelerometers and gyroscopes. Using data from these sensors, participant surgeons were classified into performance levels. The assessment of a participant was based on metrics such as time taken, smoothness of movement and intensity of movement in certain frequency bands.

- “Synchronized video and motion analysis for the assessment of procedures in the operating theatre” (Dosis et al., Arch Surg. 2005 Mar; 140(3):293-9) discloses a system for monitoring surgeon performance. Sensors are worn on a surgeon’s hand as they carry out training procedures. From these sensors, features such as number of movements and path length are derived. These features are then correlated with known expertise level.

- “Using the Waseda bioinstrumentation system WB-1R to analyze surgeon's performance during laparoscopy - Towards the development of a global performance index” (Zecca et al., IEEE International Conference on Intelligent Robots and Systems, 2007, pp 1272-1277) discloses a system in which sensors attached to the arms, hands and head of surgeons are used to monitor respiration, pulse and perspiration, as well as the movement of the body parts in order to monitor surgical performance.

There is a need for an improved approach to measuring surgical performance.

According to the present invention there is provided a method for assessing human dexterity, the method comprising: storing an arm model indicative of mass and inertia of at least a forearm; attaching a sensor device comprising an acceleration sensor at a location on a subject’s arm, the acceleration sensor being arranged to generate a series of acceleration signals indicative of directed accelerations applied to the sensor; analysing the acceleration signals to identify discrete movement events; estimating a kinetic energy associated with each movement event in dependence on the model, the said location and the acceleration signals corresponding to the respective movement event; and forming a measure of dexterity in dependence on the estimated kinetic energies.

The step of estimating the kinetic energy for a movement event may comprise estimating the kinetic energy implied by imposing the accelerations sensed during the event at a location on the arm model corresponding to the location where the location sensor is attached to the subject’s arm. The step of forming a measure of dexterity may comprise counting the number of movement events associated with an estimated kinetic energy greater than a first predetermined threshold. The step of forming a measure of dexterity may comprise counting the number of movement events associated with an estimated kinetic energy lower than a or the first predetermined threshold. The step of forming a measure of dexterity may comprise counting the number of movement events associated with an estimated kinetic energy lower than a or the first predetermined threshold and above a second predetermined threshold.

The step of forming a measure of dexterity may comprise comparing a counted number of movement events with a third predetermined threshold. The method may comprise: receiving an indication of a task to be performed by the subject; and selecting the third predetermined threshold in dependence on that task.

The arm model may be indicative of mass and inertia of a hand linked to the forearm. The arm model may be indicative of mass and inertia of an upper arm linked to the forearm. The sensor device may comprise an orientation sensor arranged to generate a series of orientation signals indicative of the orientation of the sensor, and the step of forming a measure of dexterity may comprise forming that measure in dependence on the orientation signals. The step of identifying discrete movement events may comprise estimating the velocity of the sensor device and identifying movement events as extending between successive periods when the magnitude of the velocity of the sensor device is below a predetermined threshold.

The sensor device may be an arm band. The location may be the subject’s elbow. The method may comprise the subject performing a reference task whilst the sensor device is attached to their arm. The reference task may be a simulated surgical task. The method may comprise the subject performing a surgical task whilst the sensor device is attached to their arm. The sensor may be attached to the subject’s arm above the scrub line.

The present invention will now be described by way of example with reference to the accompanying drawings. In the drawings:

Figure 1 is a schematic diagram of a surgical assessment system,

Figure 2 illustrates data flow in the system of figure 1.

Figure 1 shows a surgical assessment system comprising arm bands 1, 2, a processing unit 9 and outputs 13, 14.

Each arm band comprises a strap 15 and a closure 3 which can be used to close the strap into a loop so it can be attached around a subject’s arm. The strap may be made of fabric, polymer film or any other suitable material. At least part of the strap may be elastic to help secure it around a subject’s arm. The closure may be a snap fastener, a hook and loop fastener or any other suitable fastener. The closure may be adjustable so that the loop can be tightened around the arm of a specific subject.

The internal components of the arm bands are illustrated for arm band 1. Arm band 2 is analogous. Arm band 1 comprises a linear acceleration sensor 4, an angular acceleration sensor 5, a processor 7, a memory 6 and a wireless interface 8.

The linear acceleration sensor 4 is coupled firmly to the strap 15. For example, it may be embedded in the strap or attached to the strap by adhesive. The linear acceleration sensor is capable of sensing instantaneous accelerations of the strap. The linear acceleration sensor is preferably a multi-axis accelerometer, most preferably a threeaxis accelerometer. Such a three-axis accelerometer is preferably capable of sensing linear accelerations on three mutually orthogonal axes. The linear acceleration sensor provides an output to processor 7 which is indicative of accelerations sensed by it.

The angular acceleration sensor 5 is coupled firmly to the strap 15. For example, it may be embedded in the strap or attached to it by adhesive. The angular acceleration sensor is capable of sensing angular (radial) accelerations of the strap. The angular acceleration sensor may be a gyroscopic sensor. The angular acceleration sensor provides an output to processor 7 which is indicative of the angular (radial) accelerations sensed by it.

The processor 7 operates to execute instructions stored in memory 6. The memory stores the instructions in non-transient form. The processor receives data from the sensors 4, 5. It may perform some operations on the received data, for example to smooth or compress the data. It then passes the data to the wireless interface 8 for transmission to the processing unit 9. The wireless interface is a transmitter which may use any suitable protocol: for example, a protocol operating in the ISM band, such as IEEE 802.11, Bluetooth or ANT, or a protocol that uses another frequency band. The processor may also assist in other operations of the armband, for example pairing the wireless interface 8 to interface 10, or calibrating the sensors.

The processing unit 9 is a computer configured to analyse data received from the armbands. The processing unit comprises a wireless interface 10, a processor 11 and a memory 12. The processor is configured to execute instructions stored in the memory 12. The memory 12 stores the instructions in a non-transient form. The components of the processing unit may be in a single place or they may be distributed. On receiving data from a armband the processor processes that data in the manner to be described below. The processed information may be aggregated over time. Once an outcome has been generated the processor sends data to an output, for example a display device 13 which can display information generated by the processor, or to the internet 14 for consumption by other devices.

It has been identified that in order to extract meaningful data from arm bands of the sort shown in figure 1 it is crucial to process the data to extract significant metrics. The manner in which these are calculated may depend on the placement of the arm bands on a subject.

In use, one arm band is secured on each of the subject’s arms. It has been found that high-quality data can be obtained when each arm band is placed further up the respective arm than the mid-point of the forearm. It has been found that positioning the sensors at the subject’s elbows provides good results. Where the sensors are provided on armbands this may be done by the anterior portions of the bands overlapping the cubital fossae, or by each of them being within 4cm of the respective cubital fossa. The system is preferably able to accommodate the arm bands being positioned above the scrub line since then the system can work in the same way when the subject is performing actual surgery as when the subject is performing simulated surgery.

Figure 2 illustrates how the processor 11 extracts metrics in the case of acceleration data. Tri-axial acceleration data is passed to the processor from the accelerometer 4 of an arm band worn on the left arm of a subject and from an accelerometer 16 of an arm band worn on the right arm of the subject. A time series of acceleration data from each arm band is subjected to a feature derivation algorithm 17. The acceleration data comprises (i) data representative of tri-axial linear accelerations as sensed by the linear accelerometer 4 and (ii) tri-axial radial accelerations as sensed by the radial accelerometer 5. This extracts data on metrics such as the number of movements, the number of minor movements, the overall smoothness of movement and the proportion of time during the procedure that the subject’s arms were substantially stationery. This data is passed to a performance calculation algorithm 18. The performance calculation algorithm makes an assessment of the subject’s dexterity by comparing the metric data to predetermined measures that are representative of levels of performance, to generate an output representing the subject’s dexterity. Such measures may be continuous: i.e. they may map any possible metric data to a performance indicator such as a score out of 10. The measures may incorporate multiple thresholds. The raw metric data and the dexterity data are passed to the outputs 13, 14 for presentation to a subject (block 19). The orientation data is processed in an analogous way, except that different metrics are applied. For some metrics both orientation and acceleration may be used as inputs.

Some examples of metrics that can be applied in block 17 are as follows:

1. Number of major movements. It has been found that in order to provide a highquality assessment of dexterity, especially for surgical procedures, it is valuable to distinguish between major movements and minor movements. Broadly put, a major movement is a movement of one of the subject’s arms that involves a relatively high level of kinetic energy. The processor 11 can estimate the kinetic energy of a movement in the following way. Acceleration data for an arm is collected over time. This provides a time series of acceleration data points. Each data point is the acceleration vector provided by the accelerometer for that arm, as the vector sum of the accelerations indicated for each axis. The accelerations are vector summed over time to provide a time series of velocity data points. The velocity data may be rebased from time to time to account for any drift in the acceleration sum. The processor 11 analyses the velocity data to identify the starts and ends of periods that will be treated as movements. In one approach the boundary between successive movements may be taken as a period when the velocity falls below a predefined threshold, for example 0.2ms’¹. In another approach the instantaneous kinetic energy of the arm may be modelled as described below, and movements may be identified by peaks in kinetic energy and may be considered to be separated at times of minimum kinetic energy. For each movement, the processor analyses the acceleration data to estimate the kinetic energy expended in moving the subject’s arm during the movement. The kinetic energy is estimated based on an anatomical model, in the following way:

(i) The locations of the armbands on the subject’s arms are taken to be known. The locations at which the arm bands are taken to be may be fixed, or may be input by a subject into the processing unit 9.

(ii) The memory 12 stores an anatomical model of a typical subject’s arm. That model defines the weight and dimensions of one or more parts of the typical arm (e.g. hand, forearm, upper arm) so that the mass of each part and its inertia about a particular rotation axis can be determined. Each item of measured acceleration data for each arm is used to estimate linear accelerations and rotational accelerations of the subject’s forearm (optionally including the hand) and optionally of the subject’s upper arm corresponding to those measured accelerations. Based on the masses and inertias of the upper and lower arm stored in the model, the processor estimates the energy involved in the estimated linear and rotational accelerations.

(iii) The estimated energies for are summed over the period of an arm movement. This provides an aggregate energy value for the movement.

A threshold energy level is defined. The threshold may for example be in the range from 2 to 8J, for example 5J. If the peak energy for a movement exceeds that threshold, or alternatively if the aggregate energy for a movement exceeds that threshold then it is considered to be a major movement. The number of major movements of each arm during the course of a procedure is counted by the processor

11. At the end of a procedure this gives two values for major movements: a total for the left arm and a total for the right arm.

2. Number of minor movements. These are calculated as for major movements except that minor movements are ones whose aggregate energy is below the threshold energy level. An energy floor may be defined (e.g. 0.1 J), and movements whose aggregate energies do not exceed that floor may be ignored and not counted as minor movements.

3. Proportion of time stationary. The total time when an arm is stationary may be taken as the total time between movements, for that arm. Optionally, the duration of movements of that arm whose aggregate energy is below the floor may be added to that time. The proportion of time stationary can be derived by dividing the total time stationary during a period under analysis (e.g. the time taken to perform a procedure) by the length of that period.

4. Smoothness of movement. To derive a measure of smoothness of movement the absolute differences between successive accelerations are calculated. This gives a series of jerk values. Then the jerk values are summed over the duration of a movement or over a whole procedure to give an estimate of the smoothness of that movement/procedure. If the intervals between successive accelerations are not constant then the jerk values may be inversely weighted according to the duration over which they were calculated. A smaller aggregate jerk value is indicative of increased smoothness. A smoothness value may be obtained as the inverse of the aggregate jerk value.

In block 18 the metrics derived over the course of a procedure can be compared with one or more thresholds to provide an assessment of performance. The performance assessment is dependent on measures that depend on the metrics determined in block 17. Some performance measures may be independent of the procedure being carried out. Examples of these include the following, each of which may be associated with an assessment of higher competence:

- a lower proportion of time when both arms are simultaneously stationary;

- a lower proportion of time when either arm is stationary;

- the number of major movements being a lower proportion of the total number of major and minor movements;

- an increased smoothness.

Some performance measures may be dependent on the procedure being carried out. For example, the procedure may be a known test procedure on a surgical simulator. Thresholds may be defined for that procedure to indicate levels of performance, or subjects may be compared against each other. Examples of performance measures that may be specific to a test procedure are the following, each of which may be associated with an assessment of higher competence:

- a lower total number of movements;

- a lower number of major movements;

- a lower number of minor movements;

- a lower time spent to perform the procedure.

Each performance measure may be compared against one or more predetermined thresholds to give an assessment of competence. The thresholds may be procedurespecific or not. Examples may be as follows:

Proportion of total movements that are major Competence assessment
Greater than 30%	Poor
30% to 10%	Average
Less than 10%	Good

Number of major movements made in performing a specific standard procedure	Competence assessment
Greater than 40	Poor
40 to 20	Average
Fewer than 20	Good

An assessment of competence may be made based on multiple performance measures and/or metrics. This may be done by calculating a weighted combination of multiple performance measures and/or metrics and comparing the combination to one or more predetermined measures and/or thresholds to give an assessment of competence. The measures and/or thresholds may be procedure-specific or not. One example may be as follows, where the value X is calculated as the left hand smoothness value for a procedure (in rrr¹s³) plus the right hand smoothness value for the procedure (in rrr¹s³) plus 0.5 times the inverse of the total time for the procedure (in s).

X	Competence assessment
Lower than threshold A	Poor
A to B	Average
Higher than B	Good

The processor can output any of the metrics determined in block 17 and/or any of the performance measures and competence assessments made in block 18 to the outputs 13, 14 so they can be presented to a subject or to people assessing the subject.

The subject may signal the processing unit 9 to indicate that he is starting a procedure and optionally what that procedure is. The subject may signal the processing unit 9 when the procedure is complete. The processing unit may be configured to take the duration of the procedure as the interval between the starting and completion. The processing unit may be configured to select one or more thresholds to use for assessing the competence of the subject from a set of pre-stored thresholds in dependence on the procedure being performed.

Data may be output from the system to a user in a number of ways. First, the system may generate an overall rating for the user based on the data gathered whilst the user was performing a task. That overall rating may be a score selected from a finite set of potential ratings (e.g. good/bad, or an integer from 1 to 10) or it may be a value from a substantially continuous scale. The overall rating may be dependent on multiple factors extracted from the data collected whilst a user is operating the system. Alternatively, or in addition, the system may provide data to a user on individual factors extracted from the data (e.g. average smoothness or number of major movements). Knowledge of these individual factors may help a user to improve their performance in specific areas.

In the examples discussed above the system is used for assessing surgical performance. The system could be used to assess other tasks requiring dexterity, for example fine-scale manufacture or assembly (e.g. watch assembly or jewellery manufacture).

For some tasks it may be sufficient to gather metrics in respect of a single arm, using one armband only. The task might be executed with a single hand, or it might be that the subject’s overall dexterity can be inferred from data relating to a single hand.

Instead of the sensors being provided in an arm band that can be looped around the arm they could be attached to a subject in another way, for instance clipped onto clothing.

The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.

Claims

1. A method for assessing human dexterity, the method comprising:

storing an arm model indicative of mass and inertia of at least a forearm; attaching a sensor device comprising an acceleration sensor at a location on a subject’s arm, the acceleration sensor being arranged to generate a series of acceleration signals indicative of directed accelerations applied to the sensor;

analysing the acceleration signals to identify discrete movement events; estimating a kinetic energy associated with each movement event in dependence on the model, the said location and the acceleration signals corresponding to the respective movement event; and forming a measure of dexterity in dependence on the estimated kinetic energies.

2. A method as claimed in claim 1, wherein the step of estimating the kinetic energy for a movement event comprises estimating the kinetic energy implied by imposing the accelerations sensed during the event at a location on the arm model corresponding to the location where the location sensor is attached to the subject’s arm.

3. A method as claimed in claim 1 or 2, wherein the step of forming a measure of dexterity comprises counting the number of movement events associated with an estimated kinetic energy greater than a first predetermined threshold.

4. A method as claimed in any preceding claim, wherein the step of forming a measure of dexterity comprises counting the number of movement events associated with an estimated kinetic energy lower than a or the first predetermined threshold.

5. A method as claimed in any preceding claim, wherein the step of forming a measure of dexterity comprises counting the number of movement events associated with an estimated kinetic energy lower than a or the first predetermined threshold and above a second predetermined threshold.

6. A method as claimed in any of claims 3 to 5, wherein the step of forming a measure of dexterity comprises comparing a counted number of movement events with a third predetermined threshold.

7. A method as claimed in claim 6, comprising:

receiving an indication of a task to be performed by the subject; and selecting the third predetermined threshold in dependence on that task.

8. A method as claimed in any preceding claim, wherein the arm model is indicative of mass and inertia of a hand linked to the forearm.

9. A method as claimed in any preceding claim, wherein the arm model is indicative of mass and inertia of an upper arm linked to the forearm.

10. A method as claimed in any preceding claim, wherein the sensor device comprises an orientation sensor arranged to generate a series of orientation signals indicative of the orientation of the sensor, and the step of forming a measure of dexterity comprises forming that measure in dependence on the orientation signals.

11. A method as claimed in any preceding claim, wherein the step of identifying discrete movement events comprises estimating the velocity of the sensor device and identifying movement events as extending between successive periods when the magnitude of the velocity of the sensor device is below a predetermined threshold.

12. A method as claimed in any preceding claim, wherein the sensor device is an arm band.

13. A method as claimed in any preceding claim, wherein the location is the subject’s elbow.

14. A method as claimed in any preceding claim, comprising the subject performing a reference task whilst the sensor device is attached to their arm.

15. A method as claimed in claim 14, wherein the reference task is a simulated surgical task.

16. A method as claimed in any preceding claim, comprising the subject performing a surgical task whilst the sensor device is attached to their arm.

17. A method as claimed in claim 15 or 16, wherein the sensor is attached to the subject’s arm above the scrub line.

Intellectual

Property

Office

Application No: GB1617389.0