WO2019104388A1

WO2019104388A1 - Methods and apparatus for measurement of positions and motion states of exercise equipment in three dimensions

Info

Publication number: WO2019104388A1
Application number: PCT/AU2018/051276
Authority: WO
Inventors: Dane DOBRE; Nenad DOBRE
Original assignee: Dobre Holdings Pty Ltd
Priority date: 2017-11-29
Filing date: 2018-11-29
Publication date: 2019-06-06
Also published as: AU2018377859B2; AU2018377859A1

Abstract

A method for provision to a user of position and orientation information of a moving object comprising: in a first step, under control of software executing on a computing means, one or both of: a first stream of information comprising at least one of position and orientation of a moving object derived from outputs of distance measuring sensors moving with the object; and a second stream of information comprising at least one of position and orientation of the moving object derived from outputs of acceleration sensors and rotation rate sensors moving with the object; and in a second step, processing the first and/or second streams of information to arrive at a third stream of information on position and orientation of the object and providing the third stream of information to a user. Also disclosed is a sensing device for sensing position and orientation of a moving object operating in accordance with the method.

Description

METHODS AND APPARATUS FOR MEASUREMENT OF POSITIONS AND MOTION STATES OF EXERCISE EQUIPMENT IN THREE DIMENSIONS

Field

This invention relates to measurement of the state of motion and position, in three dimensional space, of components and body parts of persons

Background

The commercial availability of miniaturised accelerometers, rotation rate sensors, magnetometers, global navigation system (GNSS) equipment and the like has increased the number of practicable options for measurement of states of motion and positions of components and body parts of persons.

Such measurements can be important in medicine sports and fitness training and many other applications.

A difficulty is that estimations of positions, orientations and the paths followed, although able to be made from measurements over time of motion states are subject to drift and resulting inaccuracy due to noise, drift and error generated in sensors and their support systems. In many situations, this can make estimated positions orientations and paths unacceptably unreliable. Motion parameters such as velocities, rotation rates and accelerations are themselves subject to error also for the same reasons.

Patent Specifications describing developments in this area include:

• US2017/0128765 (Garretson) - [0049] discloses an 'advanced sensor fusion' that is generated between a device mounted with IMU and UWB (ultra-wide band) antenna that allows high accuracy tracking without a direct line of sight sensor such as a camera or a laser range finder.

• US2016/0346617 (Gymtrack) - This US application is based

on PCT/CA2015/050071.

. US2015/0374290 (Teesside Uni).

None of these specifications disclose a high accuracy, highly responsive system particularly but not exclusively suited to the exacting, high distance accuracy

requirements of monitoring exercise with a barbell.

Preferred embodiments disclosed herein address the problem of improving at least position, orientation and path measurements, in a range of applications to a useful degree and to provide useful additional choices for persons requiring to measure motion states and positions. Notes

The term“comprising” (and grammatical variations thereof) is used in this specification in the inclusive sense of“having” or“including”, and not in the exclusive sense of “consisting only of.

The above discussion of the prior art in the Background of. the invention, is not an admission that any information discussed therein is citable prior art or part of the common general knowledge of persons skilled in the art in any country.

Disclosure of the Invention

In a first aspect, the invention provides a sensing device for sensing position and orientation of a moving object comprising:

an array of one or more distance measuring sensors;

acceleration sensors adapted for sensing components of acceleration along each of a first three axes of sensing device;

rotation rate sensors adapted for sensing rotation rate about each of a second three axes of the sensing device; and

communication means adapted for transmitting of data and/or information derived therefrom from the sensing device; and

computing means adapted for receipt and handling of data from the distance measuring, acceleration and rotation rate sensors and for managing transmission of the data or information derived therefrom from the sensing device;

whereby to enable derivation of position and orientation information on the object from either or both of the distance measuring sensors and the acceleration and rotation rate sensors.

In a second aspect, the invention provides a system for acquisition storage and processing of information on position attitude and motion of objects comprising:

a sensing device according to any one of the embodiments disclosed herein, and wherein the computing means or a separate computing means executes software to derive from outputs of sensors of the sensing device information comprising at least one of position, orientation, acceleration and velocity of the moving object.

In a third aspect, the invention provides a method for provision to a user of position and orientation information of a moving object comprising: in a first step, under control of software executing on a computing means, one or both of:

a first stream of information comprising at least one of position and orientation of a moving object derived from outputs of distance measuring sensors moving with the object; and

a second stream of information comprising at least one of position and orientation of the moving object derived from outputs of acceleration sensors and rotation rate sensors moving with the object; and in a second step, processing the first and/or second streams of information to arrive at a third stream of information on position and orientation of the object and providing the third stream of information to a user. Preferably the first stream of information is derived from a dedicated distance sensor.

Preferably the dedicated distance sensor comprises an array of distance sensor units.

Preferably the distance sensor units are mounted in substantially the same plane.

Preferably the distance sensor units are arranged radially about an axis.

Preferably the dedicated distance sensor comprises a laser based distance measuring device.

Preferably laser based distance measuring device is a LIDAR device.

Preferably the distance sensor units comprise discrete LIDAR devices.

Preferably the second stream of information is derived from an I MU.

In a fourth aspect, the invention provides a method of assessing the attitude of an exercise component when the component is operated by a user as the user performs a repetitive physical activity with respect to a surface; the method implemented on a computing device and comprising:

receiving attitude data comprising sensed acceleration data from at least one motion sensor mounted to the component that comprises an accelerometer that is configured to sense and generate acceleration data in at least three axes, the sensed acceleration data representing a first component of attitude of the component; and receiving distance data comprising sensed distance data from at least one dedicated distance sensor mounted to the component and wherein the sensed distance data represents a second component of attitude of the component;

processing the first component of attitude and the second component of attitude in order to generate output feedback data comprising attitude of the component with respect to the surface (from which the user obtains leverage). Preferably the second component of attitude is derived from a dedicated distance sensor.

Preferably the distance sensor units are arranged radially about an axis.

Preferably the laser based distance measuring device is a LIDAR device.

Preferably the distance sensor units comprise discrete LIDAR devices.

Preferably the first component of attitude is derived from an IMU.

In yet a further broad form of the invention there is provided a display output device in communication with the sensing device as described above; the display output device outputting data derived from the sensing device.

In yet a further broad form of the invention there is provided an exercise display output device in communication with the sensing device as described above; the display output device outputting data derived from the sensing device.

In yet a further broad form of the invention there is provided a barbell exercise display output device in communication with the sensing device as described above; the display output device outputting data derived from the sensing device.

In yet a further broad form of the invention there is provided a barbell exercise display output device in communication with the sensing device as described above; the display output device outputting data derived from the sensing device in response to movement of a barbell; the sensing device located on an end of the barbell.

Additional and preferred features of the sensing device are set out in the attached description, figures and claims.

Brief Description of the Views of the Figures

Figure 1 is a schematic (block) diagram of a system according to the invention;

Figure 2 is a block diagram showing a process for arriving at estimates of position velocity acceleration and orientation estimates followed by an inertial measurement unit; Figure 3 is a perspective view of a component of a sensing device that is a first embodiment of the invention;

Figure 4 is an exploded view of a sensing device in which the component of Figure 3 is comprised.

Figure 5 is an exploded view of a further sensing device according to the invention;

Figure 6 is a perspective view of the sensing device of Figure X and a hemispherical surface showing directions of optical axes of the sensing device;

Figure 7 is a perspective view of a further sensing device according to the invention;

Figure 8 is a perspective view of a barbell fitted with sensing devices according to the invention and a base station for the sensing devices;

Figure 9 is a view of the barbell of Figure 8, as seen looking in the direction of arrow“A” of Figure 8;

Figure 10 is a longitudinal cross-sectional view of a rowing craft showing a person rowing the craft;

Figure 11 is a side view of an exercise machine being operated by a person;

Figure 12 is a view of legs and feet of a person walking on a treadmill looking from behind and in the direction of walking;

Figure 13 is a side view of the legs, feet and the treadmill of Figure 12;

Figure 14 is a perspective view of a portion of a road vehicle showing a device according to an embodiment of the invention secured thereto;

Figure 15 is a partial cross-section of a road vehicle showing a wheel with a cylindrical target secured thereto and a device embodying the invention secured thereto.

In this specification, including in the appended claims, the word“comprise”, and words derivatives thereof such as“comprising”,“comprises”,“comprised in” and the like, when used in relation to integers or steps, are to be interpreted as indicating the presence of those integers or steps but not as precluding the possible presence of other integers or steps.

Detailed Description of Embodiments

Figure 1 shows a system 10 comprising at least one sensing device 12, a base station 14 and a communications link 16 between the sensing device(s) 12 and the base station 14. System 10 is adapted to sense states of motion, orientation and position of sensing device(s) 12 and objects or parts of persons or animals to which sensing device(s) 12 are secured, using sensors comprised in sensing devices 12. More specifically, at least one sensing device 12 of system 10 comprises both an array of distance sensors (not shown in Figure 1) for sensing of distances from sensing device 12 to neighbouring surfaces (not shown), and an inertial measurement unit (IMU) (also not shown in Figure 1) for sensing of its motion. Data from the distance sensors 26,28 and IMU 24, and/or information derived from that data, is transferred to the base station 14 for display, recording, and analysis. In some embodiments, data also be displayed on device 12 itself. The IMU has at least 3-axis rate gyro sensors and 3-axis accelerometers, but as set out below may comprise one or more additional sensors.

Figure 2 shows in block diagram form a process 200 by which a combination of 3-axis rate gyro sensors and 3-axis accelerometers, as in an IMU, can be used to determine acceleration and velocity components, position and orientation in a global frame of reference (for example fixed to the earth) of a moving object (not shown) to which the IMU is secured, given an initial position and orientation of the object. Process 200 is known in the art of“strapdown” inertial navigation systems and other application of IMU devices.

In the global frame of reference, orientation of the object over time is estimated by a process 202 including integration based on rate gyro sensor outputs and knowledge of the initial orientation. Velocity is estimated by a process including integration (at 204) of acceleration, and position is determined (at 206) by a process including integration of velocity. The acceleration is determined in the global frame from a knowledge of firstly the accelerometer outputs measured in a frame of reference (not shown) that moves with the IMU and, secondly, the orientation, which allows (at 208) projection of acceleration components into the global frame, followed (at 210) by correction for gravity. The position, orientation and velocity outputs obtained this way are subject to noise and cumulative drift due to noise, drift and error in the sensors. Details of the basic algorithm shown in Figure 2, and a discussion of noise, drift and other practical issues with the application of IMUs to the process 200, are given in Woodman, O.J., 2007,“An introduction to inertial navigation”, Technical Report UCAM-CL-TR-696, University of Cambridge Computer Laboratory (ISSN 1476-2986).

Sensing device 12 allows direct measurement of position and orientation using sensors other than those of the IMU, at least some of the time during a movement, so as to limit inaccuracy in these quantities inherent in practice in process 200.

In applications of system 10 where position and orientation can be measured directly and reliably, such measurements can be preferred over I MU-based estimates, but where they are not available at all times, so that there are gaps in acquired data, those gaps can be filled using data based on the IMU sensors. Moreover, where position and/or orientation measurements are available at least some of the time during a movement, they can in some embodiments be used to improve the IMU-based estimates of position, orientation, velocities and acceleration. Further, the IMU-based position and velocity estimates may themselves sometimes be made more reliable, even before they are adjusted using direct distance measurements as mentioned above. Orientation estimates provided (as shown at 212) to the process 208 of projection of accelerations into the global frame can in suitable applications be enhanced by use of direct distance sensor information (when available) or statistical estimates based on both the rate gyro outputs and direct distance sensor data.

Note that the process 200 does not involve absolute (compass) heading information as can be provided by multi-axis magnetometers. What is provided by process 200 is information on position, orientation and motion relative to an initial position, motion state and orientation. However, some embodiments provide magnetometers either comprised in or additional to the IMU for use in applications where this is important and/or practical (for example where there is no interference from nearby ferromagnetic materials).

Figure 3 shows a key component 18 of a first embodiment of a sensing device 20 for use in the system 10. Sensing device 20 is shown in an exploded view in Figure 4. Component 18 comprises a structure 22 that is a disc-shaped printed circuit board and contains, on the structure 22, the following main components:

a nine degree-of-freedom inertial measurement unit (IMU) 24 comprising in a single assembly a three-axis accelerometer; a three-axis rotation-rate sensor; and a three-axis magnetometer (not individually shown);

an array of distance sensors 26, 28 of the time-of-flight infra-red laser type;

a microprocessor 30; and

a communications module 32.

The array of several distance sensors in sensing device(s) 12 allows position and attitude information to be determined independently of position and attitude information derived from the IMU, and as set out below. The“array” of distance sensors could comprise one distance sensor only, but this prevents determination independently of the IMU sensors of both attitude and position, so in preferred embodiments, multiple distance sensors are provided, to increase the range of possible applications.

The accelerometer, rotation rate sensor and magnetometer of IMU 24 respectively provide outputs for components of acceleration along three mutually orthogonal axes, marked as x, y and z in Figure 3, rates of rotation about the same three mutually orthogonal axes, and components of magnetic field strength along the same three mutually orthogonal axes. The magnetic field outputs allow determination of heading (relative to magnetic north) of device 20 and the acceleration and rotation rate outputs partially describe the state of motion of the device 20.

Distance sensors 26, 28 measure distances of external surfaces or objects from the device 20 so that the position and attitude of device 20 with respect to the neighbourhood in which it is located can be determined directly, in addition to or instead of, any estimation made using the IMU sensors.

The purposes of the communications module 32 are:

(a) to transmit data from (or based on) IMU 24 and distance sensors 26, 28 to base station 14 after any necessary conditioning, buffering or other manipulation of that data; and

(b) to receive control signals from base station 14 and any required software reloads or updates.

The purpose of the microprocessor 30 is to manage transmissions from and to the device 20, and any necessary receiving, conditioning, buffering and/or manipulation of data from the sensors 26, 28 and the IMU 24.

Rotation-rate sensors are known that are based on rotating masses (traditional gyroscopes), or closed laser light paths with detectors and that detect rotation rate based on the Sagnac effect (so-called“fibre optic gyros” and“ring laser gyros”).

However, the rotation-rate sensor of IMU 24 is preferably of the vibrating type in which three masses (one for each axis) are caused to vibrate in orthogonal directions and their tendency to continue doing so as those directions rotate is exploited to develop a rate- proportional output signal. Such sensors are available commercially as microelectromechanical systems (MEMS).

Multiple-axis accelerometers are also available as MEMS devices.

Magnetometers, and MEMS accelerometers and rotation rate sensors are available commercially in single packages similar to those of ordinary single-chip electronic devices for example in the form of surface mount devices (SMDs) with signal, power and other pins adapted to be soldered to a printed circuit board.

While separate magnetometer, accelerometer and rotation rate sensor packages can be used, IMU 24 is in preferred embodiments of the type in which all three types of sensor are implemented as a single SMD package in the interests of compactness. In preferred embodiments, the three accelerometer axes are collinear with the rotation rate axes and the magnetometer axes, although this is not essential. IMUs based on MEMS

technology deliver a stream of digitized samples at a certain sampling rate, rather than continuous (analog) signals.

Distance sensors 26 are time-of-flight sensors that measure the time taken for light generated by a solid state laser to travel along an optical axis 34 of the sensor 26 to a surface or object (not shown) and to be reflected back to a detector comprised in the sensor 26. That is, each sensor 26 senses the distance along its optical axis 34 to a surface that reflects at least some of the sensor’s radiation back along the optical axis 34. Suitable distance sensors for the embodiments described herein use laser light in the infra-red region (though this is not to preclude sensors using light in other parts of the electromagnetic spectrum) and have useful range capability of approximately 2m to a maximum in the range of about 4m - 10m. Although time-of-f light laser sensors are preferred, other types of distance sensors may be used, for example time-of-flight sensors based on acoustic effects, for example ultrasonic distance sensors.

An advantage of laser-based distance sensors in an array, as in component 18, is that their use is less likely than use of ultrasonic or other acoustic distance sensors to suffer from the problem of one distance sensor picking up a reflection from another sensor.

As shown in Figure 3, distance sensors 26 are spaced circumferentially around the structure 22 with their optical axes 34 (shown as dotted lines) extending radially outward from a central axis 36 in a plane (with axes shown as x and y in Figure 3) normal to axis 36. 12 sensors 26 are shown, spaced apart circumferentially so that each is 30 degrees apart from its two neighbours, but that is not to preclude other numbers or spacings of sensors 26 being used. An additional distance sensor 28 (of the same type as sensors 26) is located so that its optical axis 38 is collinear with axis z of the x/y/z reference frame of component 18.

(Note that it is not necessary for the z axis to be collinear with one of the three IMU axes; this is a matter of convenience.)

Because sensors 26 all have their optical axes 34 lying in one plane (the x-y plane), distances from component 18 to surfaces or objects outside that plane cannot be sensed by sensors 26 alone. That is why additional distance sensor 28 is provided on component 18 and so in sensing device 20: in combination with distance sensors 26, it allows distances from sensing device 20 in three dimensions to be sensed.

As shown in Figure 4, component 18 is secured in a housing 38. Housing 38 is transparent or has transparent sections or (as shown) windows 40 aligned with the optical axes 34 of distance sensors 26. Window 42 is for the additional distance sensor 28. A battery 44 of“button” configuration is secured against contacts (not shown) on one side of component 18 to provide power to the component 18. A lid 46 is threadably, snap-fittingly or otherwise engageable with the housing 38 with an O-ring seal 48 in between lid 46 and housing 38 to provide adequate sealing (for example IP68-standard sealing) against, for example, water or sweat ingress.

The particular arrangement of sensors 26 in device 20, in which the distance sensor array includes at least one complete ring of circumferentially equispaced and radially oriented distance sensors 26, as shown in Figures 3 and 4, has been found useful, although not essential.

However, for some applications and embodiments, it is desirable to provide several additional distance sensors instead of the one additional distance sensor 28 of device 20. Therefore, some embodiments have distance sensors whose optical axes extend in a range of directions with several such sensors having their optical axes not lying in one plane. Figures 5 and 6 show a sensing device 50 that is such an alternative embodiment of sensing device 12. Device 50 is essentially the same as device 20, save that it is provided with 25 distance sensors (not shown) instead of the 13 of device 20. Housing 52 is equivalent to housing 38 of device 20, with windows 54 aligned with the optical axes 56 of the distance sensors. The construction of device 50 is otherwise similar to that of device 20, having a battery 60, lid 62 and O-ring seal 64 as in device 20, except that the distance sensors must be supported on a structure 58 that unlike structure 18 accommodates the different number and orientations of the distance sensors.

To illustrate its arrangement of distance sensors, device 50 is shown in Figure 6 at the centre of an imaginary hemispheric surface 66 with 12 circumferentially equi-spaced “meridians” 68 (of which four are visible) and an imaginary circle 70 at constant “latitude” on surface 66. Device 50 has 25 distance sensors of which 12 are arrayed with their optical axes 56a in an“equatorial” plane x/y (as in device 20). A further 12 sensors (of which four are visible) are spaced and oriented so that their optical axes 56b pass through intersections of the meridians 68 and constant-latitude circle 70, and one sensor’s optical axis 56c extends along the z axis (as in the component 18 of Figure 3).

Fewer sensors can be used if required, yet still provide both a set of distance sensors with optical axes in an x-y plane, as in devices 20 and 50, plus more than one additional sensor for sensing distances out of that plane. It is not essential that the sensors additional to those in the“equatorial” plane be arranged on“meridians” and/or a constant“latitude” circle as shown: they can be arranged arbitrarily, provided their directions are known.

Figure 7 shows a sensing device 72 that is another possible embodiment of sensing device 12. This is in the form of a“bracelet”, that can be worn around a user’s wrist 74 or ankle and that has distance sensors (not shown) whose optical axes 76 extend outwardly and from the“bracelet” surface through windows 78. In Figure 7 the optical axes 76 are in one plane, and the optical axis 80 of one additional distance sensor (not shown) is arranged to extend out of that plane. (More than one of such out-of-plane distance sensors may be provided in other embodiments.) To maintain fixed relative orientations of the distance sensors’ optical axes 76, the bracelet is in two rigid halves, hinged together at 82. Embodiments such as devices 20 and 50 can of course also be provided with a wrist strap, for wearing on a user’s wrist (not shown) like a wristwatch.

Distance sensors based on infra-red solid state lasers may in some light conditions become unreliable. Therefore, in some embodiments, additional distance measuring sensors, of different type are provided as backup: for example, ultrasonic distance measuring sensors (not shown) in addition to sensors 26.

Embodiments of sensing devices 12 including devices 20, 50 and 72 can communicate with the base station 14 wirelessly, although this is not to preclude possible provision of embodiments that communicate by wire additionally or instead. For wireless

communication, radio frequency means are preferred. For example, Wi-Fi, Bluetooth (or Bluetooth Low Energy), mobile phone technology, or any other suitable RF technology may be used. The communication link between sensing device and base station may optionally include communication via a data network, such as the Internet. Where a sensing device 12 embodiment has a wired communication link (not shown), it may also be powered via that communication link.

The base station 14 of system 10 can be a dedicated device or simply comprise a general purpose computer, a mobile telephone, a tablet computer; or any suitable device with hardware and communications software, and programmed to receive, store, analyse and display data from the sensors of the sensing device 12 and/or information derived from such data in the sensing device. Preferred embodiments enable

simultaneous handling of output from several sensing devices

Different applications of sensing devices 12 such as devices 20, 50 or 72 (for example those set out below) can require different functionality from the software running on the sensing device microprocessor such as microprocessor 30. For example, it may be that one application requires nothing more than direct distance measurements from one distance sensor, requiring the microprocessor only to manage receipt, conditioning, buffering and transmission of outputs from that distance sensor to base station 14.

Another application may require use of all IMU and distance sensors (and any others where provided) of a sensing device such as 20 or 50, with or without onboard computation of derived quantities in real time. Thus embodiments, allow a user to choose and load any one of a“library” of software packages for different applications, to be uploaded from base station 14 to a sensing device 12 (such as 20 or 50) as required. Thus, less onboard program storage is needed and battery drain reduced.

Note that for many applications sophisticated real time computations may not be needed, so that sensor output data for an event can be simply transmitted to the base station 14, stored, and manipulated as required during a post-event analysis process.

Embodiments (not shown) of sensing device 12 may include a display (e.g. using any of LED, OLED or LCD technology) for output of information, numerical and/or graphical. However, for many applications, it is the base station 14 that is used for display as well as storage and analysis of acquired data.

For use in applications where warning functions may be required, as in some

exercise/training situations, a sound transducer (eg piezoelectric) may be provided in a sensing device 12 for actuation when software on the device 12, or on the base station 14 detects a hazardous situation.

Ways in which system 10 can operate are best understood by reference to examples as set out below.

Example: weight training

This example is described in detail, because the basic principles set out are applicable to following examples as well. Sensing device 20 will be referred to in this example, but it is to be understood that sensing device 50 or a suitable other embodiment may be used.

Suppose that for training in safe and competitive lifting of a barbell 84 by a person lying on a bench, it is desired to monitor the height and tilt of barbell 86 relative to a floor surface 92 throughout a lift. It may also be desired to measure dynamic quantities such as acceleration and velocity. Sensing device 20 can allow this in several ways.

Figure 8 shows barbell 84 with devices 20a and 20b (of the type of device 20) secured to opposite ends of bar 88 of the barbell 84 hence a known distance apart. Devices 20a and 20b transmit data wirelessly to base device 86 (corresponding to base station 14) for display, recording and analysis.

Devices 20a and 20b are (in a preferred arrangement) secured to the bar 88 with their axes 36 collinear with an axis 90 extending along the length of the bar 88. This results in the distance sensor optical axes 34a, 34b extending radially outward from devices 20a and 20b in planes which respectively have x1 , y1 , and x2, y2 as orthogonal axes. Planes x1/y1 and x2/y2 are vertical when bar 88 is horizontal. (Laser beams from sensors 20a and 20b along the optical axes 34a and 34b are shown in dotted lines and truncated in length for clarity.)

If the barbell 84 starts from a known stationary position before a lift is initiated, it is possible for each of devices 20a, 20b to provide a stream of estimates of the position and attitude of barbell 84 continuously in a fixed global frame of reference thereafter using only its IMU 24 sensors and process 200. However, over time the estimated positions and attitudes of the barbell 84 will become progressively less accurate due to drift and accumulation of errors in the outputs of the IMU 24 sensors.

If what is required is to know the height of barbell 84 above a floor surface 92 (eg the floor of a gymnasium) and its tilt relative to surface 92, the drift problem can be avoided by use of outputs from the distance sensors of devices 20a, 20b to measure them directly. Figure 9 shows the situation in plane x1/y1. By elementary trigonometry, it is possible to determine the distance“hi” of the axis of device 20a in plane x1/y1 from surface 92 using several of the distance sensors that can“see” the surface 92. Similarly it is possible to determine a corresponding distance“h2” (not shown) from surface 92 in plane x2/y2 at the same time. Assuming the surface 92 to be horizontal, it is then possible using elementary trigonometry to derive the height above surface 92 of any point on the barbell 84 (for example a point midway along the bar 88) and, from the difference in height between devices 20a, 20b, its tilt angle relative to surface 92 (here meaning the angle from horizontal of bar 88). These values will in general be more accurate over time than those estimated from the IMU 24 sensors of devices 20a, 20b alone.

Note that the above depends on knowledge of which distance sensors’ outputs to use at any given time in arriving at the distances hi and h2. There are several ways in which this may be done. One is to search those outputs, every time an estimation of hi (or h2) is made, for a group of three or more outputs for which the value of hi (or h2) inferred from any two is consistent with the value determined from any other two, and use those outputs.

When the barbell 84 is stationary in an initial position, the appropriate distance sensor outputs can be determined by relying on the accelerometer outputs obtained to determine which way is down and therefore which of the distance sensors are the ones most nearly looking downwards to floor surface 92.

The above measurements may be enough for many practical purposes, for example where interest lies only in the lift height and tilt, at any time, of bar 88 relative to surface 92. Because positions and tilt are determined from distance sensor data, the problem of cumulative drift in position estimates derived from IMU data is avoided. Note that angular orientation of the bar 88 about its longitudinal axis 90 (roll) can be determined from distance sensor measurements also, by keeping track of the distance sensors (provided there are more than one) that at any time are sensing distance from surface 92.

Instead of using two devices 20a and 20b, it is possible to use one only. This can be of the same type as devices 20a, 20b. In this case, the tilt must be derived from IMU data or reliance must be placed on the ability of the single sensing device to sense distances that are not only in one plane. In the latter case, device 50 is better than device 20. Device 50 may be secured to one end of the bar 88 in the same way as device 20 and, again for usage above a flat surface 92, the addition of an extra ring of distance sensors (see Figure 6) enable, through elementary trigonometry, a value of the tilt angle of bar 88 to be determined independently of the IMU 24 sensors.

If it is desired to have velocity and acceleration through a lift/hold/lower process as well, these can be obtained using IMU sensor data. For simplicity of explanation it will be assumed that only one sensing device 20 is used.

For a brief, one-off event such as lifting, holding and lowering barbell 84, the problems of noise in acceleration and both noise and drift in velocity may not be significant. Then, they can be handled by providing a“set” or“reset” signal to the IMU when the barbell is stationary in a known position and about to be lifted. This can be done manually or automatically when zero acceleration and rotation are detected, so that the IMU alone, or the IMU and distance sensors, register an initial zero-velocity position and

orientation. Embodiments of sensing devices 12 (e.g.20) and base stations 14 (e.g. 86), have manual“set/reset” controls.

For monitoring position, orientation, velocity and acceleration over periods long enough for drift issues to arise, the use of distance sensor measurements can assist in maintaining estimates of these quantities that at least limit the effects of noise and drift.

Acceleration, orientation and velocity components can be estimated more reliably than from IMU sensor outputs alone by use of suitable filter algorithms, e.g. an appropriate one of the Kalman-type filters, with height (and optionally the roll of bar 88) measurements from the distance sensors being combined by the filter with IMU-derived heights.

This can be done in near-real time, with the computation done either in the base station 86, or (if sufficient onboard computing capacity is available) in the sensing devices 20a, 20b.

It is also possible to apply the filtering offline after the lift, using data transmitted to, and stored in the base station 86 during the lift. Further, because data from the whole process is then available, better estimates than are obtainable from online filtering alone can be obtained using appropriate“smoothing” algorithms of known type.

Note that if the distance sensors operate on floor surface 92 only, the filtering and/or smoothing approach described above can be used, but the results will be less reliable than if distances from one vertical surface or two orthogonal vertical surfaces could also be sensed by the distance sensors and used in the filtering/smoothing algorithms.

Sensing devices such as 20 (and particularly 50 because of its number and distribution of sensors) can do this, giving three-dimensional position information.

Although position information can be derived directly using the distance sensor data alone, it can optionally be combined with IMU data by the algorithms mentioned. This may be useful where for example distance sensor data is liable to be interrupted for some of the time.

Base device 86 may be adapted to display values of quantities received from the sensing device(s) such as 20 or 50 simply as numbers or as graphs, or as pictorial representations. In particular, geometrical parameters for barbell 84 may be entered into memory in base station 86, and used with received sensor data (and/or information derived therefrom) to display a representation of the barbell 84 in motion.

It will be appreciated that the devices and principles described above can be applied to other forms of weight training, for example using dumbbells (i.e. pairs of weights mounted on a short shaft so that the assembly can be held in one hand).

The weight training example above has been described without reference to

magnetometers, and is one where the effects of ferromagnetic materials may affect their operation. However, it is known to deal with this by a process of calibration, and where this proves practicable, it is possible to include magnetometer output as part of the IMU sensor output and use it to increase the reliability of the sensed data.

Example: Exercise machines and equipment

Unlike the example of weight training described above, in which the weights can move freely in space, many exercise activities are carried out using machines that have parts which follow defined paths, of either fixed or variable dimension. For example, many rowing machines - and rowing craft - have a movable seat for the user which move backward and forward in a straight line. The distance moved depends on the user’s physiognomy and technique. That distance, along with parameters of motion (acceleration, velocity, repetition rate) of the seat along its path, can be measured using a device such as device 20 (or 50). Figure 10 shows a rowing craft 94 with a sliding seat 96. Device 20 is shown mounted to the structure of craft 94 with one of its optical axes 38 trained towards the seat 96, so as to measure its movement directly. Alternatively (not shown) the device 20 could be mounted on the seat 96 with an optical axis (28, say) trained on a fixed part of the structure of crat 94. Such an arrangement also allows direct measurement of seat positions, and in addition measurement of seat acceleration and estimation from the acceleration of the seat velocity.

In a further possible application to rowing, devices embodying the invention may be secured to an oars 98, or (not shown) to the paddle of a kayak.

In applications to rowed water craft with multiple crew members, synchronisation of crew movements can be particularly important. To analyse problems in this area, it is possible to deploy one device embodying the invention to each one or several of the crew members to discern differences between their movements, for corrective action.

Many exercise machines involve the lifting and lowering of a weight vertically by an arrangements of links and/or pulleys. Figure 11 shows one example: a person 100 moves a bar 102 back and forth, with resistance provided by a weight 104 through a cable 106 and pulley 108 arrangement. Three possible ways of using sensing device 20 (or 50) are shown. First, it may be placed at a fixed location with an optical axis 116a trained on the weight (as at 110). Second, it may be placed on the weight 104 with its optical axis 116b trained on a fixed surface (as at 112). Third, it may be placed on the movable bar, as at 114. In the third case, surfaces 118 and 120 are suitable for direct measurement of position bar 102, being“seen” by optical axes 116c.

Applications such as this are similar to the weight training example above, and the approaches described above apply, but may be simpler to execute.

Example: Walking, running, gait assessment

It is known to carry out studies of gait of persons by causing them to walk on a treadmill that has a moving surface provided by an endless moving belt. Cameras may be used to study abnormalities, differences between one foot/leg and the other, and the like.

Figure 12 shows one way in which device 50 (preferably) can be used to provide quantitative information on the gait of a person 126 walking on a treadmill. Figure 12 is a view of the legs and feet 122 and 124 of the person 126 from directly behind looking in a direction (arrow“B” of Figure 13) parallel to the moving treadmill surface 128 and its direction of motion. To the left and right lower legs 122 and 124 devices 50a, 50b are secured by attachments 130. The orientation of devices 50a, 50b is adjusted such that some of their respective distance sensors have their optical axes 56a, 56b directed downward towards the surface 128. The precise position can be set with the subject person 126 standing on the belt surface 128 before it is set in motion.

Accordingly, IMU sensors and distance sensors of the devices 50a, 50b can acquire quantitative data that depend on the gait of the person 126, including for example height above the treadmill surface 128 measured by the distance sensors in essentially the same way set out above for the weightlifting example. The height measurements, being direct as opposed to estimations based on the IMU sensor outputs, can be used to limit inaccuracies developing over time due in the IMU sensor outputs. The treadmill surface 128 plays the same role as the surface 92 in the weightlifting example.

Optical axes 56a, 56b of other distance sensors of the devices 50a, 50b will be directed in generally forward and rearward directions parallel to arrow“B”. Therefore if, as shown in Figure 13, a flat surface 130 is placed as shown relative to the treadmill surface 128 ahead of the person 126, as can easily be done, it is also possible to measure distances from that surface 130 to the devices 50a, 50b directly, as opposed to estimating from their IMU sensor outputs. Such measurements can be used in addition to the height measurements to further enhance reliability of the data as described in the weight training example above. Surface 130 is helpful in the way (mentioned in that example) that a single vertical surface can provide more data for the filtering and/or smoothing algorithms.

It is also possible to arrive at quantitative information on differences in the movement of the two legs 122 and 124. For any measured quantity, it is possible to compute a difference between corresponding and at least substantially simultaneous

measurements for the left and right legs 122 and 124 and examine the computed quantity throughout the course of complete strides.

It will be appreciated that with suitable attachments, devices 50a and 50b could be attached to other parts of person 126, for example directly to the feet themselves or to locations close to the knees or hips, thus enabling further study of gait characteristics, abnormalities and asymmetries. For example, orientation of the devices 50a, 50b when attached directly to the feet (not shown) could both confirm and quantify the extent of any rolling inward or outward of the feet (pronation) when walking.

Thus, examples of assessments known to be used in the general fields of walking, running, gait analysis and the like, and for which embodiments of the invention can be useful include (without limitation):

Ankle height during forward lifting phase of step;

Ankle height during rearward swing phase of step;

Knee height during stepping;

Stride length; General running or walking smoothness and stability;

Different types of attachments for securing the devices X to body parts are provided for different applications. Straps may be used for attaching devices X to wrists, ankles and other parts of arms and legs, for example.

Example: Range of movement

People undergoing rehabilitation after various forms of surgery, injuries and the like must often be assessed for their range of movement. For example, it may be desired to measure a range of angular movement of a lower leg relative to an upper leg or an angle of wrist rotation, or to measure the height above a flat surface that a person can reach, or to measure the height of a step a person can climb. For height measurements (not shown), the sensing device (eg 20, 50 or 72) can be secured to the relevant part of the body, or (in the case of reach height) to a handheld object, not shown, and the height measured directly relative to a flat floor based on outputs from the distance sensors. For anatomical angle measurements (not shown), the sensing device (20, 50 or 72) is simply secured to the relevant body part, and the change in angle measured as a change in orientation. Change of orientation may be measured using the distance sensors where practical but if not, a user can simply rely on orientation change determined from the IMU sensors, with or without filtering or smoothing, given the short term, one-off nature of the activity.

Example: Non-cyclic physical activities

The weight training example described above does not involve regular repetitions of particular motions of objects or body parts over a period of time, but rather involve movements that are executed one at a time in a brief period. Other examples of such activities include:

Bowling, pitching or throwing of balls (as in cricket or baseball);

Other“throwing” sports, such as (hammer, javelin and discus);

Archery and shooting (of small arms);

Punching and kicking (in combat sports) where height reached by a hand or foot can be important;

Sports involving the use of racquets (eg tennis) or clubs (eg golf) or other implements.

Generally, the remarks made for the weight training example apply here also. For example, filtering and smoothing of acquired data may not be necessary for brief activities.

For the archery and shooting examples, a sensing device such as device 50 may be secured on a bow (or firearm) or a sensing device such as device72 may be secured to a user’s wrist or arm to sense orientation and movement during the process of aiming and shooting.

Other activities in the above list involve bigger movements or are essentially outdoors activities.

At least for training purposes, it can be helpful in these cases to provide an environment in which surfaces are available that can be sensed reliably by the distance sensors throughout - or nearly throughout - the action of interest. This may simply be a room having not only a horizontal floor, but one or more walls with unobstructed areas and even an unobstructed horizontal ceiling. As mentioned in the weight training example, the position of the sensing device (e.g. 20 50 or 72) secured to a user or to an implement such as a racquet can be determined directly using distances sensed not only by reference to a floor but to one or more walls or a ceiling, increasing the quantity of data from the non-IMU sensor data to enhance reliability of position and motion state estimates. Kalman-type filters are known to be quite robust where some data streams are interrupted.

Generally, it is possible to ensure that at least two positions are available at which direct distance measurement can be effected, because the actions have beginning and end “rest” positions, where it will often be possible to deliberately orient the sensing device to obtain distance sensor-based positions. Sensing devices 12 (such as 20, 50, 72) and/or base stations 14 are in some embodiments provided with a“sense distance now” control to tell the software that a particular stationary position is to be relied on.

Example: Walking, running, jogging

These activities, carried on outdoors, are examples where the distance and path travelled is important, but in which drift in the outputs of heading and rotation rate sensors are subject to noise and/or drift effectively making those quantities difficult or impossible to rely on for movement over significant distances. Moreover, the outdoor surfaces which walkers, runners and joggers traverse may limit the accuracy of direct distance measurements.

Of course, embodiments can simply be used as pedometers for counting strides, or for making more detailed measurements (particularly velocity, acceleration of body parts) over short periods based on IMU data.

However, in some embodiments, a positioning system for establishing position on earth (i.e. latitude, longitude, altitude) is incorporated in the sensing device, along with the motion and short-range distance sensors. The system preferably relies on one or more established global navigation satellite system (GNSS), such as GPS or GLONASS, for which receivers of small size are available. Some of these can access more than one GNSS, for example GPS and GLONASS.

These can provide regularly updated position“fixes”, which can be used to at least substantially obviate the problem of growing uncertainty in position as estimated using motion sensors. The techniques of Kalman or particle filtering and sensor fusion may be used to do this in known manner.

GPS, GLONASS systems and the like work best when a receiver of their signals is outdoors, and are generally not useful in any case over the short ranges contemplated in most of the examples described herein, so the extra bulk of a separate antenna, can be avoided. However, sensing devices such as 20, 50 and 72 may be provided with connections for an external antenna if required. The antenna may be secured in the near vicinity of the sensing device and connected to it by a short cable.

Magnetometer data can be used in outdoor activities where interference from metal objects is generally less than in activities such as weightlifting, and can further reduce uncertainty in position.

Example: Vehicle motion and the like

Sensing devices 12 can be used in applications other than exercising and the like, for example in monitoring movements of machinery components, vehicles and the like.

For example, the suspension dampers (shock absorbers) of a motor vehicle may be considered suspect and device 20 or 50 could be used to test their performance in simple ways. Figure 14 shows device 20 temporarily mounted on a vehicle body 132 above a wheel 134, device 20 being oriented so that some of its distance sensors’ optical axes 34 are directed downwards to impinge on road surface 136. Only four of the optical axes 34 are shown. Data from all distance sensors of device 20 can be used, or it can be loaded with software that uses only a suitable nominated subset of distance sensors. When the vehicle is driven, excessive vertical motion of the body 132 in the area of the wheel 134 can be detected by measurement of variation in the height of the device 20 above the road surface 136 (and/or excessive vertical acceleration or velocity).

Another possible approach is shown in Figure 15. A cylindrical target 138 is temporarily secured to wheel 134 as shown, with its axis of symmetry aligned with the axis of rotation 138 of the wheel 134, and device 20 is mounted to the vehicle body 132, again with a (nominated) distance sensor optical axis 34 directed downwardly to impinge on the target 138. Measurement of vertical movements (distance, velocity and

acceleration) of the wheel 134 relative to the body 132 are then possible using device 20. (It would also be possible to measure both the position of the body 132 relative to target cylinder 138 and the road surface 136, using more of the distance sensors of device 20.)

Note that these are applications where the distance sensors to be used can be, and preferably are, predetermined by a user, namely the sensors whose optical axes are suitably oriented when the device 20 is secured to the vehicle body. Other possible vehicle-related applications can include determination of overhead clearances where vehicles must pass through tunnels, or under bridges, garage doors and the like.

One way for securing devices to steel vehicle bodies or machine parts is to use a permanent magnet either directly fixed to the devices themselves or incorporated in a fixture 140. Such a fixture 140 is shown in Figures 14 and 15 and has a magnet 142, and a ball joint 144 that can be secured in a fixed orientation by a screw 146, this allowing the device 20 and particular optical axes 34 to be suitably oriented for the application at hand.

Still another potential area of application is in movement of heavy objects. In civil construction, for example, it is often required to move heavy objects such as beams, bridge modules and the like, into precisely known positions with cranes. Devices such as 20 or 50 can be used to assist humans in monitoring the position and orientation of such objects as they are manoeuvred into place.

Example: Measurements of dimensions, spaces and the like.

A sensing device 12 (such as 20 or 50) is versatile and may be used as a simple distance measuring device or“electronic tape measure”, in which its motion analysis capabilities are simply not required and not used.

A further possibility is to use sensing device 12 (such as 20 or 50) for mapping or surveying of spaces. If a device 12 is handheld and simply waved through a range of positions and orientations by a person in a space such as a room, its distance sensors can generate a so-called’’point cloud” in essentially the same way as a LIDAR-type device with a rotating scanner as known in the surveying and robotics arts, and so provide a form of map of the space.

For these applications, base station 14 may be a mobile telephone (not shown) and sensing device 12 may be secured (for example by an adhesive pad) to the mobile telephone, with communication between the mobile telephone and the sensing device by Bluetooth or other short-range wireless technology.

Other potential applications of embodiments of the invention, to analysis of motion states and positions of humans, animals and machinery, will readily suggest themselves to persons skilled in the art.

Claims

1 A sensing device for sensing position and orientation of a moving object

comprising:

an array of one or more distance measuring sensors;

2. The sensing device of claim 1 wherein each one of the first three axes is collinear with one of the second three axes.

3. The sensing device of claim 1 or 2 further comprising magnetic field strength sensors adapted for sensing magnetic field strength along each of a third three axes of the structure.

4. The sensing device of claim 3 wherein each one of the third three axes is

collinear with one of the first three orthogonal axes.

5. The sensing device of any one of claims 1 to 4 wherein the first three axes are orthogonal to each other and converge at a single point.

6. The sensing device of any one of claims 1 to 5 further comprising a battery for powering the sensors, computing means and communication means.

7. The sensing device of any one of claims 1 to 6 further comprising a position sensing device that receives and processes signals from an earth satellite-based global positioning system.

8. The sensing device of any one of claims 1 to 7 wherein the acceleration sensors are combined in a MEMS device.

9. The sensing device of any one of claims 1 to 8 wherein the rotation rate sensors are combined in a MEMS device.

10. The sensing device of any one of claims 1 to 9 wherein the distance measuring sensors are time-of-flight sensors that estimate distance to a surface based on time taken for a radiated signal to travel from the sensor to the surface and to return.

11. The sensing device of claim 10 wherein the radiated signal comprises

electromagnetic radiation.

12. The sensing device of claim 11 wherein the electromagnetic radiation comprises infra-red radiation.

13. The sensing device of any one of claims 1 to 12 wherein optical axes of the

distance measuring sensors extend outwardly from a point in the sensing device.

14. The sensing device of claim 13 wherein the optical axes of a plurality of the

distance measuring sensors lie within a plane fixed relative to the sensing device.

15. The sensing device of claim 14 wherein the optical axes of the plurality of

distance measuring sensors are circumferentially spaced apart in the plane.

16. The sensing device of claim 14 or 15 wherein the plane is one of a plurality of planes fixed relative to the sensing device and wherein each of said plurality of planes includes a plurality of the distance measuring devices.

17. The sensing device of claim 16 wherein each the plurality of planes include a single point.

18. The sensing device of any one of claims 13 to 17 wherein the point from which the optical axes of the distance measuring sensors extend outwardly is a point through which the first three axes pass.

19. The sensing device of any one of claims 1 to 18 further comprising display

means for displaying visually and/or causing to be audible at least one indication of an aspect of the attitude or state of motion of the sensing device relative to the earth.

20. The sensing device of claim 19 wherein the indication is of a departure of an axis of the exercise component from a preset direction.

21. The sensing device of claim 6 wherein the data processing device is adapted to use outputs of sensors comprised in the sensing device to determine information computed from the outputs from said outputs of sensors for output from the sensing device via the communications module.

22. The sensing device of any one of claims 1 to 21 wherein outputs from sensors comprised in the sensing device can be output from the sensing device via the communications module.

23. The sensing device of any one of claims 1 to 22 further comprising means for securing the sensing device to an object.

24. A system for acquisition storage and processing of information on position

attitude and motion of moving objects comprising a sensing device according to any one of claims 1 to 23, and wherein the computing means or a separate computing means executes software to derive from outputs of sensors of the sensing device information comprising at least one of position, orientation, acceleration and velocity of the moving object.

25. The system of claim 24 comprising a base station adapted to transmit control commands and software to the sensing device.

26. The system of claim 24 or 25 wherein the sensing device and the base station communicate digitally with each other.

27. The system of claim 26 wherein the sensing device and the base station

communicate wirelessly with each other.

28. The system of any one of claims 24 to 27 wherein the sensing device is

securable to the base station.

29. The system of any one of claims 25 to 28 wherein the base station comprises a mobile telephone.

30. The system of claim 25 wherein the sensing device can be loaded with any one of a plurality of software items each adapted for use with the sensing device in a particular application.

31. The system of any one of claims 24 to 30 wherein the base station can store parameters enabling drawing on a display of a representation of a particular item or component and thereafter can draw the particular item in motion on the display using motion parameters obtained from the sensing device.

32. A method for provision to a user of position and orientation information of a

moving object comprising: in a first step, under control of software executing on a computing means, one or both of: a first stream of information comprising at least one of position and orientation of a moving object derived from outputs of distance measuring sensors moving with the object; and a second stream of information comprising at least one of position and orientation of the moving object derived from outputs of acceleration sensors and rotation rate sensors moving with the object; and in a second step, processing the first and/or second streams of information to arrive at a third stream of information on position and orientation of the object and providing the third stream of information to a user.

33. A method according to claim 32 including the step of deriving and providing to the user information on acceleration and orientation of the moving object from the acceleration and rotation rate sensors.

34. A method of claim 32 or 33 wherein the first stream of information is derived from a dedicated distance sensor.

35. The method of claim 34 wherein the dedicated distance sensor comprises an array of distance sensor units.

36. The method of claim 35 wherein the distance sensor units are mounted in

substantially the same plane.

37. The method of claim 35 or 36 wherein the distance sensor units are arranged radially about an axis.

38. The method of claim of any one of claims 34 to 37 wherein the dedicated

distance sensor comprises a laser based distance measuring device.

39. The method of claim 38 wherein the laser based distance measuring device is a LIDAR device.

40. The method of any one of claims 35 to 39 wherein the distance sensor units comprise discrete LIDAR devices.

41. The method of any one of claims 31 to 40 wherein the second stream of

information is derived from an IMU.

42. A method of assessing the attitude of an exercise component when the

component is operated by a user as the user performs a repetitive physical activity with respect to a surface; the method implemented on a computing device and comprising:

receiving attitude data comprising sensed acceleration data from at least one motion sensor mounted to the component that comprises an accelerometer that is configured to sense and generate acceleration data in at least three axes, the sensed acceleration data representing a first component of attitude of the component; and receiving distance data comprising sensed distance data from at least one dedicated distance sensor mounted to the component and wherein the sensed distance data represents a second component of attitude of the component; processing the first component of attitude and the second component of attitude in order to generate output feedback data comprising attitude of the component with respect to the surface (from which the user obtains leverage).

43. The method of claim 42 wherein the second component of attitude is derived from a dedicated distance sensor.

44. The method of claim 43 wherein the dedicated distance sensor comprises an array of distance sensor units.

45. The method of claim 44 wherein the distance sensor units are mounted in

substantially the same plane.

46. The method of claim 43 or 44 wherein the distance sensor units are arranged radially about an axis.

47. The method of claim of any one of claims 43 to 46 wherein the dedicated

distance sensor comprises a laser based distance measuring device.

48. The method of claim 47 wherein the laser based distance measuring device is a LIDAR device.

49. The method of any one of claims 44 to 48 wherein the distance sensor units comprise discrete LIDAR devices.

50. The method of any one of claims 42 to 49 wherein the first component of attitude is derived from an IMU.

51. A display output device in communication with the sensing device of any one of claims 21 to 23; the display output device outputting data derived from the sensing device.

52. An exercise display output device in communication with the sensing device of any one of claims 21 to 23; the display output device outputting data derived from the sensing device.

53. A barbell exercise display output device in communication with the sensing

device of any one of claims 21 to 23; the display output device outputting data derived from the sensing device.

54. A barbell exercise display output device in communication with the sensing device of any one of claims 21 to 23; the display output device outputting data derived from the sensing device in response to movement of a barbell; the sensing device located on an end of the barbell.