AU2014250691A1 - Positional feedback device - Google Patents

Abstract

Positional feedback device Abstract: An apparatus comprising at least one sensor to detect the posture and / or physical location of a body part of a subject, the sensor in communication with a computing device to process sensor data and optionally a transmitter to transmit sensor data between the sensor and the computing device and / or 5 one or more additional computing devices. F gure Overall Flex Processing Smartphone ~ Internet/ Sensor Unit With External Bluetooth Databases rFCurvature rComms Back Accleroar.e...Bl.e..th.S.s.tphon ___________________ INyo p

Description

Positional feedback device Background of the invention: More physical injury is caused by people placing themselves in bad posture over extended periods than from sharp forces on their body. This is because bad posture over a period of time applies more force over time, compared to forces of a similar magnitude applied as a short impulse. The kinds of posture 5 that could result in long term posture related injuries could also posture whilst using a computer, lifting heavy objects, or back strain whilst undertaking a repetitive physical task, such as hammering nails in an awkward position. Currently there are a number of methods attempting to curb such posture issues. For example, there is a direct message advertising campaign that carries a message on how to maintain good posture in the 10 workplace and home while using computers. There are also training given to labourers who often have to carry heavy items, especially with common tasks like restocking high shelves. Bio Corrective Physiotherapist already use posture tapes to fix bad posture after it happens. This often requires trips to the Physiotherapist, which takes up their time that could be better spent on more urgent injuries. 15 Bio Measurements In the medical field there are already expensive devices (often op- tical fibres), used to get exact angle of the body for research purpose. This is important, but since its for research and is bulky and expensive, it is unsuitable for prevention of back injuries over time. Bio Feedback For example simple programs that at specific intervals recommend you to perform stretches to relieve muscle strain before using the computer again. In addition, just recently in a higher 20 tech manner, there are some devices such as the LUMOBack that measures how often you are sitting down or running, or conducting other activities based on a single orientation sensor attached to the lower back. Often they encourage healthier lifestyle by turning good activities like running into a score, which essentially gamifies and provide incentives to do better and be healthier. None of the current methods sufficiently incentivise the user to correct their behaviour, or are priced at a 25 point for general usage (and this thus limits the method's effectiveness to a reactionary solution, as opposed to a preventative solution). The reference to any prior art in this specification is not, and should not be taken as, an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge. 30 Summary of the invention: 1 Positional feedback device In a first aspect of the invention, there is provided an apparatus comprising at least one sensor to detect the posture and / or physical location of a body part of a subject, the sensor in communication with a computing device to process sensor data and optionally a transmitter to transmit sensor data between the sensor and the computing device and / or one or more additional computing devices. Posture and / 5 or physical location can be sensed in relation to any suitable body part. For example, it may be a joint or a series of joints such as the spine, a shoulder, an elbow, a wrist, a digital joint, a hip, a knee, an ankle, or it may be a particular location such as particular bony prominence or other anatomical landmark. The invention is useful for health and medical uses such as fix postural or repetitive strain type injuries. However, it may equally be used for other applications, such as in support or other human movement 10 areas. In some embodiments, the invention is used for example to monitor and alter a golf swing, a tennis swing, a football kicking action etc. In some embodiments without the optional transmitter, the computing device is in physical communication with the sensor. In some embodiments, the computing device is not physically near the sensor so that communication must be via wireless or other method. Sensor data may be processed for 15 example by intelligent components of the sensor prior to transmission or it may be transmitted directly to the computing device and then processed. In some preferred embodiments, the sensor comprises a disposable strip which can be reduced to a required length and the strip optionally comprises one or more predetermined locations at which length can be reduced. As an example, the strip may comprise one or more perforations at one or more 20 predetermined locations so that it may be torn or cut at a certain length to suit the length of the vertebral column of a particular subject. In such examples, one end of the strip may for example plug directly into a device such as a data handler, a computing device and / or a transmitter whilst the other end comprises one or more of such locations at which length can be reduced. Reduction in length may be attained by any suitable means. In some embodiments, the strip is 25 telescopic so that the strip may be slid down to an appropriate size. In other embodiments, the strip comprises a series of separate smaller strips which can be put together to create a particular size and / or shape. In some embodiments of the invention, the sensor comprises a resistive ink and a conductive ink. Each of these inks may be of any suitable type. 30 In some embodiments, the sensor comprises an adhesive section to adhere to the body of the subject wherein adhesion to the body optionally comprises one or more of adhesion to a close fitting garment and adhesion to the skin (and or hair) of the subject. The adhesive section may comprise any suitable adhesive. In some embodiments it is a medical grade adhesive suitable for direct contact with skin. 2 Positional feedback device In some embodiments, the sensor comprises a perforated section to adhere to the body of the subject wherein adhesion to the body optionally comprises one or more of perforations to a close fitting garment and adhesion to the skin (and or hair) of the subject. The perforatated section may comprise any suitable perforation. In some embodiments it is a manufaturing perforation suitable for direct contact 5 with skin. In some embodiments, the sensor comprises folds or manufacturing scoring sections to adhere to the body of the subject wherein adhesion to the body optionally comprises one or more of scores to a close fitting garment and adhesion to the skin (and or hair) of the subject. The folded sections may comprise any suitable folding. In some embodiments it is a manufaturing scoring or folding suitable for direct 10 contact with skin. Some embodiments of the invention comprise one or more disposable components for low-cost replacement which optionally comprises a sensor strip. In some of these embodiments, the sensor strip itself is disposable. Some embodiments of the invention further comprise a data handler to receive sensor data from the 15 sensor and optionally store, and /or manage communication of said data to the computing device. Communication between the sensor and data handler may be by any suitable means, for example physical connection, wireless communication, Bluetooth, zigby, cellular network, satellite and so on. In some embodiments, the sensor communicates directly with the computing device. Such communication may be by any suitable means, including physical connection, wireless communication, 20 Bluetooth, zigby, cellular network, satellite and so on. According to another aspect of the invention there is provided, a system for monitoring the posture and / or physical location of a body part of a subject comprising a sensor and a computing device in communication with the sensor, the computing device able to receive, store and process sensed posture data into a form suitable for providing posture feedback to the subject. Some embodiments of the 25 invention provide a system comprising a mobile computing device to notify the subject about posture feedback and optionally store and / or process sensed posture data. In some important embodiments, the apparatus and / or system of the invention uses 3 key parts. First it uses a wearable, adhesive tape sensor which can accurately measure the entire spine, calibrating to both neutral and best-achieveable spinal position for a user. An important aspect of this is the adhesive 30 and folding aspects of the sensors which stick to the body and greatly reduce measurement errors. In these embodiments, the tape sensors connect to a separate processing and transmission device as the sensors are intended for single to a few uses only (they are disposable). In some embodiments, the transmission device connects to a smart mobile phone over bluetooth and allows gamification incentives, recording of the postural history of a user. With software upgrades to the transmission device 3 Positional feedback device and mobile, the disposable adhesive strip allows for future applications to different parts of the body, for example the shoulders, which are an obvious extension to the spine in terms of posture correction. Without measuring the entire spine, the user or physicians cannot get accurate real time or recorded data on the entire spine and as such similar devices. 5 Throughout this specification (including any claims which follow), unless the context requires otherwise, the word 'comprise', and variations such as 'comprises' and 'comprising', will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps. 10 Brief description of the drawings: Figure 1 depicts an example PCB design according to one embodiment of the invention. Figure 2 shows a breakdown of various subsystems of an example system according to the invention. Figure 3 shows measurement of an optical fiber based sensor. -90 is left, +90 is to the right. Figure 4 shows a typical foil strain gauge design. 15 Figure 5 shows an example of interweaved elastic tape used for clothing Figure 6 shows resistance vs elongation from 40mm relationship of Bare Conductive painted elastic sensor. Figure 7 shows resistance vs elongation from 40mm relationship from one batch of Bare Conductive soaked elastics. 20 Figure 8 shows variance of infused sensor stretch test. Figure 9 shows that as the sensors bend, micro cracks appear ad the resistance increases. Figure 10 shows resistance vs flex angle profile of two individual SpectraSymbol 2.2" FS flex sensors. Both data series are normalised by subtracting out the intrinsic resistance. A 4th order polynomial trend line is included. 25 Figure 11 shows SpectraSymbol 2" FS flex sensor. Resistive element of the sensor is the black strip, with conducting elements placed on top to reduce resistance. Two sensors are arranged in opposite directions forming half of a Wheatstone bridge. Figure 12 shows normalised voltage of SS flex sensor pairs. A sweep left (+)90 and right (-)90 results in hysteresis 30 Figure 13 shows a handmade resistive sensor using BC paint, exible PET card and aluminium foil. 4 Positional feedback device Figure 14 shows resistance vs flex angle profile of hand made resistive ink based flex sensors using Bare Conductive paint, aluminium foil on flexible PET card. Both data series are normalised by subtracting out the intrinsic resistance. Figure 15 shows printing pass 1 - Carbon (cyan). Pass 2 & 3 - Carbon and Silver (black) 5 Figure 16 shows single and double pass resistance curing in open air. Figure 17 is a microscopic view of tin particles used in an inkjet additive process. Figure 18 shows raw resistance vs flex profile for 6 individual Methode ink flex sensors on Methode PET. A linear trend line is included. Variance is consistent as the sensors are bent between -90 to +90 Figure 19 shows variance of sensors on Methode ink/PET across one batch. A 3rd order polynomial 10 trend line is included. Figure 20 shows a computer generated model of final spine sensing tape. Figure 21 shows normalised voltage to flex angle profile of Methode ink flex sensor pairs. A 3rd order polynomial trend line added for each sensor's raw data. Figure 22 is a breakdown of various subsystems of an example processing unit 15 Figure 23 is an example PCB design with highlights. Multiplexer on underside of PCB. (Original image at Figure 1) Figure 24 shows LTC4080 Typical Application Schematic Figure 25 shows an opAmp configured as a resistance to voltage converter. Requires negative Vref for positive output. 20 Figure 26 shows a schematic of a half bridge Wheatstone and an instrumentation amplifier Figure 27 depicts creation of an instrumentation amplifier out of discrete opAmps Figure 28 shows an AD8236 Breakout Figure 29 shows a MCP42XX Digipot Pinout Figure 30 shows a Voltage Splitter TLE2426 Schematic 25 Figure 31 shows a TLE2426 Pinout and Vi/Vo curve (Vi:Vin)(Vo:Vout) Figure 32 shows a ADXL345 Breakout Figure 33 is a high level representation of the firmware code structure according to one example embodiment. 5 Positional feedback device Figure 34 is a flowchart of a scheduler and a delay based system, of which the scheduler is used in SpineSensorV2A FirmwareVO 8 1.ino Figure 35 is a functional flowchart of readSensoro Figure 36 depicts a visual explanation on how body overall curvature is obtained, and the body's 5 orientation is ignored. Figure 37 Shows how the component vector a.b relates to c in terms of getting the angle e in a consistent manner Figure 38 is a functional flowchart of calibrateSensoro Figure 39 depicts overall Mapping of Atmega32U4 SRAM 10 Figure 40 Shows how the stack and heap grows within an AVR micro controller Figure 41 Shows the front side and the backside of the bottom mount microusb SMD port in our final PCB. Figure 42 shows an example connection arrangement for serial USART in asynchronous mode Figure 43 shows a pinout according to the RN42 data sheet, as well a flatbed scanned underside 15 dimensions of the pads Figure 44 Shows how SPI consist of a ring of shift registers with clock and chip select Figure 45 depicts addressing individual SPI devices with multiple CS# lines Figure 46 depicts cascading multiple SPI devices with a ring topology. All sharing same CS# lines Figure 47 depicts a typical SPI communication of a command and a byte being transferred 20 before executing the command and value on rising edge of CS# Figure 48 depicts a Command Byte breakdown diagram Figure 49 depicts how to interpret SPI modes visually. First mode highlighted. Figure 50 shows typical wiring of an 12C device. Take note that there is a mandatory need for pullup resistors for SCL and SDL. 25 Figure 51 depicts an ADXL345 register map Figure 52 depicts an ADXL345 POWER CTL register Figure 53 depicts an ADXL345 DATA FORMAT register Figure 54 shows SO to S3 used as multiplexer channel selection. 6 Positional feedback device Figure 55 shows wireframe draft art of the User Interface for the PhoneGap Application. Left: Front-back flex. Right: Side to side flex Figure 56 shows an example PhoneGap architecture. Figure 57 shows a benchmark of popular jQuery syntax compatible libraries for html and 5 class functions. Figure 58 depicts a canvas element. Measurement unit is in pixels. The origin, (0,0) is on the top left. The canvas is 500 x 375 pixels. The bottom right position is (500, 375) Figure 59 depicts a comparison of javascript canvas libraries Figure 60 shows Javascript and PaperScript scoping 10 Figure 61 shows JavaScript, PaperScope and window global variables (data store) Figure 62 depicts an example calculation of the deviation from user-saved best posture position Figure 63 depicts an example 3D representation of the mobile application. Separate canvas for each curve. Figure 64 depicts an example Phone application process 15 Figure 65 depicts a jsperf.com benchmark with many loop implementations is available. Looping performance benchmark. Figure 66 depicts example rendering: Left: No subpixel rendering (aliased), Right: Antialiasing is used to smooth the sprite as the origin point is not a set integer.) Figure 67 depicts First is the automated rig testing a flex strip. Second is the automated rig testing an 20 optical break flex sensor. Figure 68 shows which ex sensor represents which subplot Figure 69 depicts an example multiplexer Figure 70 depicts an example microcontroller Figure 71 depicts example Power Systems 25 Figure 72 depicts a RN42 Radio Module Figure 73 depicts a Zif Socket Figure 74 depicts an example Amplifier and Tuning System 7 Positional feedback device Figure 75 depicts an example Vibrational Motor Driver Figure 76 depicts an example Vibrational Motor Driver Figure 77 depicts an example Multiplexer 5 Detailed description of exemplary embodiments: It is convenient to describe the invention herein in relation to particularly preferred embodiments relating to sensing posture and in particular by reference to spine position. However, the invention is applicable to a wide range of environments and situations and it is to be appreciated that other constructions and arrangements are also considered as falling within the scope of the invention. Various modifications, 10 alterations, variations and or additions to the construction and arrangements described herein are also considered as falling within the ambit and scope of the present invention. Examples of such modifications include sensing the location in space and movement of other joints, such as a shoulder, knee, elbow, wrist, and so on. Example Implementation - EWo3-IP 15 This example provides a human Spine Corrective Device, consisting of a flexible tape that can be stuck to a person's back which is calibrated to the user and sends wireless signals to a smartphone for recording and review. The device is easily applied with or without medical assistance to help self correct spinal and posture problems. For people suffering from spine-related issue, this device is aiming to provide society a cheap easy and 20 accurate alternative to monitor their spine regluarly. The signal will be sent from device and displayed on the screen to a handheld device such as mobile. Various sensors can be used or interchanged as detailed below but the initial rollout will use a custom sensor design with linearity and performance at a low cost. The Ewo3-IP example is a viable consumer and medical device, that helps reduce the occurrence of 25 spinal injuries related to lifestyle and work habits. This involves any extended movements and static posture that would result in series of muscle strain and injuries over time. These are some of the variables the devices achieves; Accuracy - Must be able to inform the user with enough information to correct their posture. Cost - It must also be cheap enough, similar to single visit to a physician 30 Motivation - Accuracy doesn't mean anything if the user doesn't follow the instruction, so the device incentivises the user to correct their actions with stiumuli. 8 Positional feedback device Reliability - Reliability is required so that the users can trust the results, accuracy and improve their posture Interoperability - The device ties in with other solutions in the space, eg. physicians The device is aimed towards the consumer market instead of the medical market. This means increased 5 emphasis on minimising cost. This particular example is focused more on preventing upper back injuries, over the lower back, as upper injuries being more common. This example also prioritises forward and backwards measurements over side to side posture, as most people know how to maintain side to side posture naturally. Biofeedback is used for incentives and feedback (via long term graphs, medium term visual feedback by 10 mobile phone app, to immediate feedback via haptic feedback alerts). User can also message their physiotherapist or chiropractors on their progress. Figure 2 : Shows the breakdown of various subsystems of this example. It illustrates how the overall posture system can possibly be broken down into including; - Curvature of back 15 - Flex Sensors - Accelerometer/Gyro - Data collection unit - Bluetooth communication - Smart phone with Bluetooth 20 - Smart phone app with Calibration - Send data remotely to health professionals e.g. physiotherapist Sensors The overall system design revolves around a spine sensing tape. The flex sensor should has a measurable characteristic that changes with every bend. Each sensing tape's configuration consists of a 25 number of sensors such that it can provide resolution to detect how much a person is bending (or slouching in the incorrect posture), both in the front and side directions. The size profile is slim so it is easy to apply without being invasive on the user. The tape has been designed to be low cost and conform with existing manufacturing processes. There are three types of sensor characteristics; unidirectional, bidirectional and bipolar. 30 Unidirectional sensors will only change properties when bent in one of two opposing directions. 9 Positional feedback device Bidirectional sensors change properties when bent in both opposing directions, measuring only the magnitude of motion. Bipolar sensors change properties in both opposing directions, each direction yielding a different measurement. 5 The performance of different sensing technologies were assessed, the final sensing tape configuration consists of a resistive based flex array for forward back motion and (optional) accelerometers for enhanced functionality and accuracy. Fiber Bragg Grating Sensors Fiber Bragg (FBG) sensors consist of a specially made optical fiber where the sensing area is treated to 10 selectively act as a mirror for certain wavelength of light. Straining the fiber creates a shift in the spacing of the grating, which can be detected as a change in light reflected from input light source, or its corresponding decrease in light intensity on the other end of the fiber). This is accomplished by periodically changing the refractive index of the sensing area of the core fiber. It has the advantage of very high accuracy. 15 Fiber Optics Sensors By Selective Abrasion or Breaks A break type of flex sensor may also be used in some applications. This involves a 0.5-1 mm gap cut in the optical fiber, covered over with heat shrink. A white LED on one end and a light dependant resistor on the other end, the automated rig reported these results in Figure 3. This shows that the optical flex sensor and LDR combo is not linear, but has high repeatability over at least 3 consecutive trials. The 20 data suggests this sensor is bidirectional, and cannot be used to detect forward or backward motion, only the magnitude of motion. Micro Electromechanical Systems (MEMS) MEMS are ICs that map mechanical changes, such as accelerometers or gyroscopes. MEMS are extremely accurate relative to their cost. It is often seen in biomedical applica- tions involving the human 25 body. Figure 3 : shows the measurement of an optical fiber based sensor. -90 is left, +90 is to the right. An average of the trials is included in the chart. Resistive Based Sensors Bare Conductive paint is a non-toxic water based paint that is infused with carbon polymers. It has a resistance of 55 Ohms/Sq/micron, suitable for screen printing or painting. 30 Conventional strain gauges are passive devices that rely on resistance between fine fila- ments of conductive foil on a insulating flexible base such as polyimide. As the sensors are elongated, the cross 10 Positional feedback device sectional area changes as the filaments displace one another. The long parallel filament layout limits the direction to one dimension. Strain sensors have a multitude of different uses, particularly in mechanical and civil engineering disciplines, used to measure force applied to an object. Typically, 5V to 12V is applied and the change 5 measured is in millivolts. It is commonly connected through a Wheatstone bridge configuration before an amplification stage. Strain gauge sensors are able to detect both flex and elongation, which make it suitable for our example. By integrating two columns of strain sensors embedded on the tape, it is possible to detect both forward and side motion. Note: Ohms/sq(/micron) is the industry standard measurement for conductive inks. Sq., is defined as a 10 ratio of length/width of the resistive section, actual measured resistance is subject to variance due to application methods (particularly /micron (z-direction, application thickness) The characteristics of conventional strain sensors were replicated through the use of elastics fibres applied with Bare Conductive paint. (See Figure 5 : Interweaved elastic tape used for clothing) As the length of the elastic is increased, the distance between conductive carbon particles increase both within 15 the cross-braid structure. Elongation of the elastic fibres increase resistance when measured end to end. The braiding technique for the elastic conveniently limit stretching only in one dimension (lengthwise) analogous to foil based gauges. The resistance profile can be seen from Figure 7, by integrating the strain sensors pre-stretched, bipolar sensing can be achieved. Different application methods may be used; direct painting and infusing/soaking. Initially, a direct paint 20 application onto the surface of the elastic produced a significant increase in resistance after a short change in length. By infusing the sensors with Bare Conductive paint diluted with water, more carbon particles are introduced and available for conductivity. From Figure 8, there is significant variance between batches. Hardware / software calibration is possible, but would require a prolonged set up routine for the end user. The inconsistency issue can be resolved through mass manufacturing 25 processes. Resistive Ink Based Sensors These flex sensors are typically composed of a strip consisting of a resistive strip on an insulating material much like strain sensors, but do not require the same amount of manufacturing precision. The resistive ink is brittle, but is able to form a strong bond with the base material. As the sensor is bent, the 30 insulating base within the localised area of bending will exhibit stretching which will pull the conductive particles apart and induce permanent micro cracks. This is the property that is exploited; further bends will open and close these gaps, resulting in a change in resistance with flexion. When at rest, the flex sensor returns to its intrinsic resistance (dependent on the shape of the resistive strip). 11 Positional feedback device The flex sensor resistance varies depending on the radius at the flex location. The smaller the radius of a sharper bend the greater the resistance will be, due to micro cracks being further apart compared to a bend with a large radius, while it may also cause permanent damage. Furthermore, as the flex sensors are typically 2" to 5" in length, multiple bends along one sensor will give a unrepresentative resistance 5 and flex angle. Thus, each sensor must limit bending exposure to one particular point while also maintaining a consistent bend radius for best accuracy. Flexpoint, SpectraSymbol and Infusion Systems are manufacturers of commercial grade flex sensors and like most flex sensors on the market, they are all unidirectional sensors. Flexpoint claims there is no hysteresis associated with their sensors. It is worthy to note that the resistance to bend profile of the 10 Flexpoint sensor is non-linear. Figure 6 shows Resistance vs elongation from 40mm relationship of Bare Conductive painted elastic sensor. A sensor using the soaking method is added for comparison. Both data series have a 3rd order polynomial trend line. Figure 7 shows Resistance vs elongation from 40mm relationship from one batch of Bare Conductive 15 soaked elastics. A 3rd order polynomial trendline is inserted with each sensor. Figure 8 shows Variance of infused sensor stretch test. It is notable that variance increases significantly across the same production batch. Figure 9 shows As the sensors bend, micro cracks appear ad the resistance increases. At localised area of bending, distance between induced micro fractures increase, increasing resistance of the sensor. 20 Resistive Ink Flex Sensor Development Analysis of the SpectraSymbol FS Flex Sensor Commercial flex sensors by SpectraSymbol were used in one example. The sensor consists of a resistive element layered with conducting stripes, Figure 11. The main element is the resistive component which changes due to flex. The conducting white stripes may serve the purpose of 25 decreasing the intrinsic resistance of the sensor end to end. From figure 10, it can be noted that it is non-linear within the region of operation 0 - +90 (away from the printed side). When bending into the region of operation, the length of the base material increases, thereby increasing the distance of each each micro crack. Flexing towards the printed side (-90) does not decrease resistance. This is due to the high density of resistive fragments; the distance cannot be decreased any further, thus resistance only 30 changes by a minimal amount. Since the SS flex sensors are unidirectional, one sensor alone is not enough to measure both forward and backward directions of motion. By placing two opposite back to back and securing with adhesive (figure 11, at least one of the two sensors will be in the active region of operation. The sensor pair is 12 Positional feedback device connected as a half Wheatstone bridge, used to measure the voltage difference with changes in angle flexion. While flex sensors do not typically exhibit hysteresis , we encountered this problem during our measurements as seen in figure 12. A slower measurement with pausing in-between only showed a 5 minor improvements. The adhesive between the sensors may be the cause; it increased the mechanical resistance to flex by limiting room for slippage between the pair. Figure 10 shows Resistance vs flex angle profile of two individual SpectraSymbol 2.2" FS flex sensors. Both data series are normalised by subtracting out the intrinsic resistance. A 4th order polynomial trend line is included. 10 Figure 11 shows SpectraSymbol 2" FS flex sensor. Resistive element of the sensor is the black strip, with conducting elements placed on top to reduce resistance. Two sensors are arranged in opposite directions forming half of a Wheatstone bridge. Figure 12 shows Normalised voltage of SS flex sensor pairs. A sweep left (+)90 and right (-)90 results in hysteresis 15 Handmade Resistive Flex Sensors Figure 13 shows an example handmade resistive sensor using BC paint, exible PET card and aluminium foil. In another example a resistive flex sensor was created b hand using BC paint, flexible PET card and aluminium for the conducting stripes. The resistance behaved similarly to the SS flex sensors. Due to manufacturing inconsistencies, there were some variations within the same batch of both intrinsic and 20 operating behaviour due to uneven BC paint application (in the z-direction). While the range of resistance varies widely, 7/10 of the sensors created exhibit a similar change when subject to flex. Inkjet Printed Resistive Flex Sensors There are many methods used for commercial manufacturing of flexible PCBs. Along with conventional semi-additive processing; screen and inkjet printing methods are given in this example. Figure 14 25 shows Resistance vs flex angle profile of hand made resistive ink based flex sensors using Bare Conductive paint, aluminium foil on flexible PET card. Both data series are normalised by subtracting out the intrinsic resistance. Methods Developed for the Device Semi-additive method is a common method to manufacture PCBs. DuPont's flexible polyimide based 30 Pyralux is used as drop-in replacement for fiberglass backing. A solid ink printer is used to create the subtractive mask. This would only solve the issue of conductive tracks, not application of resistive ink. 13 Positional feedback device Screen printing is an additive method used for large format poster printing. With the introduction of conductive and resistive inks, they are now commonly used in the in- dustry for circuits involving customised shapes. Both Flexpoint and SpectraSymbol sensors are printed with this technique due to references in their MSDS and tech- nical data sheets. The cost to set up each template mask is $200 5 300, however on-going manufacturing costs are minimal. Inkjet printing is also an additive method using inkjet printing technology with special inks where the particle size is in the nanometre scale. Due to the required amount of processing, the cost of ink is very high, but does not involve additional setup procedures, therefore number of different designs are not limited by cost compared to screen printing. As the device involves a lot of testing and prototyping, inkjet 10 printing was the most suitable option for this example. Most inkjet inks are water based, they contain other additives such as cellulose resin and humectant to stabilise the ink for storage and printing by controlling the viscosity to suitable levels (<20 cP ) so it can be suspended within the print nozzle without leaking. Most importantly, colour pigments or conductive particles must be in the nanometre scale to not clog up the printhead. This means, diluted Bare 15 Conductive paint (or from other manufacturers such as Conductive Compounds) cannot be used in an inkjet printer due to large particle size. Sonication is required through a ultrasonic bath to reduce size before it is fit for use. Methode Electronics Conductive Ink There are many providers of inkjet compatible inks in the market, such as InkTec, Methode Electronics 20 and Plextronics. InkTec and Methode provide silver and carbon based inks, while Plextronics also produces organic polymer based inks used for photovoltaic and OLED circuitry. In this example embodiment, Methode ink is used Methode PET are specially designed for Methode Electronics and their inks. Both have been treated to encourage ink drying and adhesion. White marks appear when treated with damp cotton bud. Both PET 25 versions are semi-transparent. Printing technique The sensors were printed using a 3-pass process. Firstly, a carbon layer is applied, followed by two more layers of silver + carbon on top. Figure 15 shows Pass 1 - Carbon, Pass 2 & 3 - Carbon and Silver. A multiple pass technique is required to prevent any microscopic gaps between silver and carbon 30 sections as the printer can only print one type of conductive/resistive ink at any one time, to maintain ink integrity. Using a layering technique, there will be at least be one continuous path end to end (through the bottom carbon layer). Conductivity decreases linearly each time a new layer (1 micrometre z) is applied, figure 16. 14 Positional feedback device Alternative Flex Sensor construction Sensors were designed and printed on both Methode PET and Epson polyester. Individual sensors were tested across a 180 degree range, from figure 18 there is very significant linearity across the -90 to +90 range. The data has not been normalised to showcase the minor variation achieved with inkjet printing. 5 Variance is charted vs flex angle in figure 19; the amount is very minimal and consistent among the entire 180 degree range of operation. Figure 16 shows single and double pass resistance curing in open air. Figure 17 shows Microscopic view of tin particles used in a similar inkjet additive process. Despite having a similar layout compared to the commercial flex sensors, we were able to create a bipolar linear sensor compared to the non-linear unidirectional characteristic. This is due to the low 10 density of resistive particles applied by the print head. A microscopic scan of a similar inkjet application process from Figure 2.18 shows gaps between each droplet. While these gaps are reduced and covered through multiple passes, this introduces spacing in the z-direction, creating microscopic distances between micro fragments. This gives additional room for particles to 'travel' as the material base is flexed in either direction. When bending towards the printed side, the material base reduces in length 15 and reduces in distance, leading to a resistance decrease. Figure 18 shows Raw resistance vs flex profile for 6 individual Methode ink flex sensors on Methode PET. A linear trend line is included. Variance is consistent as the sensors are bent between -90 to +90. Figure 19 shows variance of sensors on Methode ink/PET across one batch. A 3rd order polynomial trend line is included. 20 Despite already creating a sensor that is both bipolar and linear within the range of operation, we decided for this example to continue using the pair configuration analogous to the commercial flex sensor, (forming half of a Wheatstone) without significant changes to the existing circuit design on both the sensing tape and processing unit. The only cost is the amount of time associated with joining the segments by hand due to the constrained A4 size of the printer itself. 25 Each pair of sensors are laid out in a vertically cascading format to both increase resolution and reduce missing capture of bends (compared to end to end layout). The sensing tape consists of twelve pairs of sensors, each pair detecting for either forward and backward directions of motion. Figure 20 shows Computer generated model of final spine sensing tape. Figure 21 shows Normalised voltage to flex angle profile of Methode ink flex sensor pairs. A 3rd order polynomial trend line added for 30 each sensor's raw data. Main VCC and Ground lines power the sensing tape sensors. The silver #9101 conductive ink has a significant resistance due to the size of the nanoparticles. To cover the long distance (y-direction), resistance is minimised by widening cross sectional area of VCC and ground lines (x-direction) and 15 Positional feedback device printing multiple passes (z-direction). Furthermore, due to the flaw of VCC and ground being placed along the side, induction is reduced by cross-connects at various points using conductive Kemo Li 00 silver paint. Pairs of sensors are joined using CircuitWorks silver conductive expoxy similar to a PCB via. The sensors use the vias as anchor points. There is no additional adhesive holding the pairs together to 5 reduce any hysteresis caused by inflexibility observed with the commercial sensor. Due to space constraints, the silver signal tracks have resistance in the range of 200 Ohms, however this is insignificant as the intrinsic resistance of the flex sensors are in the range of KOhms and the Wheatstone configuration. There is minimal noise due to the passive sensor design. Additional expansion capabilities are possible with exposed 12C and power contact pads that transverse 10 to the top of the tape where accelerometer clips can be attached. A 20pin 1mm width ZIF pad is printed directly using the printer that connects to ZIF on the processing unit. PCB and Hardware Design Figure 22 shows the breakdown of various subsystems of the processing unit This section concerns itself with mainly Processing Unit, Accelerometer/Gyro and the design process it 15 took to reach the final design for both. Some attention will be given to Flex Sensor strip in regards to how it interacts with the Processing Unit. Figure 23 shows Final PCB design with highlights. Multiplexer on underside of PCB. (Original image at Figure 1) 1. Rail splitter (Outputs half supply voltage) 2. AD8236 micropower Instrumentation Amplifier 20 3. MCP42100 100kOhm 256steps SPI digipot 4. Voltage divider for battery sensing 5. Haptic Motor NMOS Driver 6. Atmega32U4 AVR microprocessor with USB support 7. RN42 Bluetooth Serial module 25 8. Lithium Ion battery charger 9. ZIF Socket 10. Underside(not shown) CD74HC4067 16 channel analog multiplexer Figure 24 shows the LTC4080 Typical Application Schematic Power System 16 Positional feedback device The device's power system was developed around the LTC4080 IC. It is a 500mA stand alone lithium ion battery charger with 300mA synchronised buck converter. It was decided to adopt the typical application schematic (Figure 24) but modified to suit this example's requirements. The ratio of the feedback resistance was changed. R2 in Figure 24 was reduced to 315.78kOhms. The 5 buck circuit is triggered whenever the voltage drops below 0.8V at the feedback pin. Through a voltage divider place between the feedback pin and Vout, the ratio will allow for 3.3V at output to be seen as approx 0.8V to the feedback pin. O 8V = Vout * 3 O V out = 0.8 * 315,78-0*0 315.78 Vout = 3,33 V rounded to two decimal placed Refer to Figure 71 for the schematic for the power system. 10 Pads for a fallback LDO and battery charger were placed on the PCB. Ultimately the LTC4080 did not work with the battery, until an extra diode was included. The diode was needed due to an oversight in the PCB design where EN BUCK was wired to USB power rather than the lithium battery. This meant that the buck was only functioning when connected to USB power, which defeats the purpose of incorporating it with a battery. Through the diode the EN BUCK at Vin can still be powered from the 15 battery, but current from the USB is protected from charging the lithium ion cell directly (preventing a potential fire hazard). Measurement of the entire system power consumption from the final board. System Current is Status Sys Current Lipo (mA) Lipo (mA) Draw (mA) (Via Buck) (Via LDO) 1st 5 see 30 293 46 Discovery 49(Pulscd) 5O(Pulsed) 56(Pulsed) Unconnected 12.8 13.1 17.6 18.0 19.3 19.7 Connected & Transmitting 38.1 38.6 40,9 41.5 42.5 The LTC4080 also has a burst mode, which would be useful if system uses less than 1 OmA, at a cost of 20 increased rippling. (Buck Efficiency vs Load Current plot ). While the corresponding PCB tracks for burst mode was routed, the system never drops below 1 OmA, so this feature was not implemented. The processing unit is currently designed for a 1 OOOmAh Li-Po from SparkFun. Assuming continuous Bluetooth transmission battery life can be calculated. 17 Positional feedback device dein c ni barter y a c m 24.A h|consumpi A} The runtime is an estimate. The difference in run time between the Buck and the LDO, is approximately an hour. The LDO is favoured due to cost and size. Areas to improve upon in current usage is the issue of data transmission rate. There is a correlation between Bluetooth transmission fequency and power 5 consumption. As the user will not require real data, until when viewing the display directly, battery life could be improved by slowing down the flex sensor reading to once a minute. Also due to time constraints, sleep mode was not implemented, so the micro-controller is always actively wasting power. Figure 25 shows This opAmp is configured as a resistance to voltage converter. Requires negative Vref for positive output. 10 Low battery sensing Framework for low battery sensing was made by placing pads for two voltage division resistor to reduce the voltage of the lithium battery (voltage between 2.7 to under 4.5V) into a range that the ADC can sense with. Since accuracy is not important to us, two 1 MOhms resistors were used as sensing voltage divider ratio. This will reduce the voltage of the lithium battery to half, which is between 4.5/2 = 2.35V 15 and 2.7/2=1.35V. Signal Processing and Amplification Since the observable changes in the printed flex sensor is very small signal require amplification before data can be acquired. This includes designs such as the resistance to voltage amplifier, shown in Figure 25. In addition, the ADC signal input in the Atmega32U4 in the final design is configured as a single 20 sided ADC input from ground to VCC. Moving from a single sided to differential amplifier design, a Wheatstone (half bridge) circuit design was adopted. A Wheatstone bridge measures the difference in voltage via an instrumentation amplifier. In a half bridge design, one half of the bridge is a pair of flex sensor, and the other half of the bridge is a voltage divider providing the rest voltage reference. The rest voltage reference is the voltage seen by the 25 flex sensors when straight. Figure 26 shows the schematic of a half bridge Wheatstone and an instrumentation amplifier A test circuit was built to assess this concept. A breadboard was wired to the design in Figure 26, where the rest reference voltage is represented by a hand adjustable potentiometer. 18 Positional feedback device Instrumentation Amplifier Choice Initially INA1 26 was chosen as the first instrumentation amplifier, from the datasheet there is a 0.9V voltage drop from the upper and lower rails. This means the range of swing the ADC will see in a 3.3V system is only 5 1.50002V = 3.3V - 0.9V * 2 It is desirable to utilise full range of the ADC and not lose 1.8V in sensing range. There was an investigation into the viability of building an instrumentation amplifier out of LM324N. The LM324N consists of 4 opAmps with an output swing of OV to 1.5V of upper supply rail voltage. It was wired as an 10 instrumentation amplifier as shown in Figure 27. As a workaround on the issue of upper rail voltage drop, the voltage supplied to the LM324N was at least 2V higher than voltage supplied to rail splitter (detailed in section 3.2.3) and the half Wheatstone bridge. 5V was supplied to LM324N and 3.3V was supplied to the sensors as well as ADC AREF(ADC voltage reference). While the LM324N can be configured as an instrumentaton amplifier and a higher supply voltage, it was 15 ultimately dropped in favour of Analog Devices's AD8236 shown in Figure 27. This is because using the LM324N would require supplying two seperate voltages, one for the LM324, and another voltage to power the flex sensors. The AD8236 has rail to rail voltage swing. Secondly the SOIC chip is smaller than the LM324N. Since the AD8236 is an integrated instrumentation amplifier, as opposed to the LM324N configeration, less discrete components are needed. Resulting in a smaller PCB. 20 All the amplifier designs trialed used a single resistor to set the gain of the device. The AD8236 is similar in nature, the breakout in Figure 28 shows the various resistance and the approximate gain that can be derived. To find the exact values needed to get a particular gain value, Where RG is gain resistance, and G is the desired Gain value. Note there is a maximum gain of 200. Figure 29 shows the MCP42XX Digipot Pinout. Raz =I~ k()hmi 25 Digital Potentiometer The primary role of the digital potentiometer in this design is to set the Voltage Reference as well as the Gain of the AD8236 Instrumentation Amplifier. The MCP421 00 was chosen for this job, being a 30 1 OOkOhms digital potentiometer with two internal separately controllable potentiometer. To learn more 19 Positional feedback device about controlling it over SPI, refer to section 5.4.3. The MCP421 00 stores a value between 0-255 for each potentiometer. This means the theoretical resolution for this potentiometer is: '06 hmStep = Mk hms The reason a 1 OOkOhm digital potentiometer was chosen is because the range of gain desired was 5 between the gain of 10 to 200. This is since most flex sensors we tested had a typical swing around 0.1 to 0.3V. This requires gain of 33 and 11 respectively, to which a 1 OOkOhm digital potentiometer setting the gain, will be able to easily swing from gain of 10 to 200. This gives us plenty of leeway to eventually move to future sensors like real strain gauges with minimal modification to the circuit. For example in National Instruments white paper "Measuring Strain with Strain Gages", there was a 10 mention that strain gauges typically outputs less than 1 0mV N (per excitation voltage). This means for 3.3V, it can be expected to see around 3.3V * 0.01 = 0.033V. To amplify such signals from these strain gauges(For a 3.3V system), the gain should be set to: 100 = 3.3/0.033 This should correspond to a gain resistance of 4.42K. And given the digipot has a step resolution of 15 0.39kOhm, the gain resistance of 4.42K can be reached by setting the digipot resistance register to 11 = 4.42/0.39. The other purpose of the digital potentiometer is to act as a voltage reference of the flex sensor at rest for the instrumentation amplifier to compare against. As a voltage divider the digipot can split 3.3V into 256 steps for a step resolution of 0.01 288V per step. 20 Since in the digipot datasheet the typical "Full Scale Error" and "Zero Scale Error" for 'V+ = 3V' is typically at 0.35 of LSB which stands for LSB = V +/256 where V+ is the voltage supplied to the voltage divider. Thus LSB is the voltage step resolution calculated previously. Hence for 3.3V we can expect an error range of about: + - 0 -0045 = U35 * LSB = 0,35 M error 25 The digipot is set to 0.01 28V * 128 = 1.6384V , the real value is expected to be between 1.638437V and 1.638437V. Since the voltage changes in this example's printed flex sensors is typically no more than 0.1V, this error is well below the error margin. Of course this will be a different matter if dealing with strain sensors that require more accuracy. Figure 30 shows the Voltage Splitter TLE2426 Schematic 20 Positional feedback device Rail splitter and virtual ground As seen in Figure 36 the instrumentation amplifier requires a ground voltage reference. Since the Atmega32U4 ADC can only take between ground to the AVCC AREF, it means it cannot detect any negative voltage that would usually be seen if we set the instrumentation amplifier voltage to OV. Setting 5 the instrumentation amplifier to this new virtual ground. Allows sensing of negative voltage, as well as positive voltage. The TLE2426 provides a high precision virtual ground operated at typically 170uA with 5V input. This device output precisely half of the input voltage. The noise reduction pin was connected to ground via a 1 uF capacitor for further accuracy. 10 The rail splitter acts as a voltage divider where V out = V in / 2. Since the accuracy of the voltage divider is based on the ratio of the two resistances, most rail splitters are limited by a tightly controlled resistance tolerance. An OpAmp config as buffer can be seen in Figure 30. While the TLE2426 is designed to in-between 4V and 40V (As seen in Figure 31), it works still when supplied with 3.3V. This is most likely due to the low current draw of the instrumentation amplifier's 15 unbuffered ground reference input pin. Multiplexer The first design initially went for a 74HC4051 (8 channel multiplexer, 3 data/control lines), but later went for CD74HC4067 from Texas Instruments (16 chn mux, 4 data/- control lines) due to a need for more flex sensors. 20 Refer to 5.6 for commanding the analog multiplexer to switch channels. External Communication Bluetooth In this example, Bluetooth was chosen over other wireless technologies such as ANT+ or WiFI due to widespread adoption for the standard Bluetooth protocol amongst the current smartphone user base 25 compared to ANT+, as well as the relative low energy usage. We initially chose the HC-05, but switched to the RN42, both Bluetooth 2.0 module, further reasoning can be found in section 5.3. Both modules can be interfaced to MCU by serial UART (Explained in 5.2). Expansion (12C) and Additional Features Accelerometer 30 In one implementation the MPU6050 is used. It has a motion processing unit to merge and fuse acquired data from both sensors, for greater accuracy. A custom breakout board for the ADXL345 was created 21 Positional feedback device (Figure 32) which is a LGA (Land Grid Array) packaged accelerometer that had to be reflowed with a hot air gun. Once the ADXL345 was seated correctly, it hooks up to the 12C line. Its communication to the main processing unit is detailed below. Haptic Feedback 5 Haptic feedback was accomplished by an NMOS switch for a 3V haptic motor, a voltage divider is used to restrict the maximum current the motor receives, preventing burnouts. Also a diode is placed in reverse to power supply, to protect the circuitry from the vibrational motor's inductive kickback when shutting the NMOS gate. Refer to schematic at Figure 75 for schematic of the driver. Firmware Design 10 Firmwares is the code that is executed from the internal flash instruction memory of the Processing Unit's microprocessor. The microprocessor is based on the SparkFun Pro Micro 3.3V/8MHz, an enhancement of the Arduino Pro Micro 16MHz. The Atmega32U4 micro-controller contains an internal USB controller, used by the Arduino IDE for serial communication via USB. Overall structure of the firmware Arduino sketch is represented in Figure 33 which shows all the major components and 15 subsystem running inside the device. Scheduler Older versions of code consisted of an infinite while loop that performs a sensor read and then uses the Arduino delay() function to provide the interval between readings. The inclusion of the ability to wirelessly command the device to vibrate via the PWM sequencer required the use of a scheduler loop. 20 This allowed the system multitask to a certain extent, by periodically checking and then running a particular function or routine after a certain preset amount of time has elapsed. Unlike the original delay( architecture, this has the advantage of preventing stalling other functions while a routine is waiting for the next round to run again. Figure 34 shows how the scheduler code operates when compared to a delay( based flow. In the delay 25 based flow, the time taken for the loop to execute is always INTERVAL * 2. With the scheduler, the delay is minimal, only running routine functions when needed. The inclusion of the scheduler improves the responsiveness of the device. Sensor Reading and Reporting Incoming voltage signal from the flex sensor is defined to be at rest, when the signal is at half the ADC 30 voltage reference from the Instrumentation Amplifier's output. As seen in Figure 22, a sensor read routine will need to sequentially control both the multiplexer and the two digital potentiometer in order to effectively read all the flex sensors attached to the system. 22 Positional feedback device The example's firmware performs the reading and reporting of flex sensors in the device through a set of functions shown as follows; int readSensor (int selectSensor) 5 This function outputs a selected sensor value after setting the digipots, multiplexer as well as applying the offset to the ADC sensor reading. Results from the function can be expected from -512 to 512, where the maximum ADC stepsize is 1024. If the ADC reads 512, then the output of readSensor should be 0, since the sensor is defined to be at rest when the instrumentation amplifier is at midpoint of ADC reference voltage. 10 To understand what steps is taken in selecting and recording a flex sensor, refer to Figure 35 Take note that in this function, setMuxO uses two arrays to control the map- ping of parameter selectSensor to the multiplexer's pin channels. The mapping is cascaded as g_FlexConnectorMapping[ gFlexStripMapping[selectSensor] ] and is used because the multiplexer pins on the PCB does not correspond to the ZIF socket on a one to one manner. In addition on the strip itself, the ZIF traces does 15 not correspond to the sensors arrangement on the strip on a one to one basis. The reason for this, is due to the need to avoid overlapping traces in the PCB and the flexible strip. Also by keeping the mapping as two separate cascaded arrays, it allows for future modifications of flex sensor layout with minimal modifications to source code. void readAllSensors(int *sensorsBuff) 20 This function uses a foro loop to sequentially iterate from the first flex sensor [readSensor(O)] to the last flex sensor [readSensor(TOTALMUXCHANNELS - 1)]. From the array pointer given in the function parameter sensorsBuff, the results from each readSensoro readings are inputted to its's corresponding position in the sen- sorsBuff[] array to return as the result. void readSensorsDetailso 25 This function checks the number of flex sensors and accelerometers enabled or de- tected, and prints it in JSON format for the phone app to use. It allows the phone app to know what to expect from the device and modify it's behaviour and display to suit the incoming data. void readSensorsAsJSONo This retrieves the latest values from readAllSensorso and converts the integer range of readSensoro ( 30 512 to 512) into a floating point representation of degrees. It also retrieves the accelerometer values if possible from the accelerometers on the 12C lines. Using the accelerometer raw x,y,z values, it converts 23 Positional feedback device these values into degrees for overall side to side as well as forward to backward bending. These values once via BT sent as a JSON string to the smartphone application. Accelerometer Processing Reading accelerometer values occurs in readSensorAsJSON(, for more information about the specifics 5 of connecting to the accelerometer refer to section 5.5 The ADXL345 outputs three raw values x,y,z. These represents accelerations along the x,y,z axis. Since gravity is always constant towards the ground, it can be assumed that the readings on the accelerometers are component vectors pointing downwards. 10 The angle of interest for this example is the side to side flexing of the back which couldn't be done on flex sensors alone. This processing unit supports up to two accelerometers acting as tilt sensors. Two accelerometers are used, so that the actual overall curvature of the back can be obtained at any orientation of the body. This is by taking the angle from the accelerometer readings from the top, and subtracting from the Accelerometer reading from the bottom (Figure 36). 15 During testing, the ADXL345 breakout was working as expected for side to side motion, however data was inconsistant with forwards and backwards bending. This is due to an oversight, when calculating the side to side degrees, the original calculations took only two axes, when all 3 axis should be considered, e.g, x will not just decrease when bending side to side, but also forwards and backwards. Figure 35 shows Functional flowchart of readSensoro. Figure 37 Shows how the component vector a.b relates to c 20 in terms of getting the angle e in a consistent manner // Original angle calculation. Faulty due to 2D thinking. pitch = atan2( z, x); roll = atan2( y, x); Pythagoras's theorem ( a2 + b2 = c2 ) should have been used to find the component vector of X vs Y for 25 Z or X vs Z for Y. Figure 37 shows how this concept can be spatially represented. // Amended calculations to take an extra dimension into account float xy,zy; xy = sqrt( x*x + y*y ); zy = sqrt( x*x + z*z Calibration The calibration function in this firmware declared in the source as void calibrateSensoro , mainly 30 concerns itself with finding the right value for the reference voltage digipot; such that the voltage difference between the positive and negative input of the instrumen- tation amplifier, is OV when the flex 24 Positional feedback device sensor is bent to the same angle it was calibrated at (Refer to flowchart at Figure 38 for see how this function works.) This function works by sweeping the reference voltage digipot towards centerpoint of the flex sensor using readSensoro as reference. Ref voltage incremented to match read sensor. This process is 5 repeated, until both match up to 3 times or the maximum no. of attempt is reached (cannot calibrate). A rather convenient effect is that it is unlikely for this system to be able to lock on to a floating output. Thus we can use the effect of floating output when cutting the strip to detect the number of valid sensors. Alternatively a small pulldown on the sensor strip will allow for a default low voltage. After a suitable reference digipot setting is found, the flex sensor setting is tested 3 times and the output 10 is averaged. The average of 3 hardware calibrated flex sensor value for that particular flex sensor is used as a software offset for the strip in readSensoro. The reason for this, is to make up for any hardware steady-state errors or offset that cannot be accounted for by the reference digipot, this is since the digipot resolution is only 0.01 V (3.3V / 255 = 0.01294). The flex sensor output variation from ADC reference voltage midpoint can often be smaller than 0.01V, and the amplifier can still amplify this 15 difference. On average it takes about 10 seconds to lock on to a sensing tape, but varies depending on strip construction or delays between each trial. There are strategies to speeding up the process. One of which, is to use a proportional control system, as opposed to an incremental approach to calibration. This speed the process up by reducing the number of steps required to reach equilibrium, since a 20 proportional control system will initially make large increments before gradually reducing it as it gets closer to the centerpoint. Due to time constraints, the simple incremental approach used from half of AVCC AREF as an initial guess. Most of the time the flex sensors will be around the midpoint of the ADC reference voltage, setting the digipot to midpoint reduces the numbers of iteration before calibration. On the issue of improving accuracy, an approach could be to place two resistors on the upper and lower 25 pins of the digital potentiometer, this cost the reference digipot its full range from 0V to VCC, but gains in accuracy inbetween. Alternatively consideration could be given to switching to a higher resolution digipot or a digital to analog converter (Since high resolution digital potentiometer can reach up to 1 Obits, but a high end DAC can reach up to 24bits.) Figure 38 shows the Functional flowchart of calibrateSensoro. Figure 39 shows the Overall Mapping of Atmega32U4 SRAM 30 Ram Issues The Atmega32U4 contains 2.5KB of SRAM memory. Initially, code size was not an issue. However code density increased, random crashes started to occur. We know there are similar processors on the market with a slightly high price range which could solve the memory constraints. 25 Positional feedback device Due to the choice of the Arduino platform, the debugging efforts were restricted to placing serial print statements everywhere. Suspecting overflowing RAM to be an issue, a ram check function was obtained from JeeLab and inserted in the initialisation section of the firmware as well as the standard loop. Issue with memory can be seen in Figure 40. 5 Bare bone code showed that [memCheck]:2377, which is a big contrast to the 109 bytes of memory during the initialisation phase of the firmware. This indicates that the At- mega32U4 after the Arduino overhead, would give us only 2.377kB of memory. This matches up with Jeelabs article,"The trick is to keep RAM usage low, because its a scarce resource: an ATmega has a mere 2048 bytes of RAM." which is equivalent to 2.047kB of memory. With this in mind, the firmware code were modified to reduce 10 the number of strings that needs to be loaded to RAM before serial transmission. Referring to Figure 40, as variables are added and declared, the heap grows in size in the SRAM towards Ox1 100. At the same time, as more function calls are given, the stack pointer also grows in size towards OxO1 00. In this example it was theorised that the wasteful use of print statements had likely caused the gap between the heap and the stack to grow too small that a stack- heap collision had 15 occurred. This means that there is a possibility that either the heap or the stack had been partially overwritten, or corrupted. In the case of the heap, it means the scheduler will take an undefined amount of time to trigger the next interval. For the stack, returning to an address referenced from the top of a corrupted stack, would likely jump into a random instruction space instead of the original position the function was invoked from. 20 Annotated (via '>>') typical symptom of a firmware crash. freeRamo inserted to monitor amount of free memory space >>> 'stands for omitted outputs "fBA":[94.22,98.61,110.92,68.38,-91.23,-78.93,-104.06,... 25 @SYSTEM STARTED@ "freeRam":[ 09], "freeRam":[438], { "fBA":[-71.89,-71.72,-68.91,-111.45,-125.51,-100.55,-1... "freeRam":[427], 30 >> Program Crashed at this point Code snippet from JeeLab, which measures amount of free RAM 26 Positional feedback device int freeRam ( extern int __heapstart, *_brkval; int v; return (int) &v - (_brkval == 0 ? (int) &_heapstart : (int) __brkval); 5 1 Protocols And Communications This section involves the interaction between the processing unit's ICs, as well as how it communicates with both accelerometers and Bluetooth module. Our hardware uses a range of components which have their own requirements. We will detail how each 10 method of communication works as well as how we interfaced to each particular chips to each other, and the supporting code required to make each work. USB USB is a highly popular connectivity standard for computers and devices that defines a physical connector, power, and protocol standards. 15 Physical Connector An important aspect of the USB standard is to have a consistent physical interface or port, so that there is a reliable way to connect the device to many other types of devices. In this example we will be using the ubiquitous micro USB port which is found in many consumer devices. Below is the standard USB pin configuration 20 Pin Name Cable Color Description 1 Vcc Red +5 VDC 2 D- White Data 3 D+ Green Data + 4 OTG ID None OTG Slave/Host Select 25 5 Gnd Black Signal/Power Ground USB Power 27 Positional feedback device The main selling feature of USB is that it is one of the few standards that carry power along with signalling, unlike the older serial and parallel ports. According to the USB standard, only 5V can be supplied A major reason for choosing micro USB versus the barrel plug or other standards is the ubiquity of USB 5 charging. This essentially means that our users will be unlikely to ever be out of a port to charge their devices. This reduction in propreitary sockets reduce the number of E-waste. USB Data Signalling USB signaling is via a unidirectional balanced pair link between two cable at half duplex, with a master/slave architecture. 10 While we chose micro USB for easy access to power, what also influenced our decision was that our development platform of choice, Arduino, has the capability to uploads new code and serial console access over USB. These feature allows for rapid prototyping by removing the need for an external programmer beyond the loading the Arduino bootloader. In addition serial access over USB would also allow for future upgrades for end users, to enable access to new accessories on the 12C bus or to fix 15 bugs within the firmware. UART and USART USART stands for Universal Synchronous/Asynchronous Receiver/Transmitter. UART is Universal Asynchronous Receiver/Transmitter. A distinct difference between this com- munication standard and other wired standards like 12C or SPI, is that USART and UART does not require a dedicated clock 20 signal to signal the arrival of a new bit. Instead the clock is determined by the speed of the data transmission itself. This speed is deter- mined by the matching baud rate that is set separately at each end of the connection, any mismatch in baud settings could lead to data corruption. USART defines a receive pin as RX and transmit pin as TX. Thus for full duplex com- munication, you need to connect the TX of one device to the other device RX pin, and vice versa. Refer to Figure 42 for 25 an example of the connection arrangements. The Atmega32U4, has a peripheral feature described as 'Programmable Serial USART with Hardware Flow Control'. USART also has the capability of transmitting a clock signal as master or receiving a clock signal as slave, this synchronous mode allows for faster transmission compared to asynchronous mode. Since transmission speed is minor, the USART in this example is configured to operate in asynchronous 30 mode only. The example's device micro controller is connected by USART to an external Bluetooth module. The exact details of how UART works is not important for the completion of this objective, as it has been abstracted away by the Atmega32U4. What is important is that the voltage output of the TX corresponds 28 Positional feedback device to the rated input of the other devices RX. Overlooking this requirement runs the risk of burning the RX input of the UART device on the other end. The baud rate settings, parity, and flow control mode should be the same on both sides. EWo3-IP device communicates over USART in 9600 baud speed with no hardware flow control. 5 In Arduino, the baud rate for Serial object is set by this function Serial.begin( BAUDRATE), where BAUDRATE would be typically set to a value of 9600. It is important when you select a higher baud rate, that the oscillator on both ends are accurate enough. This means it is not possible to use the internal RC oscillator of the Atmega32U4 without calibration. 10 To send a string of characters over USART via Arduino, type: >> Serial.print(hello); // Prints a string over serial >> Serial.print(hello); // Prints with new line char Bluetooth Bluetooth is a wireless communication technologies for short range personal area network- ing with local 15 low powered electronic devices. It is often recognised for its usage as mobile phone wireless headset. In this example, Bluetooth technologies is used as a convenient way to connect the process- ing unit to the smartphone. There are two versions of Bluetooth as of 2013, Bluetooth Standard, and Bluetooth Low Energy (BLE). BLE will be the primary choice in the com- ing years, despite its lower data transmission capability compared to standard Bluetooth, due to its lower power consumption and ease for the user to 20 connect to such devices. However even though BLE is the better choice for this application, this protocol has no default Serial Profile (SPP) that is found in standard Bluetooth, and our priority lay higher for a simple and reliable communication protocol as opposed to a device with longer run-time (due to better energy efficiency). Roving Networks RN42 25 The RN42 is a Low power Bluetooth transceiver with 26 uA sleep, 3 mA connected, 30 mA on transmit mode. This is marked improvement from the HC-05 standby connected current of 8mA at minimum, to the average current of 25mA. The RN42 uses a different baud rate of 115200 which is extremely fast. To reduce the baud rate, the easiest method to do this is to set the pin 2 (which corresponds to GPIO7)for the RN42 to force the 30 board to function at 9600 baud. However this come at a cost of future flexibility of allowing for software reconfiguration of the baud rate to a higher level such as 115200. 29 Positional feedback device Figure 43 shows the Pinout according to the RN42 data sheet, as well a flatbed scanned underside dimensions of the pads Second method is to add code which upon reset of RN42 (which will provide us with a 60 seconds window for getting into configuration mode), will send configuration commands to the RN42 in a non 5 permanent way. The last method is to manually enter configuration mode like the second method, but instead of issuing a temporary command, a persistent SET command is available. This was what was used for the final PCB due to technical difficulties in getting temporary settings to work. Figure 44 Shows how SPI consist of a ring of shift registers with clock and chip select. 10 Command set references SU,<rate > Baud rate, 1200, 2400, 4800, 9600, 19.2, 28.8, 38.4, 57.6, 115K, 230K, 460K, 921K, only the first 2 characters are needed. EXAMPLE: SU,57 sets the baud rate to 57600 baud. SU,96 is what we used to set our breakout to 9600 15 S-, <name > Serialised Friendly Name of the device, 15 characters maximum. This command will automatically append the last 2 bytes of the Bluetooth MAC address to the name. Useful for generating a custom name with unique numbering. Example: S-,MyDevice will set the name to MyDevice-ABCD these are the two command most likely sought after, if seeking to persistently set the RN42 for a particular application. 20 SPI (Serial Peripheral Interface) An SPI Bus is a simple serial interface designed for communication between ICs, such as a microcontroller to a digital potentiometer. The diagram drawn in figure 44, illustrates a generic view of most SPI devices. The actual internals circuit and how each device reacts to Chip Select CS# may differ. It is thus very important to consult the 25 datasheet for exact specification of how to communicate to each chip. The processing unit digipot supports an SPI interface. It is best visualised via the above diagram for reference, as two shift registers connected as a ring memory structure. To transfer a byte from master to slave, each bit is sequentially shifted from one set of shift register to another shift register on each CLK(Clock) cycle until all the bits are in the slaves shift register. 30 For this particular digipot (MCP421 00 dual 1 OOkOhm SPI digipot), it uses the Chip Select pin to determine when to execute the next command. To which the digipot will set its resistance only after 30 Positional feedback device shifting all bits from the microcontroller to the digipots shift register and then having its CS# pin go high to let the digipot know that its not to drive the MISO line anymore (The MISO line will be tri-stated so that the bus is free for other SPI devices to use again). CS# : chip select 5 Lets the slave chip knows if its being talked to. The slave MISO pin is disconnected upon CS pulled high, CLK: Clock Clock signal to indicate that the next bit is ready to be shifted in MOSI : Master out Slave In 10 This pin allows a slave chip to receive bits shifted into it, on very CLK edge (Rise or fall depends on spec) MISO : Master in Slave out This pin allows for the slave to shift bits out towards master, on very CLK edge. It is quite possible to leave out MISO, if the IC in question. But only if the master is never ever expecting 15 to hear a reply from the slave. This is the choice we made for our SPI communication to the digipot which is merely an output device and thus we had no need to know of its current state. Addressing multiple chips in SPI: Below are two most common methods of interfacing with SPI. There is also a third method called mSPI. Chip Select 20 In this scheme, all the SPI slaves are connected to a common SPI BUS (CLK, MOSI, MISO), and each slave is addressed by a separate Chip Select line from master as shown in Figure 45. More than 4 slaves would require a digital multiplexer (with negated output since CS# is active low) where the first pin is not connected so that all slaves can be disconnected. Daisy Chaining 25 The SPI Chip Select is not the only way to address various ICs. If time is not as critical of a factor, and PCB routing space is at a premium, there is an option of daisy chaining the MISO of one chip to anothers MOSI input as seen in Figure 46. This does mean that if each slave shift register is 8bit long, then adding additional slaves to the SPI line would in theory add an additional linear amount of time (Also represented as O(N) time [Further Readings on time complexity concept: 30 en.wikipedia.org/wiki/Time-complexity ] ). 31 Positional feedback device MCP42100 The MCP42100 dual 1OOkOhm digipot is an SPI device that has its own command speci- fication shown in its datasheet. Commands to the digipot takes the form of two bytes (byte = 8bits). As shown in Figure 47 first byte represents the command to be executed (Command Byte), and the second byte represents 5 the value to be used in a command (Data Byte). Figure 48 points out that in addition to a write command, it is possible to have a shutdown command which will disconnect a digipot channels pinA, and short B and C together. These are the Digipot write command in binary. Of which these combinations can be made: Description : Command Select Pot Select rite to pot 1: 0001 0001 Write to pot 2 : 000: 1 0010 Write to pot 1 and 2 : 000 1 0011 10 Communicating to the MCP421 00 via bit banging While researching on how to communicate to the MCP421 00, it was initially decided to avoid the use of the SPI bus. This was since there was enough extra pins for MOSI, CLK, and CS#, and there was a desire to avoid any potential conflict with the external ICSP programmer when first loading the Arduino 15 bootloader. This was accomplished by bit banging. Communicating to the MCP421 00 via Arduino SPI.h Later on due to space constraint in PCB, we decided to use the SPI.h library included with the Arduino IDE for communicating with SPI devices. The benefit of this approach is that the library utilises the internal SPI core of the microcontroller speeding up communication to the digipot as the SPI core will 20 autonomously send any bytes in its buffer. To help keep the Arduino sketch clean and for future extensibility, the digipot code were collated into a library. When using the Arduino SPI.h, you should ideally set it up before usage. First of which SPI.setBitOrder(MSBFIRST) should be set, which indicate that the most significant bit of the byte should 25 be sent first. LSBFIRST replaces MSBFIRST if least significant bit is required to be sent first in some SPI design. 32 Positional feedback device Next that should be set is the mode of the SPI, which indicate the clock polarity and the clock phase the micro-controller should account for when speaking to the slave over SPI. Clock Polarity is what state the CLK should be when idle. Clock Phase is whether Data is latched on the rising edge or falling edge of CLK. |MODE : Clock Polarity (CPOL) | Clock Phase (CPHA) SPI MODE0 0 0 SPI MODEl : 0 1 SPI_MODE2 : 0 SPI MODE3 Note: CPOL: = 'low idle' ; I 'high idle' CPHA: 0 'rising edge' ; I 'falling edge' 5 Reference Source: {34] Referring to timing diagram in Figure 47 and comparing against Figure 49, it would be best to select positive Clock Polarity and the positive clock edge for Clock Phase: SPI.setDataMode(SPI MODEO); Sending a byte is conducted by: SPI.transfer(byte); //Where byte is the byte to be sent to slave 10 Figure 50 shows the typical wiring of an 12C device. Take note that there is a mandatory need for pullup resistors for SCL and SDL. 12C (Inter Circuit Communication) 12C is a proprietary peripheral bus by a company called Philips and the standard consist of two lines, one clock and one data. The advantage of this setup is that the data-line is bidirectional, clocked and 15 bussed. The synchronous nature of the standard means the complexity of the device 12C handler is simplified compared to UART/USART. Clocked data is not dependent on timing and thus require less logic. The bidirectionally and the lack of chip select compared to SPI means decreased PCB complexity, due to reduced numbers of wires to route. There is also an advantage of wide support and selections of 12C devices in the market, making this near plug and play in engineering terms. 33 Positional feedback device ADXL345 12C Communication The posture sensing device uses 12C to access the 12C accelerometer ADXL345. A library based on the Wire.h Arduino standard library was created, which was helpful in the ability to read from two accelerometers. To assist us in writing this code, we referred to the ADXL345 datasheet. 5 Key points in ADXL345.cpp library: Wire.beginTransmission(i2cAddress) Setups a connection to 12C address. The 12C address would refer to the address of the accelerometer. Wire.endTransmissiono Tells Arduino to end the connection, and also shows what variables its at. writeToReg( DATA FORMAT, OxO1) Set range to -+4g. Refer to Figure 51 for the corresponding bit 10 position, and it's meaning. writeToReg( POWER CTL, Ox08) Turns on measure bit Setting the measure bit in POWER CTL to 1 to enable measure mode "A setting of 0 means standby mode", "1 means measurement mode". ADXL345 is on standby mode by default. Refer to Figure 52 for the corresponding bit position, and it's meaning. 15 readFromReg( byte(Ox32) , 6 , accBuff )Set Reg Pointer to 0x32, and read 6 bytes to Byte array at accBuff (Aka: pointer to accBuff[O] ). It possible to get all 3 integers values of x, y, z from one call, since the results Registers in Figure 51 shows that retrieving bytes sequentially between Ox32 to 0x37 will reach all Data 1 and Data 0 results register for all 3 axis. Figure 52 shows ADXL345 POWER CTL register from 20 Figure 53 shows the ADXL345 DATA FORMAT register from Figure 54 shows SO to S3 used as multiplexer channel selection. Parallel Signalling Parallel signalling is used for controlling the CD4097 1 6channel analog multiplexer used in the final PCB. A 4 bit signal is sent along the control lines SO,S1,S2,S3 as seen in Figure 54, where SO is the 25 least significant byte (LSB), and S3 is the most significant byte (MSB). With 4 control lines, 16 channels can be controlled. Changes in control line, leads to near instantaneous changes in multiplexer channels. The code to controlling the multiplexer - This function breaks a channel selection value into its corresponding bits to switch the control line of the multiplexer. Phone Application 30 This section focus on smartphone with Bluetooth and the smartphone application. 34 Positional feedback device The example involves creating a consumer product. In order to decrease costs while also spread adoption, it is important to have the device information be easily displayed to the user, the everyday smartphone instead of a dedicated display device. Figure 55 shows the Wireframe draft art of the User Interface for the PhoneGap Application. Left: Front 5 back flex. Right: Side to side flex The smartphone application will eventually cover both dominant smartphone ecosystems, Android and iOS. The ecosystem chosen for development was Android, mainly due to no monetary barriers to entry. iOS requires enrolling in the MFi program to use Bluetooth 2.0. Mobile development 10 1. Connect to processing unit via Bluetooth 2. Reconstruct and display a spine curve from sensor data 3. Save the user's optimal position 4. Provide both visual and haptic feedback when spine is not in an optimal position The following development tools were used 15 Native Android SDK uses Java has a steep learning curve, but has the most features and flexibility. PhoneGap (Apache Cordova) is by Adobe, a cross platform development tool using HTML5 and JavaScript. Bluetooth plugins are available, including device specific features such as Android hardware button support and NFC. Icenium (Apache Cordova) is also a variant built upon Cordova. Icenium is compat- ible with PhoneGap 20 and its plugins. Unity's cross platform 3D engine is commonly used for game development. There is no preexisting Bluetooth library. Corona Labs SDK is cross platform that uses LUA on top of C++ and OpenGL. Blue- tooth support is hazy. 25 Appcelerator Titanium SDK is an independent HTML and JavaScript based plat- form. The Bluetooth serial plugin is developed by a third party, and requires licens- ing in the form of seats. Although a limited demo is available, there are additional licensing costs for full version. Selection criteria in order of importance 1. Bluetooth serial support 35 Positional feedback device 2. Ease of use, but flexible with good documentation 3. Cost to entry 4. Programming language Biuetooth Programming Documentation Cost to develop (serial) support language Native Yes 3 Java Android Free PhoneGap Yes IHTML/JS Free leenium Yes 3 HTML/JS Free Unity No 2 C#/C+-+- Free Corona Labs SDK No 2 LUA Free Appeenrtor Free+Bluetooth Yes 2 JS/XML Titanini SDK serial licensing PhoneGap PhoneGap was chosen in this example as the development platform for the prototyping. Post prototyping the phone apps will be built in their native platforms eg. iOS and Android. The main reasons for PhoneGap were the amount of accessible documentation and existing support for the Bluetooth 10 serial profile. HTML and JavaScript are both very powerful visual languages despite being less syntax strict. Through the increase in web adoption, JavaScript engines are now comparable in speed compared to native code. Figure 56 shows The PhoneGap architecture. PhoneGap consists of phonegap.js, a JavaScript library that acts as the interpreter between the application's JavaScript and native OS Java. Any functionality beyond displaying the application such 15 as hardware button support or Bluetooth is done via PhoneGap plugins. Smartphone Application The entire application is contained within one HTML file, with references to JavaScript and styling libraries. Because the application is a web page, all manipulation within the HTML Document Object Model (DOM) must be done through JavaScript. The application uses elements from the latest version of 20 HTML5 and CSS3. Manipulating the DOM 36 Positional feedback device By using a JavaScript framework like jQuery for DOM manipulation, it is possible to 'chain' queries together, by running multiple queries with a single statement, allowing more functionality with less lines of code. ${element-type}.selectingElementByldentifier(.action1 ().followedByAction2(); 5 Most of the functions are English-named which along with chaining, makes self docu- mentation very easy and straightforward. Staying true to the cross platform philosophy, frameworks make OS compatibility easier to manage. The smartphone application makes use of a jQuery syntax compatible library called tt.js, a high speed implementation of selector based queries designed specifically for mobile devices. The decision was 10 based on the performance of the most commonly used jQ queries within the application, html and addClass functions. By using the jQuery syntax, any compatible library can be used as a direct drop-in replacement, such as the Intel App Framework during Endeavour (major sponsor). The tt.js library consists of two sections, TTWorker and tt.object. Both are used within the application. TTWorker 15 TTWorker is involved in selecting elements and classes, setting visual styles and DOM manipulation. tt.object tt.object processes all the non-visual functions, including parsing JSON, AJAX loading and array manipulations. Canvas element 20 Figure 58 shows A canvas element. Measurement unit is in pixels. The origin, (0,0) is on the top left. The canvas is 500 x 375 pixels. The bottom right position is (500, 375). The canvas is a graphical element container that allows manipulation through JavaScript using the HTML5 canvas API. A canvas context object is required, either 2d or webgl (3D) which contains the methods and and properties required to render graphics. Each time the canvas is resized, the contents are cleared. Animations will require 25 clearing of the canvas and redrawing on each new frame as there is no stored memory feature of existing graphics or lines present. Figure 57 show the Benchmark of popular jQuery syntax compatible libraries for html and class functions. Higher bars are better. Figure 59 shows a Comparison of javascript canvas libraries. While native JavaScript can be used to 30 control the canvas element and DOM, there are readily available canvas libraries that abstract difficult functions to make the canvas more easier to manipulate. In native JavaScript, plotting a line would 37 Positional feedback device involve a strict syntax of x and y coordinates, [40, 20], [20, 52]... compared to the use of libraries which can take the data as objects or nested arrays. PaperJS Canvas Library There is a big selection of canvas JavaScript libraries available. While JavaScript libraries provide 5 flexibility for the developer, this may sometimes come at the cost of performance. jsperf.com was used to compare the performance of different libraries. While each library has it's own implementation of a specific piece of code, it can provide a rough estimate. The library PaperJS performed consistently better than the others, with speeds compa- rable to native speed. Based on the popular Adobe Illustrator plugin Scriptographer, it has a very extensive function 10 library with detailed documentation on the example website. As a bonus, it also has powerful features such as simplifying points to save on computing power. The tradeoffs are the library is relatively new but is under active development; some basic functions such as on demand animations are yet to be developed. The use of PaperScope scoping within JavaScript which can be seen as both advantage (for experts) but disadvantage for newer developers. Figure 60 shows Javascript and PaperScript scoping. 15 Each individual canvas element is assigned its own PaperScope. By having a scoped PaperScope with PaperJS, it does not pollute the global public namespace with variables, and it also means different PaperScript code can be run simultaneously without conflicting one another, which is particularly useful with multiple canvases or parallel manipulations involving similar variable names. The scoping feature is a difficult concept. As PhoneGap uses JavaScript, it cannot directly access 20 variables in the PaperScope. In order to use PaperScript with general JavaScript functions (or JavaScript libraries like jQuery), the current PaperScope present in the global namespace/javascript space before the features of PaperJS can be used. An approach is storing each PaperScope as a JavaScript object, such as mypaper[0], mypaper... and switching, installing each PaperScope into global namespace before manipulating each canvas 25 element. The code by Zack Grossbart was posted in the PaperJS Google group. The loader.js file is a function that automates the switching of multiple canvases easily, keeping the while also keeping each paperscript in separate .pjs files for neatness. loader.js 30 This code's function checks if external the external PaperScript .pjs file and canvas is valid. If both are valid and exist, both are loaded through AJAX and attached to the selected canvas in focus. The code within the selected .pjs file is then evaluated. 38 Positional feedback device Javascript and Paperscript interoperability By default, PaperJS cannot work natively with JavaScript variables due to the scoped structure. The Javascript interoperability reference page on PaperJS' website has still yet to be completed by the developer. In order for PaperJS to work with native JavaScript, the variables must be placed into the 5 global namespace suggested by PaperJS developer Jurg Lehni as an interm solution. The variables must be assigned into the global namespace, 'window'. Not only does this expose a set of PaperScript with JavaScript, but also allows sharing of data between canvases/PaperScopes by having a central data store without passing data via the loader.js function. Figure 61 shows the JavaScript, PaperScope and window global variables (data store). 10 Touch events Prior to the smartphone, the HTML and Javascript were designed for the desktop com- puter with a mouse cursor. JavaScript only supports clicks, not taps. On a smartphone, this causes a delay in response and/or require a double tap for each button. By using a touch framework, tap and gesture responses can be recognised within the web page application at native speed for the end user. 15 Hammer.js, a jQuery syntax friendly gesture framework is used. Although not part of the jQuery set, chaining can be completed in a similar syntax style Bluetooth.Serial PhoneGap Plugin The application makes use of the Bluetooth.Serial plugin developed by Don Coleman. The plugin acts as the interpreter with the default Android Bluetooth stack, Bluedroid. Bluetooth commands are sent via 20 JavaScript and converted into native commands to the Bluetooth radio on the smartphone. Android versions from 2.0 to 4+ are supported. The application makes use of the following pieces of code bluetoothSerial.subscribe This instantiates a long running callback to run in the background, which triggers when any new data is received by the processing unit. The subscribe function manages bluetoothSerial.readUntil and the 25 read/write buffer. bluetoothSerial.write Sends text via Bluetooth. A success callback is called if successfully sent. bluetoothSerial.list Scans the area for previously paired Bluetooth devices and returns data in JSON format. This data is 30 subsequently processed into a list for the user to select bluetoothSerial.connect/disconnect 39 Positional feedback device Allows connection/disconnection with the selected device MAC address. Application Functions Due to the limitations of scoping with PaperJS, there are 4 canvas elements split by direction (front and side) to allow for simultaneous canvas drawing and manipulations, eg: updating one curve/canvas will 5 not mean redrawing the others. As each curve is updated with new data, PaperJS redraws the entire canvas. This is processed on the fly at a high frequency. The app has the option of calibrating to the optimal spine of the end user, and detect any deviations from this position. The optimal posture reference is updated only once or twice each time you use the application, thus there is no need to continuously redraw with each data update, reducing the amount of 10 processing by half for each direc- tion. The optimal posture reference canvases are layered directly under the data curve to give the appearance of a single canvas element. On document ready This is the analogous to onDOMReady with jQuery. The following section of setup queries is run once the phone application has loaded in the device. This sets up and binds the hardware keys, the 15 bluetoothSerial.subscribe callback and Hammer.js queries to detect touches. Converting data to JSON Data sent by the processing unit is a text string in JSON format. The string is parsed into JSON format once new data is received. This initiates canvas manipulations. Calculating curve positions 20 are completed by trigonometry from the bottom position. The code calculates the amount of data points received in the data object and estimates the length of the spine. The curve is simplified using PaperJS inbuilt smoothing functions, and the resultant spine curve is displayed within the canvas. Haptic feedback Signals are sent to the processing unit via a designated single character string. The code can choose 25 from a varied selection of pulses and vibration power. Calculating the deviation from optimal posture Figure 62 shows Calculating the deviation from user-saved best posture position Figure 63 shows a 3D representation of the mobile application. Separate canvas for each curve Each time new data is being sent, before the curve is being displayed, the PaperJS code runs a 30 comparison check to see if certain points along the body fall within a tolerance threshold, measured by 40 Positional feedback device distance. If the distance is greater than the tolerance, it will trigger a haptic feedback signal within the processing unit via Bluetooth followed by a visual colour change of the figure. Multiple points can be used as a comparison once suitable threshold levels are obtained from a medical professional. This profile can be saved within the app and additional reference profiles can be loaded on the fly, which 5 would make it possible for specific postures, such as for sports or Yoga. Performance Rounding sub-pixels HTML canvas supports sub-pixel rendering/anti-aliasing if the supplied data points are in the form of floating numbers. This causes performance issues in various browsers such as iOS and Mac platforms. 10 Since our formula to calculate points sometimes gives non-integer values, rounding may be an necessary option to increase speed. Options involve using built in Math.floor rounding or hacks. Hacks seem to perform faster, e.g. adding 0.5 to the number, and truncate to zero decimal point, with bit-wise shift. rounded = ~~ (0.5 + somenum); http://jsperf.com/math-round-vs-hack/3 15 Using lookup tables instead of JavaScript.Math functions Calculating the curve points using are computationally expensive. Currently, inbuilt JavaScript Math functions are used each time the curve is updated. The idea of using a lookup table in replacement for sine and cosine functions. If using Math functions as the data is given in degrees, besides the the trigonometric calculation we also need to convert degrees value to radians, with respect to floating 20 Math.PI (or cached rounded value). var cosLUT = [preprecalculated values for degrees] cosLookup(angle): takes angle as nth position in cosLUT return value at angle/nth position Input of the function is populating an array (array nth element = degrees lookup). The function would need to take negative degrees into consideration as well, and the extra conditional statement is needed. 25 This produced varying results. http://jsperf.com/ testing-lut-neg-and-trig-functions. Originally, the lookup table LUT technique was significantly faster than using Math functions. Testing with Chrome 30.0 results in a LUT advantage more consistent with OS platforms. On the Samsung Galaxy S2, the Math function was faster than LUT by factor of 18 (1 8M Op/sec). A separate test with rounded LUT values to 5dp showed a consistent increase compared one with more decimal places. This 30 approach has been placed on hold until more consistent testing can be completed. For loop efficiency 41 Positional feedback device A loop calculating the curve points from acquired data is run each time a new set of data is received by the MCU via bluetooth. This high frequency piece of code therefore needs to be very fast in order to save computing power. By increasing the amount of calculations/sec, the speed of curve plotting can be significantly and load reduced. 5 Figure 65 shows a jsperf.com benchmark with many loop implementations is available. Looping performance benchmark. The current function was in the format of old n busted. Without rewriting an initial parameter caching the length the for loop runs to, we can increase operations/second by a factor of 1.5. The while implementations are negative counter (comparison against 0, not a variable like len), which may reduce 10 computing cycles. While our curve point loop is different, this small addition without drastic alterations to current code. Note: This varies depending on which browser is used as not all JavaScript engines are created equal. In addition, i++ and ++i incrementing work the same way. Figure 66 shows - Left: No subpixel rendering (aliased), Right: Antialiasing is used to smooth the sprite as the origin point is not a set integer.) 15 Certain implementations of the WebView across both iOS and Android support floating numbers for points on the canvas. The WebView renders these across multiple pixels nearest in the form of anti aliasing. This results in extra computations to smooth out the graphics. In our case, most of the calculations are in decimal places, thus it this is also another area to focus on to speed up the application. 20 Proposed Bluetooth Error Detection Whilst there was initially bluetooth character dropping issue, we considered to suffix a cyclic redundancy check suffixed to the end of the JSON string to check if the JSON string itself was valid. The mobile application would encode the JSON string and compare it with the CRC attached. The data would only be processed by the curve updater function if passing this check. Any corruption to any of the two would 25 result in the string being dropped. This is considered a safe-side failure. The dangerous-side failure is both the checksum and data are corrupted such that they are both consistent, but its probability can be minimized by increasing the checksum length, at a cost of longer strings may trigger higher probability. This approach has been discarded since the Bluetooth plugin has been updated to rule out this error altogether. 30 Measurement Rig Design To help speed measurements, a manual rig and an automated rig was created. Please refer to Appendix A.3 for the manual rig. 42 Positional feedback device Automated Rig For the automated rig, it consist of a foam base, with a 6V servo motor hooked up to a Sparkfun 3.3V pro micro board. Voltage reading is done automatically via the Sparkfun 3.3V pro micro board. For resistance readings, many hour was taken up manually reading a standard multimeter. 5 Figure 67 shows - First is the automated rig testing a flex strip. Second is the automated rig testing an optical break flex sensor. In terms of servo accuracy, it is dependent on the angle of the servo motor. It is most accurate at zero flex or position 90 (0 to 180) or 0 (-90 to 90), can expect a tolerance of +-1 degree. However the tolerance diverges up to +-8 degrees when moving towards either side of the servo motor. 10 Overall Results & Discussions Results that has been obtained though testing the flexible sensors. For accelerometers, the side to side motions was working as expected, however its getting confused by forward and backwards sensing. Approaches to fixing the problem is known and will be fixed during 15 commercialisation and manufacturing. As for the flex sensors the output has good DC performance, but the movement is very jittery. This can be solved by using averages such as the recommended exponential moving average filter. Unfortunately there is no success with implementing an EMA filter onto the micro controller due to implementation problems from lack of micro controller ram. These issues will be addressed during commercialisation 20 and manufacturing. What did work well is the zeroing calibration, and it means that the rest flex was able to be reached consistently at all trials. The magnitude of the flex sensor response also appears to be consistent. In terms of the phone app, the app is able to receive and parse JSON commands to display a curve. It was also able to 'software' calibrate to a nominal spine curve, so that the app knows when to trigger an 25 alarm. This alarm could either be displayed as an indicator in the visual display, or as vibrational notification on the posture sensing device. While there was issues with the device, those encountered were only implementation problems to be resolved in manufacturing taking the current proof of concept - a cost effective spine sensing device. Additions 30 Despite the many goals that was fulfilled from the current device, there are still multiple areas for improvement. These includes how to ensure good accuracy. A common issue is that the flex strip cannot 43 Positional feedback device stretch to conform to the human back like a real skin. We are looking for either stretchable material in manufacturing or die cutting and creasing the sensor strips to have areas for stretch, similar to origami, and selectively stiffen just the flex sensing areas to prevent sensing. This will also allow for cheap side sensing, as unstiffen sensor could be arranged to detect side tilts. 5 Smoothing is a must, for it helps deal with any sensor jitter in the posture sensor, and also prevents disruption to the visual display due to dropped packets. Power usage reduction should be investigated, for it would reduce the psychological barrier to usage if the user don't have to charge it every day. To this end, Bluetooth Low Energy is the next logical step that would allow for potentially year long run time. 10 Don't forget to take into consideration that standing perfectly still is also a harmful posture. Notify users if they are standing straight too long. More haptic motors situation on the back may be an option to convey more information to users. A firm extension will also be to have strips that cover more of the back including the shoulders. Due to the nature of the adhesive stripes, it is also possible to attach these sensors, calibrate and 15 measure other areas of the body cost effectively. 20 25 44

Claims (18)

1. An apparatus comprising at least one sensor to detect the posture and / or physical location of a body part of a subject, the sensor in communication with a computing device to process sensor data and optionally a transmitter to transmit sensor data between the sensor and the computing device 5 and / or one or more additional computing devices.

2. An apparatus according to claim 1 wherein the sensor comprises a disposable strip which can be reduced to a required length and the strip optionally comprises one or more predetermined locations at which length can be reduced.

3. An apparatus according to claim 1 comprising a resistive ink and a conductive ink. 10

4. An apparatus according to claim 1 wherein the sensor comprises an adhesive section to adhere to the body of the subject wherein adhesion to the body optionally comprises one or more of adhesion to a close fitting garment and adhesion to the skin (and or hair) of the subject.

5. An apparatus according to claim 1 comprising one or more disposable components for low-cost replacement which optionally comprises a sensor strip. 15

6. An apparatus according to claim 1 further comprising a data handler to receive sensor data from the sensor and optionally store, and /or manage communication of said data to the computing device.

7. An apparatus according to claim 6 wherein the sensor and data handler are in physical communication.

8. An apparatus according to claim 1 wherein the sensor communicates with the computing device by 20 physical connection.

9. A system for monitoring the posture and / or physical location of a body part of a subject comprising a sensor and a computing device in communication with the sensor, the computing device able to receive, store and process sensed posture data into a form suitable for providing posture feedback to the subject. 25

10. A system according to claim 9 wherein the sensor comprises a disposable strip which can be reduced to a required length and the strip optionally comprises one or more predetermined locations at which length can be reduced.

11. A system according to claim 9 comprising a resistive ink and a conductive ink.

12. A system according to claim 9 wherein the sensor comprises an adhesive section to adhere to the 30 body of the subject wherein adhesion to the body optionally comprises one or more of adhesion to a close fitting garment and adhesion to the skin (and or hair) of the subject. 1 Positional feedback device

13. A system according to claim 9 comprising one or more disposable components for low-cost replacement which optionally comprises a sensor strip.

14. A system according to claim 9 further comprising a data handler to receive sensor data from the sensor and optionally store, and /or manage communication of said data to the computing device. 5

15. A system according to claim 14 wherein the sensor and data handler are in physical communication.

16. A system according to claim 9 wherein the sensor communicates with the computing device by physical connection.

17. A system according to claim 9 wherein communication between the computing device and sensor is optionally via one or more of: physical connection, wireless communication, Bluetooth, zigby, cellular 10 network, satellite.

18. A system according to claim 9 comprising a mobile computing device to notify the subject about posture feedback and optionally store and / or process sensed posture data. 15 2