ITRM20090002A1 - WEARABLE DEVICE AND ITS RELATED SYSTEM FOR DETECTION AND GUIDANCE OF THE MOVEMENT OF A USER. - Google Patents

Description

WEARABLE DEVICE AND RELATED SYSTEM FOR DETECTION AND GUIDANCE OF A USER'S MOVEMENTS

The present invention relates to a wearable device, comprising at least one instrumented garment, and the relative system for detecting and guiding the movements of a user, which allows in a simple, effective and reliable way to guide in real time a user wearing said at least an instrumented garment, in particular an instrumented glove, in carrying out movements, for example to search for and / or grasp an object or to perform a particular manipulation (in the case of surgical methods or craftsmanship), the system is also easily economically achievable and adaptable to different applications.

The present invention also relates to the relative driving procedure performed by the system.

There are many situations in which an operator must search and / or manipulate an object without the aid of a direct view of it.

By way of example, a visually impaired or blind user often needs to search for objects of which he does not remember or does not know the exact location within an environment. Other examples are those of operators who have to handle tools and / or objects during the manufacture or maintenance of complex products in working conditions that inhibit a complete or perfect view of the work environment (e.g. laboratory rooms where the € ™ direct interaction of operators).

Currently, there are systems that are able to indicate the position of an object on the basis of a signal emitted by the latter, for example an electrical signal emitted by a transponder which the object is equipped with, the indication of the position by means of a visual and / or acoustic signal. Such a system is described in US Patent No. 7034684.

However, such systems are rather limited, inflexible, and scarcely efficient if the system can indicate the position of a plurality of objects, being able to provide false or incorrect information. Moreover, often such systems still require a direct view of the object, after its location, in order to grasp it.

It is also known that operators in many technical or sports sectors must perform a series of specialized movements for which specific skills are required, achieved through long training.

This is the case, purely by way of example, but not limiting, of a surgeon who must carry out a specific method of intervention, or of an athlete (for example a fencer, a tennis player or a football player) who must perform specific movements of his sport, or even of a craftsman who has to manually manufacture a product according to traditional techniques. Another illustrative case is that of a patient undergoing a motor or postural re-educational treatment, for example following trauma or surgery, who must perform predetermined movements or assume and / or maintain certain postures.

In these cases, the training of the operators takes place under the direct supervision of a trainer, whose indications on the correction of incorrect movements or postures are usually limited and not completely reliable.

In this regard, some types of gloves are currently marketed, also known under the name of Dataglove or Cyberglove (registered trademarks of Sun Microsystems and Immersion Corporation, respectively), which only allow the detection of movements (motion capture) made by users who they wear them, in order to reproduce the movements of a user either in a virtual world or through a robotic manipulator.

In this context, the solution proposed according to the present invention is inserted, which allows to solve the aforementioned problems.

The purpose of the present invention is, therefore, to allow in a simple, effective, reliable way and in real time, both the detection and the guidance of the movements of a user, for example to search and / or grasp an object or to perform one or more of a series of specialized movements.

The specific object of the present invention is a wearable device comprising at least one instrumented garment, the device comprising

- electronic means of control, equipped with

- means of communication for communicating with external processing means,

characterized in that at least one instrumented garment is provided with

- electromechanical actuator means controlled by said electronic control means on the basis of command signals received through said communication means and coming from said external processing means.

Always according to the invention, said electromechanical actuator means can comprise one or more, preferably at least four, electromechanical actuators.

Still according to the invention, said electronic control means can comprise a microcontroller unit and said communication means can comprise an interface module for communicating with said external processing means by means of a wireless or wireless communication, preferably packet, again more preferably according to one or more standards selected by the group comprising the wi-fi, bluetooth and zigbee standards, and / or by means of a wired communication, preferably a serial communication.

Further according to the invention, said electronic control means can be connected to said electromechanical actuator means and can be connected to said electromechanical actuator means by means of a wired or wireless connection.

Also according to the invention, at least one instrumented garment can be provided with

- sensor means adapted to detect a position and / or an orientation of the respective instrumented garment, sending corresponding detection signals to said electronic control means,

said electronic control means by sending, through said communication means, said detection signals, preferably following an at least partial processing by said electronic control means, to said external processing means.

Still according to the invention, said electronic control means can operate according to software that can be updated by connecting an external memory to at least one interface of said electronic control means, said at least one interface preferably being a USB socket or socket.

Preferably according to the invention, said at least one instrumented garment is made of elastic material, in such a way as to remain adherent to a user who wears it.

Further according to the invention, said at least one instrumented garment can be selected from the group comprising a glove, a legging, a stocking, a suit, a shirt, and a trousers.

Another specific object of the present invention is a system for detecting and guiding the movements of a user, comprising:

- means of processing,

- a device wearable by said user comprising at least one instrumented garment,

characterized by the fact that said wearable device is a wearable device as previously described, and by the fact that the system also includes

- detection means adapted to detect a position and / or an orientation of at least one instrumented garment of the wearable device and to send corresponding detection signals to said processing means,

said processing means processing said detection signals and sending command signals to electronic control means, through respective communication means, of at least one instrumented garment of the wearable device, said electronic control means generating one or more control signals of said electromechanical actuating means of said at least one instrumented garment of the wearable device on the basis of said command signals, said command signals depending on a difference in the current position and / or orientation of said at least one instrumented garment of the wearable device detected by said sensing means with respect to a desired position and / or orientation (i.e., an ideal or target position and / or orientation).

Always according to the invention, said detection means can comprise sensor means of at least one instrumented garment of the wearable device.

Still according to the invention, said detection means can comprise one or more detection means selected from the group comprising:

- electro-optical means for artificial vision, preferably comprising an artificial vision device, more preferably made through a pair of glasses, even more preferably equipped with one or more, preferably two, cameras,

- detection means based on geo-magnetic detection,

- radiofrequency detection means, preferably based on RFID devices, e

- detection means based on accelerometers (inertial platforms), said detection means being preferably provided with fixed and / or relative spatial references.

Further according to the invention, said desired position and / or orientation (or ideal or target) can be a position and / or orientation of an object detected by said detection means.

Always according to the invention, said processing means may be capable of memorizing a succession of positions and / or spatial orientations of at least one instrumented garment of the wearable device detected by said detection means, and said desired position and / or orientation ( or ideals or targets) can be selected from said succession of stored spatial positions and / or orientations.

A further specific object of the present invention is a procedure for detecting and guiding the movements of a user wearing a wearable device comprising at least one instrumented garment, said wearable device being a wearable device as previously described, comprising the step of:

A. detect a position and / or spatial orientation of at least one instrumented garment,

the procedure being characterized by the fact that it also includes the following phases:

B. determining a difference in the current position and / or orientation of said at least one instrumented garment of the wearable device detected in step A from a desired (or ideal or target) position and / or orientation, and

C. sending command signals to electronic control means, through respective communication means, of at least one instrumented garment of the wearable device, said electronic control means generating one or more control signals of electromechanical actuator means of said at least one instrumented garment of the wearable device based on said command signals.

The wearable device, comprising at least one instrumented garment, (and the relative system) according to the invention, once worn, allows in a simple, effective and reliable way both a guided search for objects and a guided training of the movements of the parts of the body. of the user, in particular of the joints, with which said at least one garment comes into contact.

The present invention will now be described, for illustrative but not limitative purposes, according to its preferred embodiments, with particular reference to the Figures of the attached drawings, in which:

Figure 1 shows a perspective view of a preferred embodiment of the wearable device according to the invention when worn by a user;

Figure 2 shows a schematic block diagram of the preferred embodiment of the system according to the invention;

Figure 3 shows a schematic flow graph of a preferred embodiment of the process according to the invention.

In the Figures identical reference numbers will be used for similar elements.

In the following description, explicit reference will be made to an embodiment of the invention in which the wearable device is an instrumented glove that a user wears on one hand (the left hand is shown in Figure 1) and the respective forearm. However, it must be borne in mind that the wearable device according to the invention can comprise, alternatively to the glove or also in combination with it, one or more different instrumented garments wearable by a user on parts of the body other than the left hand, always remaining within the scope of the present invention. By way of example, but not limited to, a nurse, a surgeon, or a pool player could wear two instrumented gloves that possibly also cover the forearms; a footballer could wear one or two instrumented leggings instead of gloves; a fencer could wear a fully instrumented suit, possibly divided into two or more pieces (for example a shirt and trousers). Such instrumented garments are made of elastic material in such a way as to remain adherent to the part of the body on which they are worn. Another exemplary application of the wearable device according to the invention is that of a patient undergoing a motor or postural rehabilitation treatment, for example following trauma or surgery, who must perform predetermined movements or assume and / or maintain certain postures; also in this case repeatability and precision of execution are much better controllable and correctable by means of the wearable device according to the invention than under the sole supervision of a physiotherapist.

With reference to Figure 1, a user 1 can be seen wearing a preferred embodiment of the wearable device according to the invention consisting of a left instrumented glove 2. The glove 2 is worn on the left hand and also covers an anterior portion of the € ™ forearm.

The system according to the invention, illustrated in Figure 2, ensures that the instrumented glove 2, once worn, is able to provide both the ability to train the movements of the arm, in particular the shoulder / elbow / wrist joints, and guided object search.

In particular, the system according to the invention provides for the real-time evaluation of the spatial position of the left hand (and possibly of the respective forearm) of the user, preferably through one or more sensors 3, available for example by the company Polhemus, with which glove 2 is provided. Other embodiments of the system according to the invention may provide that some further sensors are located on the entire arm of the user (for example through adherent straps which constitute further instrumented clothing belonging to the wearable device) in order to evaluate the spatial position of the entire arm in real time.

The system shown in Figure 2 is also provided with an artificial vision device 4, preferably made with a pair of glasses, equipped with one or more, preferably two, cameras 5, also worn by the user. Alternatively, the artificial vision device 4 can be placed in a fixed position in the environment to be monitored (possibly equipped with handling devices that allow it, by way of example but not limited to, a rotation movement and / or a pitching movement and / or a vertical translation movement) to observe at least a portion of the environment to be monitored.

In particular, other embodiments of the system according to the invention can provide only the sensors 3 or only the device 4.

Still other embodiments of the system can be provided, as an alternative to or in combination with the sensors 3 and / or the device 4, with different position evaluation apparatuses, with fixed and / or relative spatial references, based on vision or on geo-magnetic detection, and / or on radiolocation (for example through passive RFID-type devices placed on the object whose position is to be determined and interrogation / detection antennas suitably arranged), and / or on inertial platforms (for example through accelerometers from whose signals it is possible to accurately calculate the position and velocity components of the object from the known initial conditions).

The sensors 3 send the detection signals to a microcontroller unit 6 which, through an interface module 7, provides wireless communication functionality with a processor 8, preferably a workstation (i.e. a PC). In this way, the work-station 8 processes the spatial detection signals received by the sensors 3 to determine the spatial position of the instrumented glove 2.

The system of Figure 2 uses one or more different wireless communication standards for the communication between the work-station 8 and the microcontroller unit 6 of the instrumented glove 2. Preferably, one or more of the standards selected from the group comprising wi-fi are used, bluetooth and zigbee (IEEE 802.15.4).

Other embodiments of the system according to the invention can provide, as an alternative to or in combination with the wireless channel 9, a wired channel, for example an RS232 serial channel.

Using a plurality of wireless transmission standards, the system exploits different software protocols for communication, thus optimizing the band available on radio channel 9. The preferably used standard is in any case the zigbee one, thanks to the low power consumption it requires.

The microcontroller unit 6, dedicated to wireless communication, is equipped, through module 7, with a plurality of communication interfaces and is able to accept future wireless devices (e.g. if they prove to be more efficient from the point of view of consumption of power and / or use of the radio band and / or range of action).

The microcontroller unit 6 is also equipped with a battery that ensures its operation for a few hours, depending on the wireless communication standard chosen. As shown in Figure 1, the microcontroller unit 6 of a preferred embodiment of the system according to the invention is housed, together with the interface module 7 and the power supply battery, in a container 10 of reduced dimensions, comparable to those of two superimposed packets of cigarettes, which can also be easily worn by the user, for example by attaching it to a belt (see Figure 1) or, by means of a strap, to the arm. Other embodiments of the wearable device according to the invention can provide that the microcontroller unit 6 and the interface module 7, and possibly also the power supply battery, are housed in the same instrumented glove (or other garment).

As will be illustrated in greater detail below, the user wearing the instrumented glove 2 receives information on the correct movement carried out and / or on the correct spatial position assumed by his arm and / or his hand through a sensory feedback obtained through one or plus actuators 11 incorporated in the instrumented glove 2 and suitable for providing adequate sensory feedback. Preferably, the actuators are at least four and are arranged on each of the two sides, on the back and on the palm of the housing for the hand.

In the embodiment of the wearable device according to the invention illustrated in the Figures, the actuators 11 are small vibrating motors placed inside the fabric of the instrumented glove 2. Each time the system detects an incorrect hand positioning, for example too high, the motor placed inside the fabric of the instrumented glove 2 in correspondence with the back of the housing for the hand will begin to vibrate, with an intensity preferably proportional to the position error detected (i.e., greater à Is the error, the greater the intensity of the vibration).

The actuators 11 present inside the fabric of the instrumented glove 2 are controlled by the microcontroller unit 6, which preferably in turn receives control signals of the actuators from the work-station 8.

In particular, the activation of the motors, which in the embodiment of the system according to the invention shown in the Figures constitute the actuators 11, is achieved by means of a PWM signal generated by the microcontroller unit 6. The communication protocols between the work- station 8 and the microcontroller unit 6 preferably provide for the packetized transmission of the values that indicate the intensity of the vibration for all motors 11. Purely by way of non-limiting example, in the case of an IEEE 802.15.4 wireless channel, the structure transmitted as package from work-station 8 to microcontroller unit 6 is the following (where the maximum number of motors MAXNUM 11 is equal to six, a number that can be easily reduced or increased thanks to a simple hardware modification within the reach of average technicians in the sector ):

#define MAXNUM 6

typedef struct

Table 2

{

unsigned int id;

unsigned char tau;

unsigned char status [MAXNUM];

} __ attribute __ ((__ packed __)) MOTOR;

In particular, the sensors 3 and the actuators 11 are connected to the microcontroller unit 6 through wired connections 12, as illustrated in Figures 1 and 2. Other embodiments of the system according to the invention can provide a wireless connection between the components of the glove (or other garment) instrumented 2 and the microcontroller unit 6.

The work-station 8 is also connected to the artificial vision device 4, provided with a corresponding interface module 13, via a communication channel 14, wired or wireless, to receive the images detected by the device 4, possibly process them, and control the same device 4.

The software executed by the microcontroller unit 6 which implements the communication protocol (s) and the control functions of the actuators 11 can be replaced with a simple procedure, for example by loading the executable (binary) files on a memory device or memory key (pen drive) USB, by inserting the memory key in a corresponding USB socket or socket on the microcontroller unit 6 (and subsequently switching off and on the microcontroller unit 6 itself).

In turn, the work-station 8 runs a software which allows to process data coming from the sensors 3 of the instrumented glove 2 and from the artificial vision device 4 and consequently to control the actuators 11 of the instrumented glove 2 (and preferably also the device 4). Other embodiments of the system according to the invention can provide that at least part of the data processing is performed by the microcontroller unit 6 and by the artificial vision device 4.

In a particular non-limiting example, the work-station 8 of the system shown in the Figures runs a software that first of all allows the search and guidance of the hand of the user wearing the instrumented glove 2 towards an object whose position is known. The spatial position of the object to be grasped (determined by means of the images provided by the device 4 and processed, for the recognition of the object, by the work-station 8) and the position of the user's hand (wearing the glove) instrumented 2 and determined by means of the detection signals supplied by the sensors 3, possibly in combination with the images supplied by the device 4 and processed by the work-station 8) are compared by the software executed by the work-station 8, which in real time sends to the microcontroller unit 6 the commands for activating the actuators 11. The microcontroller unit 6 receives the commands, interprets them, and consequently activates the actuators 11 (for example vibrating motors) present inside the instrumented glove 2.

By way of non-limiting example, if the hand (wearing the instrumented glove 2) is in a too high position with respect to the object to be grasped, an actuator (or actuators) 11 will be present on the back of the housing for hand to start functioning and to signal, vibrating in a more or less intense way, the position error. If, on the other hand, the hand is too low in relation to the object to be grasped, an actuator (or actuators) 11 present on the palm of the housing for the hand will come into operation. If the hand is too lateral, for example too far to the right (or left), with respect to the object to be grasped, an actuator (or actuators) 11 will be present on the opposite side, in the case of the example on the left (or to the right) of the housing for the hand to operate. Obviously, other embodiments of the system according to the invention can provide a different operating order of the actuators 11 incorporated in the instrumented glove 2.

A second function offered by the software executed by the workstation 8 of the system shown in the Figures allows the recording of a movement of a hand wearing the instrumented glove 2, or rather the memorization of the succession of spatial positions occupied by the hand (or by the entire arm ) of the user wearing the instrumented glove 2 during the execution of a movement. The sequence of positions is recorded by the work-station 8 which, during a second execution of the movement itself, is able to compare the current positions with those previously recorded and detect any differences by guiding the second execution of the movement by activating the actuators 11 (for example vibrating motors) as already described.

By way of example, but not limiting, this allows the memorization of a movement of an expert user and the subsequent training of a second non-expert user.

With reference to Figure 3, it is possible to observe the driving procedure performed by the system shown in Figures 1 and 2, in which steps 100 to 104 are performed by the work-station 8 and steps 105 and 106 are performed by the microcontroller unit 6 .

In phase 100, the work-station 8 receives the signals relating to the spatial position of the instrumented glove 2 worn by the user's left hand and possibly of the object to be grasped. In particular, the work-station 8 receives, through the microcontroller unit 6, the detection signals coming from the sensors 3 (alternatively or in combination, the work-station 8 could also receive the images coming from the artificial vision device 4 useful for determining the position of the instrumented glove 2). In the event that the system is performing the search function of an object, the work-station 8 receives the images for the search of an object, the work-station 8 receives the images coming from the artificial vision device 4 useful for determining the position of the object.

In step 101, the work-station 8 determines the current spatial position of the instrumented glove 2 by processing the detection signals received by the sensors 3 (alternatively or in combination, the work-station 8 could process the images coming from the device 4); moreover, the work-station 8 determines the desired or ideal spatial position (or target) of the instrumented glove 2. If the system is performing the function of searching for an object, the ideal spatial position of the instrumented glove 2 is a function of the current position of the object that the work-station 8 determines by processing the images coming from the artificial vision device 4. In case the system is performing the function of repeating a memorized movement, the ideal spatial position of the instrumented glove 2 is recovered from the memory of the work-station 8.

In phase 102, the work-station 8 compares the current position and the ideal position of the instrumented glove 2, preferably verifying whether they differ from each other beyond a maximum threshold: if the outcome of this verification is positive (ie the current position is incorrect, as it is different from the ideal one for more than said maximum threshold), then the procedure carries out step 103; otherwise, if the outcome of this check is negative (ie the current position is correct, as it is substantially coincident with the ideal one), then the procedure returns to carry out phase 101.

In step 103, the work-station 8 generates the command signals of the actuators 11 on the basis of the difference between the current position and the ideal position of the instrumented glove 2.

In step 104, the work-station 8 transmits the control signals of the actuators 11 to the microcontroller unit 6.

In step 105, the microcontroller unit 6 receives the command signals of the actuators 11 from the work-station 8.

In step 106, the microcontroller unit 6 generates the PWM signals for activating the actuators 11 on the basis of the command signals received.

In the foregoing, the preferred embodiments have been described and variants of the present invention have been suggested, but it is to be understood that those skilled in the art will be able to make modifications and changes without thereby departing from the relative scope of protection, as defined by claims attached.

Claims (11)

CLAIMS A wearable device comprising at least one instrumented garment (2), the device comprising - electronic means (6) of control, provided with - communication means (7) for communicating with external processing means (8), characterized in that at least one instrumented garment (2) is provided with - electromechanical actuator means (11) controlled by said electronic control means (6) on the basis of command signals received through said communication means (7) and coming from said external processing means (8). 2. Wearable device according to claim 1, characterized in that said electromechanical actuator means comprise one or more, preferably at least four, electromechanical actuators (11). 3. Wearable device according to claim 1 or 2, characterized in that said electronic control means comprise a microcontroller unit (6) and said communication means comprise an interface module (7) for communicating with said communication means (8) external processing by means of a wireless or wireless communication, preferably packet, even more preferably according to one or more standards selected by the group including the wi-fi, bluetooth and zigbee standards (IEEE 802.15.4), and / or by means of a wired communication, preferably a serial communication. Wearable device according to any one of the preceding claims, characterized in that said electronic control means (6) are connected to said electromechanical actuator means (11) by means of a wired or wireless connection. 5. Wearable device according to any one of the preceding claims, characterized in that at least one tooled garment (2) is provided with - sensor means (3) adapted to detect a position and / or an orientation of the respective instrumented garment (2), sending corresponding detection signals to said electronic control means (6), said electronic control means (6) sending, through said communication means (7), said detection signals, preferably following an at least partial processing by said electronic control means (6), to said external processing means (8). 6. System for detecting and guiding a user's movements, comprising: - processing means (8), - a device wearable by said user comprising at least one instrumented garment (2), characterized in that said wearable device is a wearable device according to any one of claims 1 to 4, and by the fact that the system also comprises - detection means (3, 4, 5) suitable for detecting a position and / or orientation of at least one instrumented garment (2) of the wearable device and for sending corresponding detection signals to said processing means (8), said processing means (8) processing said detection signals and sending command signals to electronic means (6) for controlling, through respective communication means (7), of at least one instrumented garment (2) of the wearable device, said electronic means (6) control by generating one or more control signals of said electromechanical actuator means (11) of said at least one instrumented garment (2) of the wearable device on the basis of said command signals, said command signals depending on a difference in the current position and / or orientation of said at least one instrumented garment (2) of the wearable device detected by said detection means (3, 4, 5) with respect to a desired position and / or orientation. 7. System according to claim 6, characterized in that said wearable device is a wearable device according to claim 5 and that said detection means comprise sensor means (3) of at least one instrumented garment (2) of the wearable device. 8. System according to claim 6 or 7, characterized in that said detection means comprise one or more detection means selected from the group comprising: - electro-optical means (4, 5) for artificial vision, preferably comprising an artificial vision device (4), more preferably made through a pair of glasses, even more preferably equipped with one or more, preferably two, cameras (5) , - detection means based on geo-magnetic detection, - radiofrequency detection means, preferably based on RFID devices, e - detection means based on accelerometers (inertial platforms), said detection means being preferably provided with fixed and / or relative spatial references. System according to any one of claims 6 to 8, characterized in that said desired position and / or orientation is a position and / or orientation of an object detected by said detection means (3, 4, 5). System according to any one of claims 6 to 9, characterized in that said processing means (8) are capable of memorizing a succession of positions and / or spatial orientations of at least one instrumented garment (2) of the wearable device detected by said detection means, and by the fact that said desired position and / or orientation are selected from said succession of stored spatial positions and / or orientations. 11. Process for detecting and guiding movements of a user wearing a wearable device comprising at least one instrumented garment (2), said wearable device being a wearable device according to any one of claims 1 to 5, comprising the step of: A. detect a position and / or spatial orientation of at least one instrumented garment (2), the procedure being characterized by the fact that it also includes the following phases: B. determining a difference in the current position and / or orientation of said at least one instrumented garment (2) of the wearable device detected in step A from a desired position and / or orientation, and C. sending command signals to electronic control means (6), through respective communication means (7), of at least one instrumented garment (2) of the wearable device, said electronic control means (6) generating one or more control of electromechanical actuator means (11) of said at least one instrumented garment (2) of the wearable device on the basis of said command signals.