FR3015712A1 - APPLICATION PROGRAMMING INTERFACE (API) FOR INTERACTIVE GLOVE - Google Patents

Abstract

The invention relates to a method for developing a software application (A) interfaced with an interactive glove (G) comprising a step of using an application programming interface (API) comprising a list of functions ( F1, F2, F3) associated with possible gestures by the interactive glove, in which a combination of sensor values (C1, C2, C3, C4) associated with the interactive glove makes it possible to determine output values of these functions, so in operation, a software application (A) can interact with a function among the list to react to the gesture of the interactive glove associated with the function.

Description

BACKGROUND OF THE INVENTION The invention relates to the field of human-machine interfaces (HMI) for an information processing system. More specifically, the invention relates to an interactive glove type human-machine interface device and the interfacing of such a device with software applications implemented by an information processing system. BACKGROUND OF THE INVENTION In recent decades, the field of human-machine interfaces has been enriched by a large number of advanced devices such as interactive gloves. These devices make it possible to enrich the possible interactions between an information processing system and its users. The information processing system must here be understood in its broadest sense, as including computers, communication terminals, game consoles ... It may also be specialized systems in certain fields such as the medical field, etc. An interactive glove is such a device that the user generally puts on his hand, and which comprises a set of sensors whose values make it possible to determine gestures. These gestures may include: positions of the glove (and therefore of the hand): unfolded hand, tight fist, contact between different fingers, etc. and movements of the glove: translation, rotation ... of the glove and the hand. 178BULL-35455-patent-draft2 2 These gestures can correspond to real gestures of a practitioner whose precision and know-how are thus captured by the interactive glove. It can be here medical applications but also to capture gestures to make them reproduce by an automatic hand on a remote site (for example in a dangerous environment where the user can not easily go). These gestures can also correspond to a predefined ad-hoc vocabulary for communicating with the information processing system.

Software applications are implemented on the information processing systems and it is therefore necessary to be able to allow them to interact with an interactive glove connected to the system. To do this, each software application can interact with the interactive glove (s) directly or through the operating system (OS for "Operating System" in English). The software application must therefore directly encode the interaction modalities with the glove, which implies for the application developer, both to know all possible actions for the interactive glove connected to the system, but also the way to communicate physically with the sensors associated with the interactive glove. As a result, the developer of a software application must know precisely the technical specifications of the interactive glove. It also results that changing the interactive glove requires to recode a good part of the software application using it.

This situation complicates the development of software applications wishing to interface with an interactive glove and therefore increases their cost of development and maintenance as well as the risk of error. In doing so, it acts as a brake to a greater use of these human-machine interface devices. The purpose of the present invention is to at least partially overcome the aforementioned drawbacks.

To this end, the present invention provides an interactive glove application programming interface comprising a list of functions associated with possible gestures by said interactive glove, in which a combination of values of associates with said interactive glove makes it possible to determine values of outputs of said functions, and wherein a software application can interact with a function among said list to react to the gesture of said interactive glove associated with said function. According to preferred embodiments, the invention comprises one or more of the following features which can be used separately or in partial combination with each other or in total combination with one another: said output values indicate a duration of said gesture. said output values indicate a pressure. said output values are Boolean values. said functions include an implementation part allowing the matching between said sensor values and said output values, said values being representative of the gestures associated with said functions. - Said functions are grouped into families of gestures.

Another aspect of the invention relates to a method of developing a software application interfacing with an interactive glove comprising a step of using an application programming interface comprising a list of functions associated with possible gestures by said interactive glove, in which a combination of sensor values 178BULL-35455-patent-draft2 4 associated with said interactive glove makes it possible to determine output values of said functions, so that in operation, a software application can interact with a function among said list to react to the gesture of said interactive glove associated with said function.

According to preferred embodiments, the invention comprises one or more of the following features which can be used separately or in partial combination with each other or in total combination with one another: which said output values indicate a duration of said gesture. said output values indicate a pressure. said output values are Boolean values. said functions include an implementation part allowing the matching between said sensor values and said output values, said values being representative of the gestures associated with said functions. said functions are grouped into families of gestures. Another aspect of the invention relates to an information processing system comprising at least one software application and an application programming interface as previously described. Another aspect of the invention relates to such an information processing system and further to an interactive glove linked to said information processing system by means of communication.

According to one embodiment of the invention, said communication means is wireless. Other features and advantages of the invention will appear on reading the following description of a preferred embodiment of the invention, given by way of example and with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 schematically illustrates an example of a context in which the invention may be inserted. DETAILED DESCRIPTION OF THE INVENTION Figure 1 illustrates the context in which the invention may fit. An interactive glove G comprises a set of sensors including a part C1, C2, C3, C4 is indicated in the figure. The location of the sensors is specific to the manufacturer of the interactive glove and therefore outside the scope of the invention.

In general, the sensors are located in such a way as to maximize the number of gestures detected for the smallest number of sensors in order to optimize the manufacturing costs of the gloves. Typically, C1, C2 sensors are placed at the ends of the fingers (preferably each finger), at the joints, C3, or on the surface of the hand or fingers C4. This variety in the placement of the sensors makes it possible to determine a large number of different gestures: folding of one or more fingers, contacts between fingers, rotations, translations of the glove, etc. Likewise, different sensor technologies exist and the invention is not related to any particular type of technology. These sensors can take values depending on their physical activation. These values can be binary and representative of a contact or an absence of contact, or be recorded in a range of values, continuous or discrete, representative for example of a pressure exerted on the sensor. 178BULL-35455-patent-draft2 6 These sensor values are transmitted to an information processing system ST via a communication means MC connecting the interactive glove and this system.

This communication means MC can be wired or wireless. It can comply with different communication standards such as Wifi, Bluetooth, USB, etc. The management of the communication means MC by the information processing system ST is typically supported by the operating system, OS, of the system ST. From the point of view of the information processing system, it can be treated as an input computing device (such as a mouse, a keyboard, a joystick, a graphics tablet ...). A specific driver can be provided so that the operating system OS can use the interactive glove G through this driver like any other computing device. This information processing system ST can be a fixed or portable computer, a communication terminal, for example of the "smartphone" type, a digital tablet, etc.

The operating system may be a function of the type of information processing system ST. For example, this can be iOS from Apple, Android, Windows from Microsoft, Linux, etc.

Above the operating system, one or more software applications A may operate, possibly in competition or in parallel. These software applications A may wish to use the interactive glove G specifically. They can also be controlled by the interactive glove G, in a way that is transparent to them, in the same way that they can be controlled by the interactive glove G. keyboard, mouse, voice-activated interface, etc. According to the invention, a programming interface is set up to allow the development of new software applications A. This programming interface, API (for "Application Programming Interface") has been schematically represented as an intermediate layer between applications A and the OS operating system because it allows the developer to standardize certain functions of the operating system. In operating mode, that is to say when the end user uses the application, the API programming interface is then transparent to him. This API application programming interface comprises a list of functions F1, F2, F3 ... associated with possible gestures of the interactive glove G. In order to allow the maximum of possibilities for the application developers, it can be planned to define all possible gestures in a search for completeness.

Each function F1, F2, F3 can be associated with a given gesture and therefore comprises the computer code for analyzing the values transmitted by the sensors C1, C2, C3, C4. The list may also be general enough to cover as many types of interactive gloves as possible. In this way, for a given glove, certain functions will correspond to impossible gestures, but the associated function can then treat this case by returning an appropriate value or by never being activated, as we will see below. 178BULL-35455-patent-draft2 8 The following is a list of possible functions. This list is indicative, in particular by the order and the numbering of the functions that it implies. These functions can be grouped into family of gestures.

In a possible implementation: - A first family consists in associating a function with each movement of the fingers whether it is a right hand or a left hand: o Fl: Folding of the thumb o F2: Folding of the index o F3: Folding of the middle finger o F4: Folding of the ring finger o F5: Folding of the little finger - A second family consists in associating a function with each combined movement of the thumb and another finger, or even the whole hand: o F6: F1 and F2 simultaneous o F7: F1 and F3 simultaneous o F8: F1 and F4 simultaneous o F9: F1 and F5 simultaneous o F10: Fl and F2 and F3 and F4 and F5 simultaneous - A third family is to associate a function with each pinch (so folding and touching the tip of the finger (s)) between the thumb and another finger, or even the whole hand: o Thread: Contact between thumb and index o F12: Contact between thumb and middle finger o F13: Contact between thumb and ring o F14: Contact between thumb and little finger o F15: Contact between thumb and index finger and major and annular and auricular 178BULL-35455-patent-draft2 9 - Another family corresponds to a reservation of a range of optional functions for other combinations of the fingers of the hand: F15 to F30 - Another family consists in associating a function at different movements of the hand o F31: Forward movement with the hand o F32: Rearward movement of the hand o F33: Vertical rotation towards the top of the hand o F34: Vertical rotation towards the bottom of the hand o F35: Horizontal rotation to the left of the hand o F36: Horizontal rotation to the right of the hand o F37: Axial rotation (axis = radius or ulna) to the left o F38: Axial rotation (axis = radius or ulna) to the right - Another family is a reservation of a range of optional functions for other movements of the hand: F39 to F49 - A last family consists in associating the function F50 with an initialization or reinitialization of the spatial position of the m ain. It may be associated with another already existing function. For example, it may be agreed that function F10 (folding of all fingers simultaneously) will be associated with function F50.

Each of these features provides an interface for application developers that is independent of the underlying technology including the interactive glove. Thus, the developer only has to know the "external" technical specifications of the PLC programming interface: the names of the functions, to which gesture each 178BULL-35455-patent-draft2 10 corresponds, and the possible associated parameters, including the return values (binary or with indication of a pressure value, etc.). Furthermore, the API application programming interface includes a specific part to the interactive glove and to the underlying infrastructure (in particular operating system OS). This part can be ignored by the developer of applications, so that it does not have to know the technical specificities of the interactive glove used. This implementation part is related to the type of interactive glove used and can encode the mapping between sensor values transmitted by the interactive glove and the determination of a given gesture. Thus, at the level of each function, an algorithm makes it possible to determine whether the gesture corresponding to the function has been performed as a function of the sensor values. In other words, a combination of sensor values associated with the interactive glove makes it possible to determine a gesture. In some cases, a combination may be formed of the value of a single sensor, sufficient to determine a gesture. As previously mentioned, the functions can refer to the software application A different values, depending on whether the associated gesture has been determined or not. These values can be simply Boolean or be numeric. Numeric values may traverse a continuous interval or belong to a set of possible discrete values. Taking the example of the first function of thumb folding, we can have the values defined in this way: F1 returns 0 = The thumb is not bent. F1 returns 1 = The thumb generates a folding pulse. Fl returns <n> = Thumb remains folded for <n> seconds. F1 returns negative values to cover various unresolved errors or misinterpretations: 178BULL-35455-patent-draft2 11 Fl returns -1 = Function not implemented or sensor not active. Fl returns the negative value - <n>: Error codes can be related to the types of gloves and sensors used.

Rather than the holding time of a gesture, the value <n> can indicate a pressure provided by a pressure sensor. It is also possible that the returned value is formed of a pair of sub-values, for example in the form <n1, n2> where, for example, n1 represents the duration of the gesture and n2 represents the pressure.

It is obvious that many other possibilities are accessible to those skilled in the art. An implementation of these principles of separation between part "interface" and part "implementation" can be achieved through an object-oriented software architecture. Such an architecture makes it possible to separate an "interface" part used by the software applications from an "implementation" part that is hidden from them. For example, the functions may be implemented by "methods" associated with an "interactive glove" class.

The implementation can be performed in the form of a web service, a software library (or bookseller), etc. and for different computer languages: C, C ++, Java, etc.

An example of implementation in the C ++ language of the function F10 corresponding to the complete folding of the right hand is given below: One can for example define a class to describe the hand and a class to describe the fingers in a file named " APIGant.h ".

178BULL-35455-patent-draft2 12 enum TypeFinger {INCH, INDEX, MAJOR, ANNULAR, EARLY}; enum TypeHand {RIGHT, LEFT}; enum StateType {INACTIVE = -1, NOT FOLD = 0, FOLD = 1} #define OK 0 #define ERROR -99 Finger Class {// What is the type of TypeFinger type; // initialized value of finger concerned static TypeState state = NO FOLDED; public state: Finger (TypeFinger); int testState (); int state (int, TypeFord); intpliance (); I; // Builder of the class Finger Finger :: Finger (TypeFigtype) {// Initialization of the finger concerned type = whattype; I; // Function test function of the finger int Finger :: testState () {// Test function related to the sensor and therefore to the type of glove used state = sensor test function (type); return state; 178BULL-35455-patent-draft2 13 1 // Finger state assignment function int Finger :: state (int whatever, TypeFordtype = type) {// Verification that the correct type of finger has been received if given as input if (what type <> this.type) {return ERROR; 1 else {// Initialization of the current state of the finger state = which state; return state; ) 1 // Handler called during a change of state received from the sensors of the Handler glove Finger :: folding () {// Recovery of the received event status = sensor handler (); I; Class Hand {// In English because "main" will be problematic in C / C ++ TypeHand type; // Type of hand RIGHT or LEFT static TypeState state = NOT FOLDED; // Initialized value of the current state Thumb finger (INCH); 178BULL-35455-patent-draft2 14 Index finger (INDEX); Major finger (MAJOR); Ring finger (ANNULAR); Aural finger (AURICULAR); public: Hand (TypeHand); int initialization (); int fingerInvalid (TypeFinger); int testPliement (); 1; Hand :: Hand (TypeHand whatevertype) {// Initialize the relevant hand type = whattype; }; int Hand :: reset () {// Reset the state of all the fingers of the hand as not folded // The type of the hand is not changed inch.state (NOT FOLD, INCH); index.state (NOT FOLD, INDEX); major.state (NOT FOLD, MAJOR); annular.state (NOT FOLD, ANNULAR); auricular.state (NOT FOLD, EARLY); return OK; 1 Hand :: fingerlnvalid (TypeTypetype) {// Invalidate the indicated finger because no sensor or no monitoring desired switch type {178BULL-35455-patent-draft2 15 INCH: inch.state (INACTIVE, INCH); INDEX: index.status (INACTIVE, INDEX); MAJOR: major.state (INACTIVE, MAJOR); ANNULAR: annular.status (INACTIVE, ANNULAR); AURICUALIRE: auricular.status (INACTIVE, AURICULAR); default: return ERROR; 1; return OK; 1; // Full hand state test function int Hand :: testPliement () {// All fingers must be folded so that the state of the hand is PLIE // Inactive fingers are not taken into account if ((thumb.testStatus () == BRIEF) II (inch.testStatus () == INACTIVE) && ((index.testState () == BOLD) II (index.testStatus () == INACTIVE) && (( major.testState () == BRIEF) II (major.testState () == INACTIVE) && ((annular.testState () == BOLD) II (annular.testState () == INACTIVE) && ((auricular.testStatus ( ) == FOLD) II (auricular.testState () == INACTIVE))) {state = PLY; 1 else if ((thumb.testState () == ERROR) H (index.testState () == ERROR) H ( major.testState () == ERROR) II (annular.testStatus () == ERROR) 178BULL-35455-patent-draft2 16 It (auricular.testStatus () == ERROR)) {state = ERROR;} else {state = NOT FOLD; 1 1 return state; I; A software application A can interact with a function to react to the gesture of the interactive glove associated with said function An example of use by an application Software Factor Function F10 of the API API may be as follows: #include <iostream> #include <APIGant.h> int main () {// Waiting time before complete folding hand static timeOut time = 0; // Only a right hand is equipped Hand right (RIGHT); // No sensor for ring finger righthand.doigtInvalid (ANNULAR); // No sensor for the little finger 178BULL-35455-patent-draft2 17 righthand.docentInvalid (EARLY); // Using the full hand folding function // We wait for the complete folding event before exiting while (righthand.testPliement () <> PLIE) {if (righthand.testPliement () == ERROR) {cost "" Error on the hand folding test after "Wait time" second (s) "; return ERROR; } else {cost "" The hand is still not folded "; // We wait 2 seconds to allow the time to capture the event // and not to monopolize the processor in an infinite loop sleep (2); waiting time + = 2; 1 1 cost "" The hand was bent after "Wait time" second (s) "; return OK; Of course, the present invention is not limited to the examples and the embodiment described and shown, but it is capable of many variants accessible to those skilled in the art.

Claims (15)

REVENDICATIONS1. Application programming interface (API) for interactive glove (G) comprising a list of functions (F1, F2, F3) associated with possible gestures by said interactive glove, in which a combination of sensor values (C1, C2, C3, C4) associated with said interactive glove makes it possible to determine output values of said functions, and in which a software application (A) can interact with a function of said list to react to the gesture of said interactive glove associated with said function. 2. Application programming interface according to the preceding claim, wherein said output values indicate a duration of said gesture. Programming interface according to one of the preceding claims, wherein said output values indicate a pressure. The application programming interface of claim 1, wherein said output values are boolean values. 20 The application programming interface according to one of the preceding claims, wherein said functions include an implementation part for mapping between said sensor values and said output values, said values being representative of the associated gestures. to said functions. 6. Application programming interface according to the preceding claim wherein said functions are grouped into families of gestures. 30178BULL-35455-patent-draft2 19 7. A method of developing a software application (A) interfaced with an interactive glove (G) comprising a step of using an application programming interface (API) comprising a list of functions (F1, F2 , F3) associated with possible gestures by said interactive glove, in which a combination of sensor values (C1, C2, C3, C4) associated with said interactive glove makes it possible to determine output values of said functions, so that in operation , a software application (A) can interact with a function among said list to react to the gesture of said interactive glove associated with said function. 8. A method of developing a software application according to the preceding claim, wherein said output values indicate a duration of said gesture. The method of developing a software application according to one of the preceding claims, wherein said output values indicate a pressure. The method of developing a software application according to claim 7, wherein said output values are Boolean values. 11. A method of developing a software application according to one of the preceding claims, wherein said functions comprise an implementation part for matching between said values of the sensors and said output values, said values being representative of the gestures. associated with said functions.178BULL-35455-patent-draft2 20 12. The method of developing a software application according to the preceding claim wherein said functions are grouped into families of gestures. An information processing system (ST) comprising at least one software application (A) and an application programming interface (API) according to one of claims 1 to 6. 14. System comprising an information processing system (ST) according to the preceding claim and further an interactive glove (G) linked to said information processing system by a communication means (MC). 15. System according to the preceding claim, wherein said communication means (MC) is wireless.