WO2023150849A1

WO2023150849A1 - Device and system for controlling electronic interfaces

Info

Publication number: WO2023150849A1
Application number: PCT/BR2023/050045
Authority: WO
Inventors: Adriano RABELO ASSIS; Henrique PIRES FRANCO LATORRE
Original assignee: Tix Tecnologia Assistiva Ltda
Priority date: 2022-02-09
Filing date: 2023-02-09
Publication date: 2023-08-17
Also published as: BR102022002441A2

Abstract

The invention relates to a computing and assistive technology device and system for capturing and processing head movements and facial expressions by means of infrared movement and inertia sensors which activate control of an electronic interface. It is a compact, wireless and adaptable device which can be connected to any headgear, eliminating the need for external accessories, and it works on any equipment having an operating system, the apparatus to be controlled only requiring Bluetooth connectivity. It is a hybrid solution comprising an embedded system that identifies users' movements and processes the ten control commands; a web system that parameterises the capture of users' movements; and a device. The invention is designed for people with reduced mobility who are unable to use ordinary computer peripherals, promoting independence and inclusion at work, school and leisure.

Description

DESCRIPTION REPORT

DEVICE AND SYSTEM FOR CONTROLLING ELECTRONIC INTERFACES

FIELD OF INVENTION

[001] The present invention concerns a new device and system to capture and process head movements and facial gestures, through infrared motion and inertial sensors that activate the control of electronic interfaces, through connectivity via Bluetooth, working on any device equipped with an operating system, regardless of brand or model. It is a hybrid system solution for controlling electronic interfaces, fully configurable, which replaces computational peripherals such as control, mouse and keyboard. It has a very useful application as a device adapted to people with disabilities, who cannot use common computational peripherals to control equipment that has electronic interfaces such as computers, tablets, smartphones and even televisions and other equipment. Furthermore, the equipment is substantially compact and can be easily attached to any headgear, such as headsets, glasses, headbands or caps. In this way, the present invention, which dispenses with the use of external accessories, webcams and hand and arm movements, is more versatile, customizable and efficient than other products on the market, since it does not depend on ambient conditions, such as quality of lighting, can be added to any headgear, is not limited to a single device or operating system and can be adapted according to the needs of use or limitations and disabilities of each user.

[002] Among the most direct applications of the aforementioned invention, there is the replacement of traditional interface control devices, such as mice, keyboards and controls, ensuring electronic accessibility. Thus, the device fits and has application in the IT and assistive technology sector, with the purpose of guaranteeing a structure to promote independence and work, recreational and school inclusion of people with physical and motor limitations.

FUNDAMENTALS OF THE INVENTION

State of the art [003] In today's market, there are three categories of high-tech solutions designed to make mobile personal computing devices accessible to people who don't move from the neck down: accessibility via software, accessibility via hardware, and accessibility via software combined with hardware.

[004] Software-based solutions include programs that use front cameras built into the device to be controlled, such as a computer, to capture users' head movements and transform them into movement equivalent to that of a screen cursor.

[005] Hardware-based solutions, in turn, have physical components equipped with electronic sensors - whether bioelectric, such as electromyography and electroencephalography; inertial ones, such as infrared; distance, gyroscope or accelerometer - whose function is to capture intentional movements of the user's head and/or face and transform them into commands for the devices that are intended to be controlled.

[006] Hybrid solutions, on the other hand, are those that have hardware, where one or more of the aforementioned sensors or a camera with infrared lighting are located, whose identification of the users' head movement is interpreted and processed by software.

[007] It should be noted that, in the aforementioned state of the art, most solutions are limited to computer control, given that they depend on USB cables to transmit the signals captured by the sensors or cameras to where they will be processed and transformed into interface control movements.

[008] In addition, although, as a rule, solutions that rely on sensor-based hardware (including hybrid ones) present a better performance in capturing physical and motor signals from users (since they do not depend on ideal environmental conditions, such as lighting quality), compared to software-only solutions, it is common for these devices to require the assistance of external accessories, such as joysticks adapted for mouth control of the mouse (e.g., the cursor clicks from the user's biting movement), large physical buttons activated with head blows and devices attached to caps, glasses, headbands or other pre-defined headsets. [009] In view of the above, to exemplify what has been demonstrated as the current state of the art, examples of patents are available that present one or more of the technical descriptions mentioned above, in addition to problems and/or limitations that are resolved for the invention of the present deposit.

(i) Patent application No. PI 1004279-2: this is a device whose technical approach is electromyography to be implemented in a band on the user's forehead to capture their movements through sensors composed of a piezoelectric , which will later be processed in the control system.

(ii) Patent application No. BR 102018 071412: representative of a device and method to capture, process and classify eye movements, head movements and brain activity, through the electroencephalography approach.

(iii) Patent application No. BR 10 2014 025534 6: it is a system composed of a camera accompanied by a cursor control application that allows its movement in the electronic interface and is located in front of the user's lower lip .

(iv) Patent application No. WO2010064987: describes a device that uses a "mouse ball" to capture the movements of the mouth, tongue and lips to select keyboard commands and a headset in the form of an arc or strap around the forehead.

(v) Patent application No. WO2015159108A2: describes glasses that detect the movements of the user's head and transform them into cursor movements on the screen of an electronic device, through connection via USB.

(vi) Patent application No. W02020044363A1: representative of wireless glasses with mouse and keyboard function. The device includes an eyeglass frame with lenses to cover the wearer's eyes, a sensor to detect eye blinking and a sensor to detect head movements. The connection is made wirelessly via Bluetooth. A mouse click is detected by means of a wink. The head motion sensor has an accelerometer and a gyroscope. (vii) Patent application No. KR20210067149A: representative of a mouse-glasses system and mouse cursor operated based on the rotation movement of the head of the wearer of the glasses. The glasses capture the movement of the head by means of acceleration and gyroscope sensors, which are located on the left side of the frame of the glasses, and communication with the computer is carried out using l ² C technology.

(viii) Patent Application No. US4486630: representing a head accessory operated by movements of the chin and eyebrows, which allows the use of computers and video game consoles by people with limited movement. The capture of movements is performed through electrical connections, more specifically through electrodes connected to a computer signal.

[010] The present invention, in turn, differs from those described in items "i", "ii" and "ii i", as it does not use the methodology of bioelectric capture, but infrared inertial sensors. It also differs from the invention described in item “viii”, which uses electrical movement capture by means of electrodes.

[011] In addition, the present invention differs from that described in item "v", as it captures the movements of the user's head through the combination of two different types of sensors: (i) a digital level sensor, used to detect the vertical movement of the head, and (ii) a digital compass, responsible for detecting the lateral inclination of the head; while the former uses infrared motion and inertial sensors.

[012] It also differs from the invention of item “iv”, as it does not require the use of external accessories, such as the aforementioned “mouse ball” and from the invention of item “v”, since it has Bluetooth connectivity with the device that if you want to control, and not via USB.

[013] The present invention is also distinguished from the inventions described in items "v" and "viii", as the first has its connection limited to devices that have a USB connection and the second has its connection limited to computers and video game consoles, while the invention described here can be connected to any device that has connectivity via Bluetooth and operating system. [014] In addition, unlike the inventions of items "v", "vi" and "vii", the present invention is not composed of a glasses frame, being a device that can be attached to any head or face accessory of user.

[015] Also, the invention described in item “v” does not allow the user to click on the mouse, but only direct the mouse cursor.

[016] As for items "vi" and "viii", the present invention differs, as can be seen from the explanation below, by not limiting the capture of users' movements to one or more focal points which, in the case of the invention “vi” are the head and eyes, and in the case of the invention “viii” are the chin and eyebrows.

[017] As you can see, the variety of focal points used by the present invention is one of the characteristics that makes it innovative in the face of the assistive technology market. Namely, the device object of the present filing request is the only one that can be attached to any head accessory, including those owned by the user, and this versatility is extended to the technology used in the invention. This set of claimed adaptable characteristics therefore allows the device to be adapted according to different motor limitations of users, who can choose which part of the face the sensors will focus on (for example: eyes, mouth, chin, eyebrow, cheek and jaw), just position the device and configure the optional web system, according to your preferences.

[018] Thus, the way in which the embedded system, together with the configurations of the web system, allows modifying the operating characteristics of the invention (the operating modes, the characterization of gestures and the operating parameters) without, however, modifying its constructive characteristics, is the main difference between the aforementioned invention and the other products on the market, since in other solutions the firmware is written in such a way as to always rigidly execute the same instructions, when faced with the same set of circumstances. On these devices, for example, if the person tilts the head towards the right shoulder, the non-adaptive firmware will always send the same mouse command to the connected device (such as moving the pointer to the right). In the case of the present invention, this same gesture can make the device issue different commands, such as right-clicking, scrolling down the screen, moving the pointer to the right or even doing nothing, depending on the previous configuration. chosen by the user. This means that if the user changes the configuration, the present invention also changes its behavior, which demonstrates its high adaptability to the needs of each user.

[019] In this way, it is possible to meet, with the same hardware product, due to the system, numerous different situations of use, significantly modifying the operation of the device according to the needs of each user, without, however, changing the constructive characteristics of the invention.

[020] This characteristic of versatility also allows the invention to be used as a control tool for other machines equipped with an operating system, in applications that can go far beyond accessibility, such as the control of intelligent vehicles and drones, for example.

Technical problem

[021] The device object of this patent was created in order to solve numerous problems of the current state of the art, mainly with regard to versatility and adaptation to the particularities of use of this type of equipment, including disability and/or aesthetic tastes of each user. Therefore, the invention in question goes beyond the already known assistive technologies in the field of human-computer interaction commands, guaranteeing the versatility of its device, both in terms of aesthetics and practice, as well as its technological function itself.

[022] This is because solutions based only on software necessarily depend on the ideal environment conditions and on capturing the image of their users, which means that, in order to use them, the user needs to have, in addition to program, a webcam. And, regardless of whether the software solution requires or not the use of a particular webcam, wired or wireless, for example, the fact is that the user will need to purchase an extra accessory and, even so, will be facing a device with little technical precision. , bearing in mind that the solutions, in this case, depend on the ambient lighting, the position of the user's head and face in front of the webcam and the quality of the camera.

[023] The present device, in turn, adopts an inertial electronic sensor, which records variations in acceleration in any direction and, as acceleration is a physical quantity that is independent of factors such as ambient lighting, unlike solutions based on software or hardware and hybrid ones that use cameras to capture the user's movements, the light does not interfere with the detection of the movements of those who use the invention deposited herein.

[024] Furthermore, the fact that it is a computer program means that assistive technology solutions in the form of software are conditioned to specific operating systems, which means that the program can work perfectly on a computer with an operating system of Windows 7, but not on an iOS computer, which can be even more difficult on other types of devices, such as smart TVs and tablets. This is what happens, for example, with the Camera Mouse product.

[025] Hybrid and hardware solutions, in turn, are divided as technology of sensors embedded in wearable accessories or dedicated systems with infrared lighting and specialized software, and even so, they have one or more of the following characteristics, which make their use less convenient for their users:

(i) rely on wires or cables for connection to the computer or mobile device to be controlled (eg WO2015159108A2);

(ii) depend on additional external accessories (known as triggers), for the user to perform some specific functions, such as mouse clicks, such as bite keys (special buttons made to be positioned between the person's teeth) or physical buttons triggered with head strikes (ex.: patent US4486630);

(iii) are built directly into a standard piece of clothing, such as glasses or a headset, forcing the user to wear a garment on their face or head that does not necessarily match their style preferences or even is incompatible with the use of their own glasses, hats, headbands and other accessories (eg US patent US4486630);

(iv) working together with software that is only compatible with a specific device and operating system; It is (v) limiting the reading of the user's face signals to a single focal point, such as reading only the movement of the iris or mouth (eg: patents BR 102014011226-0 A2 and WO2010064987).

[026] Given the above, it is clear that assistive technology solutions, whether in the form of software, hardware or hybrid, still have aesthetic and technical-functional limitations, which generate economic and practical disadvantages for their users. Thus, through a unique, new technology, never seen in another product on the market in question, the present device and system presents an innovative differential in human-computer interaction.

[027] This is because the invention is the only one on the market to face the problems of the current state of the art, since it guarantees compatibility with any device equipped with an operating system and Bluetooth (eg computers, smart TVs, tablets, phone etc.) and identifies users' head movements and facial gestures in real time, without causing discomfort and without the need for additional external accessories such as webcams, wires, headbands, glasses or other headsets, and can adapt to any accessory of head or face. This even allows the user to choose which part of the face will dictate the control commands (eyes, mouth, eyebrow...) and, consequently, where the sensor will be positioned, which means that the present invention adapts according to the level of the motor deficiency of its users, a completely new fact for the state of the art.

BRIEF DESCRIPTION OF THE FIGURES

[028] The attached drawings exemplify the assistive device and its components, as well as examples of use and user movements, which are interpreted as electronic interface control commands.

[029] Figure 1 . Invention cabinet attached to a glasses frame - opposite side to the user's face, indicating the items:

[030] (a) Glasses frame; (b) Joint between frame “a” and temples “i” of the glasses; (c) Opening for connecting the USB cable for charging, which lines up with the female USB connector when the printed circuit board (PCB) is fully inserted; (d) Hollow ABS plastic cabinet, in the form of a rectangular prism, with dimensions of 50mm in length, 20mm in height and 18mm in width, with walls between 1mm and 5mm thick, custom-made to house the PCI; (e) Flexible fin/rectangular ABS plastic cover only 0.3mm; (f) 2.5mm diameter flexible steel wire covered by a plastic cover, which forms a rod that connects the removable clip “k” to the cabinet “d”; (g) Space between the flaps of the mobile clip where the wire described in “f” is attached; (h) Removable clip tabs; (i) Spectacle arm; and (j) Spectacle lenses.

[031] Figure 2. Cabinet of the invention coupled to a frame of glasses - side facing the user's face, indicating the items:

[032] (b) Joint between frame “a” and temples “i” of the glasses; (d) Hollow ABS plastic cabinet, in the form of a rectangular prism, with dimensions of 50mm in length by 20mm in height and 18mm in width, with walls between 1mm and 5mm thick, custom-made to house the PCB; (e) Flexible fin/rectangular ABS plastic cover only 0.3mm; (f) 2.5mm diameter flexible steel wire covered by a plastic cover, which forms a rod that connects the removable clip “k” to the cabinet “d”; (g) Space between the flaps of the mobile clip where the wire described in “f” is attached; (h) Clip tabs; (i) Spectacle arm; (j) Spectacle lenses; (k) Removable clip; (I) Longitudinal rail along which the base of the clip described in “o” slides; (m) Square opening of 2mm on each side, positioned on one of the surfaces of the cabinet, intended to let the infrared light beam out of the sensor and to allow the entrance of reflected light; (n) A light indicator, characterized by a round hole of 1 mm in diameter through which the RGB LED light can be seen when turned on; and (o) Removable clip base, present at one end of the flexible rod described in “f”.

[033] Figure 3. Position of the invention on the user's face in glasses frame, indicating the items:

[034] (k) Removable clip; (s1 ) Eyebrow; (i) Spectacle arm; (f) 2.5mm diameter flexible steel wire covered by a plastic cover, which forms a rod that connects the removable clip “k” to the cabinet “d”; (a) Glasses frame; (d) Hollow ABS plastic cabinet, in the form of a rectangular prism, with dimensions of 50mm in length by 20mm in height and 18mm in width, with walls between 1mm and 5mm thick, custom-made to house the PCB; (s7) Lip; (s8) Chin; (s6) Jaw; and (t) Computer. [035] Figure 4. Position of the invention on the user's face in a cap type hat, indicating the items:

[036] (d) ABS plastic cabinet, in the form of a rectangular prism, with dimensions of 50mm in length by 20mm in height and 18mm in width, with walls between 1mm and 5mm thick, custom-made to house the PCB; (e) Flexible fin/rectangular ABS plastic cover only 0.3mm; (f) 2.5mm diameter flexible steel wire covered by a plastic cover, which forms a rod that connects the removable clip “k” to the cabinet “d”; (g) Space between the flaps of the mobile clip where the wire described in “f” is attached; (k) Removable clip; (p) Cap; and (q) Cap brim.

[037] Figure 5. Head movements around orthogonal axes and facial gesture capture areas:

[038] (u) Horizontal movement of the head, which characterizes and informs a control command; (v) Affirmative movement of the head, characterized by rotation with alternation of direction around the axis, which characterizes and informs a control command; (r) Rotation of the head towards the right shoulder, which characterizes and informs a control command; (s1 ) Eyebrow; (s2) Eyelid; (s3) Pupil; (s4) Cheek; (s5) Mouth; (s6) Jaw; (s7) Lip; and (s8) Chin.

[039] Figures 6 and 7. Interfaces of the web system, in which the user of the invention can adjust it, according to his preferences and limitations.

[040] Figure 8. Web system flowchart.

[041] Figure 9. Embedded system flowchart.

DESCRIPTION OF THE INVENTION

[042] The invention is a system composed of a device, an embedded system and a web configuration system. The Device has a microcontroller chip that operates as the "brain" of the device, this microcontroller runs an embedded system (Figure 9) in which all the operations that the device must perform are programmed, and how it must be performed.

[043] The device contains a motion sensor with 6 axes. When the device is in operation and properly attached to a head garment or on the user's face, the motion sensor continuously measures acceleration and rotation orientation in each of its 6 axes, at an approximate rate of 1,000 measurements per second. This sequence of instantaneous measurements is converted into digital values and constantly sent from the motion sensor to the microcontroller, which runs a sophisticated embedded system (Figure 9), responsible for digitally processing these values, in real time, to determine the instantaneous variation. measurements on each axis.

[044] This embedded system combines, at all times, the values sent by the motion sensor to determine if there is a significant change in the position and orientation of the device (and therefore, if the position of the user's head is changing). If so, the embedded system then calculates in which direction the change has taken place and at what speed and determines how large the mouse pointer should be to shift (in pixels) across the screen and in which direction.

[045] The microcontroller then creates a digital package with this data (direction and displacement in pixels), respecting the format specified by a digital protocol for human interface devices (HID protocol). This package is encapsulated within another digital data package determined by the Bluetooth communication protocol and, finally, the data is transmitted via radio frequency to the connected device, where its operating system (eg Windows, Android, iOS, Linux) runs the received command and moves the pointer on the screen.

[046] As the motion sensor registers, at all times, any slight variation in the position of the device, in any direction, the embedded system is able to identify the position in which the user's head is located and, thus, identify predetermined patterns. determined in these movements and, with that, recognize common head gestures, such as the repetitive head nodding up and down (Figure 5 - v) that signals “yes”, the nodding from side to side (Figure 5 - u ) that signals “no”, among other movements indicated in the GESTURE CHARACTERIZATION, below.

[047] The embedded system, in turn, is responsible for instructing the microcontroller to process, in real time, the information sent by the sensors (variations in acceleration measured by the motion sensor and variations in the reflection of infrared light measured by the sensor of infrared) and, based on these numbers, identify whether the user is making any head movements or facial gestures intentional. When these movements or gestures are recognized by the embedded system, the microcontroller sends the corresponding mouse commands to the connected device via Bluetooth.

[048] As the system determines what a valid head movement or facial gesture is and decides which mouse command will be sent via Bluetooth to the connected device, the innovation of the present invention in relation to the state of the art was to develop a system adaptive, in order to allow the user to change the way the system itself identifies movements and gestures, and what to do when they are recognized. Thus, the technical effect of the system applied to the device lies in the usability of the present invention, which interprets head movements and facial gestures in different ways, enabling its use by people with different degrees of head and face mobility.

[049] Thus, a user unable to move his head quickly can change the behavior of the invention so that a short head movement triggers a large displacement of the cursor, compensating for his physical disability. In another example, a person who has trouble blinking their eyes vigorously can change the way the device works so that it detects more subtle movements of their eyelid.

[050] Through its web configuration system (Figures 6 and 7), the invention allows the user to assign frequent commands to certain head or face gestures, using them as shortcuts to increase productivity. For example, if a person needs to use the right mouse button frequently during work, he can configure the device to perform a right click whenever he tilts his head towards the right shoulder (Figure 5 - r), since it's a quick move to make.

[051] In addition, the device is also capable of executing any command possible on a conventional keyboard, from a simple key (for example, “Enter”), to a combination of keys (for example, “Ctrl + C”). The command can be freely chosen by the user and assigned to any of the facial or head gestures identifiable by the invention, these choices being made through the optional web configuration system. [052] In other situations, adjustments to the parameters of the invention may be intended to increase the comfort of use or even be fundamental to enable a person with very particular motor characteristics to actually be able to use a computer (Figure 3 - 1) or a cell phone.

[053] An increase in the pointer speed parameter (Figure 6) (which, in practice, is the amount of displacement in pixels reported by the invention for the connected device) is an example of an adjustment that can provide greater comfort of use, as starts to require a smaller movement of the head to follow the same path as the cursor on the screen.

[054] The change in the “operation mode” parameter of the invention (Figure 7), from “standard mode” to “virtual joystick mode”, for example, drastically modifies the relationship between head movement and corresponding mouse movement, giving accessibility to people who have little cervical control or little range of head movement (the modes of operation of the invention will be detailed below).

[055] The basic technology involved for the connection between the web system and the device is the Bluetooth communication protocol, that is, the same principle already used to pair the device with the interface to be controlled. What changes are the types of data packets sent and received by the invention when the web system is running.

[056] So that these modifications can be made, the embedded system of the invention has a limited number of parameters that can be changed by the user. As the device itself does not have a display, touch screen or buttons to allow the adjustment of these parameters, the web configuration system works as a friendly interface for the user to make adjustments in its operation.

[057] During the process, the web system (Figure 8) sends a data packet to the invention requesting the current values of its parameters. The embedded system of the invention then recognizes this request and sends a data packet containing the current value of each configurable parameter as a response (Figure 9). Thus, when the web system is running on a device that is paired with the invention via Bluetooth, such as a cell phone, the program is able to display, on the screen, the current settings of the invention being used. [058] When the user changes any of these parameters on the screen, the web system sends a data package to the device containing a command instructing it to replace the value of that parameter with the new value selected by the user. The embedded system then recognizes this request and makes this change, saving the new values internally in the microcontroller's non-volatile memory. In this way, the new operating parameters of the device will remain in effect even if the device is turned off or its battery is completely discharged.

[059] The above is due to the electronic circuits, which are capable of performing the task proposed to the invention within a compact volume, with a total weight of up to 30g, the invention being made up of the elements mentioned below and whose technical and constructive characteristics are detailed:

[060] 1. A microcontroller of at least 8 bits, in the form of an integrated circuit (microchip) (hereinafter “microcontroller”), capable of running embedded systems at a speed of 48 MHz, with integrated radio frequency communication capability in the 2.4 GHz, compatible with Bluetooth Low Energy 5.1 communication protocol. The microcontroller is responsible for running the embedded system with the pre-programmed instructions in which a system for processing the signals coming from the sensors is implemented, the “translation” of these signals into mouse commands and the sending (via Bluetooth) of these commands to the devices to be controlled.

[061] The adaptive embedded system, which is the technology that allows the present invention to interpret the movements of each person differently. The process takes place through the device's firmware, an embedded system that instructs the microcontroller on how to calculate the values measured by the motion sensor and by the infrared sensor in order to identify patterns that represent predetermined intentional gestures - for example, the pattern of repetitive movement of the head vertically representing an affirmative gesture characterized by repetitive rotation with alternation of direction (Figure 5 - v) -, in addition to sending, via Bluetooth, to the connected device, the control commands corresponding to the identified gestures.

[062] Because the aforementioned system is adaptive, it allows the reconfiguration, on demand, of both the criteria for identifying movements and gestures, as well as the mouse commands (or even keyboard) to be sent to the computer (Figure 3 - t) or mobile device. That is, as a consequence of carrying out these actions, the general behavior of the system is thus altered, without, however, altering any of its constructive characteristics. In summary, the technical effect of the embedded system of the present invention lies in the possibility of the program to reconfigure, on demand, its movement identification thresholds and several other parameters, thus changing the device's behavior according to each user.

[063] 2. A 6-axis MEMS (Micro Electro-Mechanical System) motion sensor (hereinafter "motion sensor"), integrating a 3-axis accelerometer and a 3-axis gyroscope, integrated into a single microchip.

[064] 3. An infrared light emitting and detector integrated circuit (hereinafter "infrared sensor"), capable of emitting a beam of non-visible light pulses and measuring variations in the received light reflection, thus making it possible to detect the variation of distance between the sensor and surfaces illuminated by the beam. The sensor is also capable of measuring visible ambient light (such as sunlight) and eliminating noise and electromagnetic interference caused by it, increasing its accuracy and allowing the detection of variations of less than 1 millimeter in the distance between the sensor and the illuminated surface. by the infrared beam.

[065] To work correctly, the invention must be attached to the garment that the person is wearing over the head or face, in such a way that the opening of the device's infrared sensor (Figure 2 - m) is pointed towards the region of the face that the user intentionally wants to move to perform the clicks. Therefore, if the person wishes to make clicks by blinking the eyes, the opening of the sensor (Figure 2 - m) must be pointed towards the eyelid (Figure 5 - s2) of the user when, for example, the glasses (Figure 3 - a) are positioned on the face. The distance between the sensor aperture and the facial skin needs to be between 10mm and 50mm.

[066] In this example, with the device already in operation, the infrared light beam emitted by the sensor will focus on the eyelid. During each blinking movement of the eye, the variation in the distance between the eyelid and the opening of the sensor will cause a corresponding change in the amount of infrared light reflected back to the sensor, which numerically registers this reflection oscillation and passes these information to the microcontroller, whose embedded system tries to recognize patterns in these variations and identifies whether or not this corresponds to a user's intention to perform a mouse click.

[067] If the embedded system identifies the intent pattern, the device sends the command equivalent to a mouse click to the device to which it is paired via Bluetooth.

[068] Following this logic, any other intentional facial gesture can be captured (such as a smile, cheek contraction or eyebrow lift), simply by positioning the infrared sensor opening towards the part of the face you want to move, respecting the distance between 10mm and 50mm. In order to capture more subtle facial movements, such as a slight blink of the eyelid (Figure 5 - s2), eyebrow movement (Figure 5 - s1 ) or even pupil movement (Figure 5 - s3), the sensor must be closer (saving 10mm minimum distance). To capture wider facial movements, such as mouth opening (Figure 5 - s5), cheek movements (Figure 5 - s4), jaw (Figure 5 - s6), chin (Figure 5 - s8) or lip movements (Figure 5 - s7), the sensor can be further away, but not exceeding 50mm.

[069] To perform a finer adjustment of the detection of intentional facial gestures, the sensitivity parameter of the infrared sensor can be changed through the invention's optional configuration web system (Figure 6).

[070] 4. A Lithium polymer battery (Li-po) with integrated charge and discharge control circuit and short circuit protection. The internal rechargeable battery is composed of Lithium polymer cells, which have the property of storing electrical charges, discharging these charges in the form of electrical energy when connected to circuits and, upon receiving an electrical current supplied by an external source in reverse direction, store these charges again, being able to execute this cycle hundreds of times. The nominal voltage of the battery is 3.7V and it has at least 350 mAh of charge capacity (enough to keep the invention working for more than 30 hours of continuous use). It is recharged when a continuous voltage of 5V is applied between its terminals, which can be done safely by connecting any standard 5V source via cable to the invention via the micro USB or USB type C connector. of an integrated circuit that automatically performs the safe control of battery charging, protecting it in case short-circuit between its terminals, in addition to monitoring its voltage while using the device and identifying when the charge level is low. When this happens, this integrated circuit sends a signal to the microcontroller and, then, the embedded system instructs the light indicator (LED), the sound emitter and/or the vibration motor of the invention to signal to the user that the battery is low. . The micro-USB or USB type-C connector is for connecting a standard USB cable for battery charging to any standard 5V outlet or USB port, with access to the connector through the opening (Figure 1 - c) in the case.

[071] 5. An RGB type LED (light emitting diode) light indicator (capable of turning on, independently, a red, a green and a blue light, or any combination of these colors, within the same package) serving, in the practical, as a visual cue point that can light up in any color of the visible light spectrum. The purpose of this LED is to provide the user with a visual signal that the invention is working, as described below. The light can be seen through the component (Figure 2 - n) present in the cabinet.

[072] 6. A compact direct-current vibration motor, popularly known as “vibracall”, for being present inside cell phones of all brands and models. The vibration motor is a small DC rotary motor that moves an unbalanced load attached to its axis of rotation. When the engine is started, the load imbalance causes the entire engine mechanical assembly to oscillate with the same frequency as its rotation, causing an intentional vibration. The purpose of this engine is to provide the user with haptic signaling of the device's operation, see description below.

[073] 7. An integrated sound emitter (hereinafter “buzzer”), capable of emitting sounds in the spectrum from 1 kHz to 4kHz. The “buzzer” is a small integrated circuit capable of emitting sounds in the spectrum from 1 kHz to 4 kHz, and its objective is to provide the user with a sound signal of the operating state of the invention. By "operating state", it is meant what the invention is performing or is capable of performing at any given moment. Operating states can be indicated by light signaling (RGB LED), by sound signaling (buzzer) and/or by haptic signaling (vibracall).

[074] The invention can assume the following operating states: a) Off: light off, buzzer in silence, vibracall stopped; b) Off, but battery being recharged via USB cable: light off, buzzer silent, vibracall stopped; c) On, not connected via Bluetooth to any device: light flashing, alternating between green and red, buzzer silent, vibracall stopped; d) On, not connected via Bluetooth to any device, battery being recharged via USB cable: light flashing red, buzzer silent, vibrate stopped; e) On, connected via Bluetooth to a device, activated (that is, able to move the mouse pointer and click if requested by the user): light on green, buzzer in silence, vibrate stopped; f) On, connected via Bluetooth to some device, temporarily disabled by user choice (i.e. cursor movement and mouse clicks are temporarily suspended - feature used when person wants to move head or make facial gestures without causing commands on screen): light off, buzzer beeps 2 times in a row, vibracall beeps 2 times in a row; g) On, connected via Bluetooth to a device, recently reactivated by user choice: light on green, buzzer beeps 1 time, vibracall beeps 1 time; h) On, connected via Bluetooth to a device, activated by clicking the left mouse button: light on green, buzzer signals while the click lasts, vibracall signals at the beginning of the click; i) On, connected via Bluetooth to a device, activated, clicking the right mouse button: light on green, buzzer beeps three times in a row, vibracall beeps once at the beginning of the click; j) On, connected via Bluetooth to a device, activated, scrolling the mouse vertically upwards: light on green, buzzer signals 1 time at the beginning of scrolling, vibracall signals 1 time at the beginning of scrolling; k) On, connected via Bluetooth to a device, activated, scrolling the mouse vertically downwards: light on green, buzzer signals 1 time at the beginning of scrolling, vibracall signals 1 time at the beginning of scrolling; l) On, connected via Bluetooth to a device, activated, executing a keyboard command pre-configured by the user: light on green, buzzer beeps three times in a row, vibracall beeps once; m) On, connected via Bluetooth to a device, activated, with less than 10% of battery charge: light on red, buzzer beeps ten times every 5 minutes, vibracall beeps for 2 seconds once every 5 minutes, together with with the buzzer; n) On, not connected via Bluetooth to any device, with less than 10% battery charge: light on red, buzzer silent, vibrate stopped;

[075] 8. A simple push button (hereinafter “button”), intended to turn the invention on and off. The button is activated by means of a flexible flap (Figure 1 - e) on the cabinet.

[076] 9. A rectangular printed circuit board (PCB), 42mm long by 18mm wide, where all the above components are soldered, as well as the auxiliary electronic components of the device (such as capacitors and resistors). The pushbutton is soldered to one end of the board, with the key perpendicular to the surface of the PCB. With the components assembled, the electronic board reaches a maximum height of 11 mm.

[077] 10. A hollow ABS plastic cabinet (Figure 1 - d), in the shape of a rectangular prism, with dimensions of 50mm in length by 20mm in height and 18mm in width, with walls between 1mm and 5mm thick, made under dimension to house the printed circuit board with all the electronic components mentioned above. The case has an opening at the rear, with which the soldered button on the end of the PCB aligns with when the card is fully inserted. The opening is closed by a rectangular cover made of ABS plastic, only 0.3mm thick, which, being thin, makes it flexible. When glued over the opening, this cover seals the case, covering the button key, but allowing it to be actuated by pressing the rectangle, flexing its surface towards the inside of the case. At On the top face of the case, there is an opening (Figure 1 - c) that aligns with the female USB connector when the PCI is completely inserted, allowing the insertion of the USB cable for charging the device's internal battery. The plastic cabinet also has a square opening (Figure 2 - m), 2mm on a side, positioned on one of the surfaces of the cabinet, designed to let the infrared light beam out of the sensor while allowing the entry of reflected light. On the same face of the cabinet, there is also a round hole, 1 mm in diameter (Figure 2 - n), through which the RGB LED light can be seen when turned on. Still on this side of the cabinet, there is a longitudinal rail (Figure 2 - 1) through which the base of the clip (Figure 2 - o) removable from the device slides.

[078] 11. A removable clip (Figure 1 - k), made of two pieces of ABS plastic connected by means of a flexible steel wire of 2.5mm in diameter covered by a plastic cover. The length of the wire can be 30mm (to mount the device on the arms (Figure 1 - i) of eyeglasses frames (Figure 1 - a)) or 80mm (to facilitate the mounting of the device on other garments, such as caps , hats or headbands). The plastic piece (Figure 2 - o) present at one end of the flexible rod (Figure 2 - f) has a trapezoidal and rectangular prismatic shape, 25 mm long, 4 mm high and 5 mm wide. This piece can be slid under the longitudinal rail (Figure 2 - 1) of the plastic cabinet, allowing the clip (Figure 2 - k) to be easily fitted or detached from the cabinet (Figure 2 - d) of the device. The plastic piece present at the other end of the flexible rod is a clip, made of ABS plastic in the shape of a paper clip, with dimensions of 30x24x15mm, and serves to attach the set to the user's garment (glasses rod (Figure 2 - i), cap brim (Figure 4 - q) etc.).

[079] The device can be attached, for example, to any eyeglass frame, with or without lenses (Figure 2 - j), aligning the plastic clip on the right stem (Figure 2 - i) of the frame, close to the junction between the stem and the window of the glasses, pressing its tabs (Figure 2 - h) against each other to open its lower part and fitting the stem into this opening (Figure 2 - g), securing it by pressure. In this position, the opening (Figure 2 - m) of the device's infrared sensor will be aligned with the user's right eyelid (Figure 5 - s2) when the glasses are positioned on the face, capturing the blink of the eye as a means for the user to click . If the person wants to move another part of the face to perform this command, such as twitching the cheek, for example, just flex the steel rod (Figure 2 - f) so that the sensor opening points to that region.

[080] Similarly, the loop of the invention can be attached to other parts, such as rigid brims (Figure 4 - q) of a cap (Figure 4 - p) or hat, headsets or hair bands, and provide the capture of the gesture user's preferred face, simply flexing the rod (Figure 4 - f) in order to point the infrared sensor to the desired facial muscle.

[081] Thus, instead of being constructed as a fixed garment with built-in electronic circuits, such as a “mouse-glasses” or a “mouse-cap”, the device described here is an electronic accessory that is light and flexible enough to be attachable to any existing garment, respecting the user's style preferences and ensuring the dignity of not having to use an indiscreet, uncomfortable or awkward accessibility feature.

[082] 12. The optional configuration system is one that runs over the internet (Figures 6 and 7) (that is, a system that does not need to be downloaded and can be accessed in any internet browser), runs on any computer or mobile device. The device with the embedded system does not depend on the web system to function, since this serves as a convenient interface to allow the user to modify the behavior of the device to his liking, if he deems it necessary.

[083] The web system allows changing three factors in the system in general, each one of them being detailed below, being designated as OPERATION MODE, CHARACTERIZATION OF GESTURES and OPERATION PARAMETERS (Figures 6 and 7).

[084] The intention of the configuration system is to make the same device adapt to people with different movement capabilities or different types of interfaces, allowing it to meet a wide spectrum of conditions, without the need for different versions of the invention in the market, since products that have hardware are, by definition, immutable in their physical characteristics. Software-based products, on the other hand, can always be reprogrammed and updated, and this is the non-obvious characteristic that we explored to implement this resource, enhancing the present invention, which is characterized as a hybrid system. [085] The main ways in which the invention can interpret the user's head movements and facial gestures are detailed below and are collectively referred to as OPERATION MODES: a) STANDARD MODE: In this mode, the invention makes the mouse pointer “mimics”, in real time, the movements of the user's head. Thus, if the person moves their head upwards, the invention makes the pointer on the screen rise simultaneously; likewise, if the head turns to the left, the invention will move the pointer to the left at the same time, always following precisely the head displacement and the speed of the user's movement. b) VIRTUAL JOYSTICK MODE: In this mode, the invention waits for the user to point, with their head, in which direction they want the pointer to move. When this happens, the invention starts moving the cursor across the screen, in the indicated direction, progressively accelerating the pointer and stopping it when the person returns the head to the resting (neutral) position. Thus, if the user wants to move the pointer upwards, it is enough to lift the chin slightly until the head leaves the “neutral” position, causing the pointer to start accelerating upwards. The displacement is interrupted when the head returns to the “neutral” position (the position in which the user's head is kept at rest during the invention's calibration period is considered “neutral”, which must be done in the first seconds of device operation when this mode is selected). Similarly, when the user wants to move the pointer to the left, it is enough to tilt the head slightly towards the left shoulder, causing the pointer to progressively accelerate to the left until the head returns to the initial position. c) SINGLE AXIS MODE: In this mode, the invention allows a user, who can only move his head in one direction (only vertically or only horizontally), to move the cursor to any point on the screen. For example, in case a user can only move the head vertically, initially, when moving the head up or down, the pointer will move correspondingly on the vertical axis. Then, by keeping the head at rest for a few moments, the invention emits a sound or vibration signal. From that moment on, the next time the user moves their head vertically, the pointer will move correspondingly, but on the horizontal axis (left/right). After stopping moving, the invention emits a new sound signal (or vibration) and starts to move vertically when the head moves again. The rest interval that the device waits to change the axis of displacement of the pointer can be adjusted at the user's discretion through the optional web configuration system of the invention. d) CLICK ONLY MODE: Users totally incapable of moving their heads can use the invention only to detect the intention of executing mouse clicks (without moving the pointer), through the built-in infrared sensor, which captures facial gestures such as blinking eyes or the twitching of the cheeks. In this mode, the user will be able to use specialized cell phone and computer programs and applications that offer a feature known as “scanning”, in which the options available on the screen are alternately highlighted, one at a time, automatically, and the user “confirms” with the blink (or smile, or any facial gesture detectable by the invention) when the desired option is highlighted.

[086] Regardless of the selected operating mode, the invention allows the user to assign the execution of special mouse commands (right click, double click, sustained click, vertical scrolling, etc.) and even the keyboard (for example, key combinations shortcuts for copying, pasting, cutting, deleting, etc.) to performing predetermined gestures with the head or face. Gestures can also be used to change the behavior of the invention itself without necessarily causing actions on the computer or mobile device (for example, turning the device off or on without the need to press the physical button). These commands and gestures are explained below, being designated as CHARACTERIZATION OF GESTURES: a) AFFIRMATIVE GESTURE: Repeated movement of the head up and down, alternately. For the correct identification of the intention of the gesture, the number of repetitions and the execution time of the movement can be freely changed by the user through the optional web configuration system of the invention. b) NEGATIVE GESTURE: Movement of the head to the left and to the right, alternately. For the correct identification of the intention of the gesture, the amount of repetitions and the execution time of the movement can be freely changed by the user through the optional web configuration system of the invention. c) LAY YOUR HEAD TO THE RIGHT: Tilt your head towards your right shoulder. For the correct identification of the intention of the gesture, the minimum angle of inclination of the head to the right can be freely changed by the user through the optional web configuration system of the invention. d) LAY YOUR HEAD TO THE LEFT: Tilt your head towards your left shoulder. For the correct identification of the intention of the gesture, the minimum angle of inclination of the head to the left can be freely changed by the user through the optional web configuration system of the invention. e) SHORT FACIAL GESTURE: Perform a facial gesture detectable by the infrared sensor of the invention (wink, cheek twitch, smile, etc.) in a time interval lower than a predetermined threshold. For the correct identification of the intention of the gesture, the maximum time of sustaining the facial gesture can be freely changed by the user through the optional web configuration system of the invention. f) LONG FACIAL GESTURE: Maintain the facial gesture detectable by the infrared sensor of the invention (wink, cheek contraction, smile, etc.) for a time interval greater than a predetermined threshold. For the correct identification of the intention of the gesture, the minimum time of sustaining the facial gesture can be freely changed by the user through the optional web configuration system of the invention.

[087] To each of the gestures mentioned above, the user can assign commands for the device to execute, according to his preference, simply configuring them in the optional web system of the invention. The commands that can be configured are listed below: a) LEFT MOUSE BUTTON CLICK (DEFAULT CLICK): By default, assigned to the SHORT FACIAL GESTURE. b) SCROLL DOWN: By default assigned to LAY YOUR HEAD TO THE RIGHT. c) SCROLL UP: By default assigned to LEFT HEAD LAY. d) TEMPORARILY DISABLE THE INVENTION (SLEEP): By default, assigned to the NEGATIVE GESTURE if the invention is ON. e) REACTIVATE THE INVENTION (WAKE-UP): By default, assigned to the AFFIRMATIVE GESTURE if the invention is DISABLED. f) RIGHT MOUSE BUTTON CLICK: By default, assigned to the AFFIRMATIVE GESTURE if the invention is ACTIVATED. g) HOLD CLICK (“HOLD” CLICK): Assigned to LONG FACIAL GESTURE by default. h) KEYBOARD COMMANDS: The invention can execute any possible command on a conventional keyboard, from a simple key (for example, “Enter”), to a combination of keys (for example, “Ctrl + C”). The command can be freely chosen by the user, and assigned to any of the facial or head gestures identifiable by the device. These choices are also made through configuration in the optional web system of the invention.

[088] In addition, the invention is able to switch between its operating modes (STANDARD, VIRTUAL JOYSTICK, SINGLE AXIS or CLICK ONLY) in full operation. The change of operating mode can be attributed to one of the detectable gestures, so that, upon identifying the gesture, the invention immediately switches to the next operating mode in the sequence shown above. For example, if the user configures the LONG FACIAL GESTURE to perform the change of operation mode, if the device is in STANDARD mode when this gesture is identified, it will start to operate as a VIRTUAL JOYSTICK. If the gesture is performed again, the device switches to SINGLE AXIS MODE, and so on to CLICK ONLY mode, and again to STANDARD mode, each time the gesture assigned to the mode change is performed. [089] Still through the optional web configuration system of the invention, the user can change some operating parameters of the device to suit his needs or preferences. Thus, the next time the user performs the gesture assigned to the STANDARD CLICK action, the invention will perform the action of clicking the RIGHT mouse button instead of the LEFT one, for example.

[090] User ADJUSTABLE PARAMETERS are as follows. It should be noted that some of the parameters only make sense for certain modes of operation of the invention: a) AMOUNT OF DISPLACEMENT OR POINTER SPEED: Determines a greater or lesser displacement of the mouse pointer as a function of the same head movement (in STANDARD MODE or SINGLE AXIS) or faster or slower movement of the pointer (in VIRTUAL JOYSTICK MODE). b) POINTER ACCELERATION: Only makes sense in VIRTUAL JOYSTICK MODE. When this option is OFF, the pointer moves at a constant speed (determined by parameter 1 ). When this option is ON, the speed of the pointer will progressively increase from rest, in the direction indicated by the user. c) SENSITIVITY OF THE DETECTOR OF FACIAL GESTURES: Changes the sensitivity of the built-in infrared sensor of the invention. In this way, it will detect more subtle or broader movements of the part of the face directly exposed under the infrared light beam emitted by the device. If it is positioned on an eyeglass frame, with the sensor pointed at the eyelids, the sensitivity can be increased to allow identification of the subtlest eyelid movements (in the case of people who have a weaker blink), or decreased in order to capture only stronger and more intentional blinks (ignoring, for example, unintentional light blinks) d) NUMBER OF REPETITIONS OF AFFIRMATIVE AND NEGATIVE GESTURES: Changes the number of repetitions of head movements in the vertical (affirmative) or horizontal (negative) direction needed to identify these gestures. e) TIME FOR EXECUTION OF AFFIRMATIVE AND NEGATIVE MOVEMENTS: Changes the minimum times within which the user will need to perform the number of repetitions of affirmative or negative head movements for these gestures to be identified. f) MINIMUM ANGLE OF INCLINATION OF THE HEAD TO LEFT AND RIGHT: Changes the minimum angles of inclination of the head of the user to the left and to the right so that these gestures are identified. g) MAXIMUM TIME FOR IDENTIFICATION OF THE SHORT FACIAL GESTURE: Changes the maximum time within which the infrared sensor needs to identify the beginning and end of the movement of the part of the face directly exposed under the beam of infrared light emitted by the device, so that it is considered as a short facial gesture. h) MINIMUM TIME FOR IDENTIFICATION OF THE LONG FACIAL GESTURE: Changes the minimum time for which the infrared sensor needs to identify the sustained contraction of the part of the face directly exposed under the beam of infrared light emitted by the device, for it to be considered as a facial gesture far away. i) INTENSITY OF SOUND SIGNALS: Changes the volume of the sound signals emitted by the device. j) INTENSITY OF VIBRATING SIGNALS: Alters the intensity of signaling vibrations emitted by the invention.

Claims

DEVICE AND SYSTEM CLAIMS FOR CONTROLLING ELECTRONIC INTERFACES

1 . Device and system for controlling electronic interfaces, system for capturing and processing head movements and facial gestures for controlling electronic interfaces, characterized in that it is substantially compact and has means for attachment to any head accessory, comprising a microcontroller compatible with the Bluetooth 5.1 communication protocol; a motion sensor with six axes; being equipped with an accelerometer; and at least one infrared light emitting and detector integrated circuit; comprising an integrated information system with means for interpretation, identification and parameterization of movements and gestures captured by the system components.

2. Device and system for controlling electronic interfaces, according to claim 1, characterized in that said information system comprises sets of processes that implement logical behaviors, Bluetooth communication protocol stack, hardware abstraction libraries, digital filters, sensor fusion filter, cryptography libraries and communication API over protocol according to HID standard (Figure 9).

3. Device and system for controlling electronic interfaces, according to claim 1, characterized by the fact that it has an information system, accessed through the internet, which allows the configuration of sensitivity, automated commands and mode of operation, comprising network of computers that take on the functions of hosting the web, database with user information, devices and access credentials; file server with codes and multimedia content, data packet routing, address resolution (DNS), internet browser, including the device itself and the system set forth in the claim

1 (Figure 8).

4. Device and system for controlling electronic interfaces, according to claim 1, characterized in that the fastening means is a removable clip (Figure 1 - h), comprising at least 2 (two) substantially plastic parts connected through a flexible wire of 1.5 to 3.5 millimeters in diameter (Figure

2 - f) coated with protective material, said wire having a length between 20 and 100 millimeters; having a substantially plastic part having a prismatic shape trapezoidal and rectangular (Figure 2 - o) with means for fitting and sliding on the rail (Figure 2 - 1) of the cabinet (Figure 2 - d) of the device.