WO2013098436A1

WO2013098436A1 - Optical navigation method and system for a pointing device that is suitable for computer systems

Info

Publication number: WO2013098436A1
Application number: PCT/ES2012/000324
Authority: WO
Inventors: Pablo PÉREZ TRABADO; Kevin Raymond BLANCKAERT
Original assignee: Universidad De Malaga
Priority date: 2011-12-29
Filing date: 2012-12-27
Publication date: 2013-07-04
Also published as: ES2416833B8; ES2416833B1; ES2416833A1

Abstract

The invention relates to an optical navigation method and system for a pointing device on a monitor of a computer system, and includes: detecting by means of a plurality of optical sensors movement information generated by a user on said plurality of optical sensors; in a processing module, comparing the movement information obtained by the optical sensors with movement information of a series of predefined gestures, which are associated with the execution of one or more events; if the movement information detected by the optical sensors is equal to the movement information of any predefined gesture, executing the event, or events, associated with said predefined gesture; if the movement information detected by the optical sensors is different from that of any predefined gesture, moving a cursor on the monitor of the computer system in accordance with the movement information detected on the optical sensors.

Description

METHOD AND SYSTEM OF OPTICAL NAVIGATION FOR AN APPROPRIATE POINTING DEVICE FOR COMPUTER EQUIPMENT

TECHNICAL FIELD OF THE INVENTION

The present invention relates, in general, to the field of navigation methods and more specifically to optical navigation methods for pointing devices in computer equipment.

BACKGROUND OF THE INVENTION

In general, it is called signaling devices to those devices that allow us to translate movements of our body into control inputs of a computer. This type of device exists for more than 60 years, the "mouse" being one of the first, and surely the best known today. The first computer mouse model was invented by Doug Engelbart in 1963, and patented in 1970.

Although it was much more convenient to use than the alternatives, the use of the mouse was not widespread until the 1980s. Until then there had been no great need for pointing devices, but that changed when the use of graphic interfaces began to popularize of user (GUI), replacing the command line terminal that until then had predominated. In particular, until then all word processors used the keyboard, and had no need for a pointing device.

It was very promising since its inception, and has been manufactured using various types of technology - mechanical, optical, laser, etc. Despite its age, and despite the great technological advances of the last decades, an alternative input device has not yet been found that combines in the same way reliability, comfort of use, practicality and that is also ergonomic.

The use of signaling devices can take more than two thirds of the time we are in front of a computer, with the mouse being the predominant one in use among all of them. Because the mouse is usually positioned on the side of the keyboard, it forces the user's neck and arm muscles to be in an unnatural position. This, together with the repetitive actions that the use of the mouse usually entails, increases stress on these parts of the human body, resulting in possible occurrences of musculoskeletal ailments due to repetitive activities, such as carpal tunnel syndrome.

The use of devices such as "trackball" does not eliminate this problem, but only Change the affected joint. A Trackball moves the pointer without the need to move the device because it has a sphere at the top, which usually moves with the thumb, and these movements translate into cursor movements. Thanks to this, the space needed to control the device decreases considerably. However, trackball is not immune to the problems of repetitive action injuries, because it translates the ergonomics problems of the forearm and hands to the thumb joint.

In addition to the well-known traditional mechanical mouse, there are alternatives such as optical mice whose operation is based on incorporating an LED that illuminates the surface below, and they have a small camera that takes pictures of it at regular intervals. When comparing consecutive images, a sensor can calculate the displacements made, which will be translated into cursor displacements. The optical mouse still has the same ergonomic problems as the conventional mouse, in addition to presenting difficulties to work on surfaces with high brightness, or on very smooth and uncommon surfaces, such as mirrors; the laser mice solve the problems of operation on difficult surfaces using a laser diode and detection of phase changes in the reflected light to follow the movement; however, its operation by a human user is identical to that of an optical mouse, so it does not incorporate any improvement in the field in the field of ergonomics. There are also 3D mice, devices that not only detect movements in the horizontal plane, but also detect them in the vertical plane; Such a signaling device is the Wii console controller. This control can calculate its position using an infrared camera integrated in the device, which measures the displacements that it observes with respect to an infrared light emitter, which is positioned near the monitor or television. Its prolonged use can still cause muscle aches, since they do not usually allow having arms or hands at rest, causing fatigue and muscular stress.

In recent years other input devices have appeared that have a gesture-based interface. This interface tries to convert actions that we produce (almost) naturally into input signals, giving a much more intuitive operation. Their success is also because they avoid having to hold a bulky body in their hands. Examples of these devices are the Wii console controller, the ¡Pad and the Xbox Kinect.

However, despite having a more intuitive interface, these devices consume many system resources, especially in the case of physical contactless interfaces based on image processing. In these systems one or more cameras capture images that are then processed to identify the gestures made and convert them into input signals for the operating system. In general, the problems posed by these types of input devices are three: they require relatively expensive hardware (cameras, ultrasonic or infrared rangefinders, etc.).

If they process the images using the computer's CPU, they can have a negative impact on the performance of the computer, because this processing usually requires considerable computing power. If you want to avoid this problem, it is necessary to incorporate your own processor into the sensor system (as in the case of the Kinect), which helps to make it even more expensive.

- in general, the performance of these systems is only suitable for applications such as video games, in which the working conditions and distances to the sensor are strictly determined, and in which the vocabulary of possible gestures is limited. However, in general-purpose systems, camera resolution and lighting problems appear, and this type of technology is not always able to correctly discriminate a hand or arm from the surrounding environment, sometimes the need for assistive devices appears. to the segmentation of the image (such as the use of white gloves).

Another example of gesture-guided interfaces are Touchpads and multi-touch touch screens, which have gained special acceptance with their use on devices such as the iPad or iPhone. However, in systems with capacitive sensors (such as the Touchpad) electronic problems may appear even by the mere presence (or absence) of grease on the user's skin, while the touch screens suffer from lack of precision. In addition, in the case of Touchpads, their integration into a laptop or notebook usually consumes a considerable area, which could be saved (or enlarged by the keyboard) if the presence of the signaling device is not necessary. The operation of a Touchpad is based on the translation of capacitive variations in a dielectric in cursor movements and virtual (tap) keystrokes. Touchpads are usually integrated in laptops and are also used in some desktops when there is little work space. To use them, simply move one or more fingers on the control surface; the movement produced with the fingers will translate to cursor displacements. Some have a control strip on one of the sides, which can be used to simulate the mouse wheel; an up or down movement corresponds to turning the mouse wheel in the same direction.

Touchpads do not always produce fluid movements, and multiple consecutive lateral finger movements are often necessary to move the cursor from one side of the monitor to the other, which is why users often end up using an external USB mouse. The sensitivity problems of the Touchpad may be more noticeable when handled with partially or completely damp fingers, as occurs just after washing your hands. Additionally, due to the technology on which it is based, it is not possible to use a Touchpad with anything other than bare fingers.

The capacitive touchpad allows the detection of simultaneous keystrokes with multiple fingers, a feature called "multitouch" ("multitouch"). The use of multitouch sensors to signal commands to the computer through complex gestures has recently gained great popularity thanks, fundamentally, to the use of this technology by Apple for its laptops and iPads.

In particular, Apple has commercialized a device called "Trackpad", which is a device very similar to the touchpad, but multi-touch and oriented to use with gestures. Newer models do not have buttons, since clicks are made by pressing on the work surface. Although in spite of having better significant ones with respect to a touchpad, in the background the Trackpad still has the same electrical and ergonomic problems

A final group of pointing devices that has been developed as an alternative to classic devices are touch screens. They are manufactured using a wide variety of technologies; the most important are resistive, capacitive, SAW (Surface acoustic wave) and FTIR (Frustrated Total Infernal Reflection) technology.

They have important advantages as they do not need to change the cursor from one side of the screen to another, since we can directly access the desired position by touching it with a finger. No additional peripherals are required to move the cursor. They improve the ergonomic properties of most alternatives, and have very intuitive interfaces. But, despite the ease of use, these devices have a number of disadvantages, which have prevented them from becoming popular as much as the conventional mouse:

• In some applications where there is an abundant use of signaling devices, such as videogames, these devices are often too slow and impractical since it prevents the user from seeing all the screen elements.

• They are usually quite expensive. • Because the signaling device is the monitor itself, if either of the two functionalities is damaged, the cost to replace them is quite high.

In summary, the state of the art offers a multitude of options when it comes to having pointing devices for computer equipment, but, in general, none of the existing solutions is capable of simultaneously offering simple and intuitive operation like that of the conventional mouse, a comfortable and flexible use of gestures such as that of the Touchpad, and a physical configuration that minimizes, or completely eliminates, the ergonomics problems derived from the continued use and the execution of repetitive actions, which end up causing discomfort or even injury to the user.

SUMMARY OF THE INVENTION

The present invention serves to solve the aforementioned problem by providing an optical navigation method for a pointing device on a monitor of a computer system. The method comprises the steps of:

- detecting by means of at least 2 optical sensors a travel information generated by a user on said optical sensors; comparing the displacement information obtained by the optical sensors with displacement information of a predetermined gestures, which are associated with the execution of one or more events;

if the displacement information detected by the optical sensors is equal to the displacement information of any preset gesture, execute the event, or events, associated with said preset gesture;

If the displacement information detected by the optical sensors is not equal to that of any preset gesture, move a cursor on the computer system monitor according to the displacement information detected on the optical sensors.

In the preferred embodiment of the invention 3 optical sensors arranged in line on a computer keyboard are used, in which the data collected by the lateral optical sensors activate different functionalities and gestures.

Additionally, the invention may comprise detecting a click event by processing a tracking signal, such as the squal signal, and the displacement signals on a vertical axis and a horizontal axis, so that a click is detected when a variation in the tracking signal along with a certain time interval coincides with the absence of displacement data detected by the optical sensors in the vertical and horizontal axis.

Preset gestures may comprise scroll information that emulates the wheel spin of a computer mouse; scrolling information that emulates the well-known event of dragging a selection, usually performed using a computer mouse; scrolling information produced by the event of launching an application; offset information produced by the event of opening a file; offset information that produces the event of minimizing all open applications. Or also preset gestures that can be configured by the user associating any event with a sequence of gestures to be performed on the optical sensors.

Another aspect of the invention relates to an optical navigation system suitable for aiming and activating functions in a computer equipment. The system comprises: a plurality of optical sensors that detect the displacement information of a user acting on them and provide them to a microcontroller;

a microcontroller that establishes communications between the plurality of optical sensors and a processing module to which it delivers the travel information detected by the plurality of optical sensors;

a processing module that:

or detects by means of at least 2 optical sensors a travel information generated by a user on said optical sensors; or compares the displacement information with displacement information of pre-established gestures that are associated with the execution of some events;

or if the displacement information detected by the optical sensors is equal to the displacement information of some preset gesture, execute the event associated with said preset gesture;

or if the displacement information is not equal to the displacement information of any preset gesture, move a cursor on the computer system monitor according to the displacement information detected on the optical sensors. A final aspect of the invention relates to a computer program comprising program code means adapted to perform the processing steps when said program is executed in a general purpose processor, a digital signal processor, an FPGA, an ASIC, a microprocessor, a microcontroller, or any other form of programmable hardware.

The above features and advantages do not limit the present invention, and those skilled in the art will recognize additional features and advantages upon reading the following detailed description, and in view of the accompanying drawings.

DESCRIPTION OF THE DRAWINGS

To complete the description that is being made and in order to help a better understanding of the features of the invention, according to a preferred example of practical realization thereof, a set of drawings is attached to the description as an integral part thereof. which, by way of illustration and in a non-restrictive manner, the following has been represented:

Figure 1 illustrates the system consisting of a sensor, lens and LED representing the state of the art.

Figure 2 represents an embodiment of the invention on a computer keyboard.

Figure 3 represents a flow chart of the smoothing algorithm used in an embodiment of the invention.

Figure 4 depicts a flow chart of the acceleration algorithm used in an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Next we will develop an embodiment of the invention. But to understand the operation it is convenient to first understand the operation of an optical mouse, which is as follows:

1. An LED diode 1 illuminates the surface beneath an optical sensor 2, detecting microscopic irregularities; its arrangement can be seen in figure 1.

2. A prismatic lens 3 placed at the bottom of the sensor is responsible for concentrating in the sensor chamber the light reflected on the surface under the mouse. 3. The sensor camera takes pictures of the illuminated surface at regular intervals, and delivers this data to a digital signal processor called DSP (Digital Signal Processor).

4. By comparing the images, the DSP obtains the values of the movements produced by detecting the displacement of said microscopic irregularities, and stores them in internal mouse registers.

Therefore, in order to understand the use made of optical sensors in this invention, and how they pick up the displacement by passing their finger over them, it is helpful to think of an optical mouse turned over its normal position. Thus, an embodiment of the invention comprises a plurality of optical sensors similar to those used in optical mice, and which are integrated into a system where the light of one or more LEDs, guided by one or more prisms, provides the necessary lighting at a certain height above the sensors; This height corresponds to the focal length of the lens, or lens system, which concentrates in the sensor the light reflected from the object to be followed, and can be adjusted by changing said lens.

At that height it is at which the user's finger (or other object for this purpose) slides over the sensor, so that the sensor camera will take photographs at regular intervals that, when comparing an image with the following, will allow to detect the movement of the finger or object. To facilitate the correct passage of the finger over the sensor, the invention offers a sliding surface located exactly at a height above the sensor equal to the focal length of its lens system; The surface has an opening on each sensor, which allows the passage of both the incident light on the finger or object and the light reflected by them. In one of the embodiments of the invention a transparent sheet (eg acetate) is incorporated on the opening surface. This sheet preserves the optical properties of the system, while ensuring that the finger or object slipped by the user is always kept at the correct distance from the sensor, and protects the lens or lens system from external contamination.

From here, in a particular embodiment we will use 3 optical sensors, for example of the Avago ADNS-5020-EN type, with a microcontroller such as a Microchip PIC16F877 and a processing module. The microcontroller provides an interface, in this case RS-232, between a PC and the sensors, performing low-level read and write operations against the latter. This indirect communication involves both sending commands from the PC to the sensors and receiving data from the sensors on the PC. The sequence of operations for system startup which are carried out for initialization are:

1. Reset the sensors: At system startup, it is always convenient to reset the sensors. For this, a data frame will be sent via the RS-232 serial bus to the microcontroller. If everything went well, it will answer the PC with another frame indicating the success of the operation.

2. Obtain ID: a data frame is sent via the RS-232 serial bus to the microcontroller; Upon receiving this message, the microcontroller will return a message with an identification of each sensor. This command is used to verify that the connection has been made correctly, as well as to ensure that the sensors continue to function correctly. The system uses this command for two different purposes:

a) As a "keepalive" mechanism: the microcontroller uses it, routinely and periodically, to verify that the entire system continues to function properly.

b) As a mechanism for explicit verification of the behavior of the sensors, in case of receiving a negative or anomalous response to the execution of a command.

3 Obtain microcontroller firmware data: the firmware version of the microcontroller is checked. For this, the PC sends a command to the microcontroller requesting this information; If no errors occur, the microcontroller will reply to the PC with a data frame that will contain the version of the code loaded in the Microchip, along with the number of sensors for which it is programmed.

4. Change the resolution of the sensor camera: The sensors work with a certain resolution, expressed in cpi (counts per inch). In the case of the Avago ADNS-5020-EN sensor, there are two possible resolutions with which we can work: 500 and 1000 cpi. To change the resolution from 500 to 1000, or vice versa, a frame is sent to the microcontroller. If the operation is successful, the microcontroller will return another frame as confirmation. The PC, therefore, initializes the sensors to the resolution selected by the user.

The invention, in its different embodiments, may choose different configurations in the placement of the sensors. One of them would be in the form of a triangle, placing a vertex in the lower part, where a sensor is placed mainly to record the displacements, while the other two vertices aligned in the upper part house another 2 sensors that do functions similar to the buttons of a conventional mouse.

Another embodiment chooses to place a sensor on the left, mainly intended to pick up the movements of the finger on it, and two other sensors located on its right but following an upward curve, so that they are accessible to the other fingers and perform functions similar to the buttons of a conventional mouse.

The preferred embodiment of the invention, however, comprises three fully aligned sensors, in which the central sensor is the one chosen to perform the displacements and the lateral ones for the accessory functions, similar to a conventional computer mouse. The preferred location of the invention places the 3 sensors on a computer keyboard just above the cursor arrows, as illustrated in Figure 2, but the invention can also be arranged on any other surface of the computer or electronic device, such as For example, on the edge of an e-book reader or tablet, which offers an adequate position and distance between the sensors, or can even be placed on a device external to the computer, connected to it by a serial bus such as USB .

Once the system is initialized, according to a preferred embodiment of the invention, the system enters the normal mode of operation, in which a set of stages is cyclically repeated as can be seen below:

A first stage of communications in which the PC reads through the microcontroller the data obtained by the system sensors. To do this, the PC sends a command to the microcontroller that allows it to obtain displacement information, which includes the deltas values (accumulated displacement value since the last iteration), motion (indicator of whether or not any movement has been detected in the present iteration) and squal (assessment of the detection quality during the present iteration) of each of the sensors in the system. The microcontroller is responsible for carrying out the necessary reading operations through the sensor serial buses, and returns a single frame to the PC, which groups all the sensor data.

- Once the sensor data has been obtained, a processing stage begins in the so-called processing module, and in which different algorithms are used to compare the displacement information obtained by the optical sensors with the displacement information of pre-established gestures, which They are associated with the execution of one or more events.

In the event that the processing module identifies any gesture Preset enters a stage in which it interacts with the operating system of the computer equipment to produce the mouse or keyboard events associated with said preset gestures. In the case of not using gestures, the event that occurs is the movement of the pointer on the screen in response to the movements of the user's fingers on the optical sensors.

Entering in more detail in the processing stage, and following an embodiment of the invention, the raw values obtained by the optical sensors are transformed into information on the movements made by the user, which is subsequently stored so that it can be available later.

This stage is also responsible for identifying the appearance of gestures from the raw data of the sensors, using for this purpose stored information on the conformation of the gestures (what activation of which sensors triggers which gesture, and what command it has associated).

The processing module transforms the data of each sensor into a direction of movement indicator for said sensor (NONE, LEFT, RIGHT, UP, DOWN, TAP, the latter corresponding to a complete click, with raising and lowering of the finger). The displacement information that is used to define a gesture is therefore composed of an ordered time sequence of this type of indicators. As an example, the programming of a defined gesture such as running your finger through the three sensors, from left to right, is recorded as the time sequence of indicators (with certain tolerance margins in the timing between indicator and indicator):

SENSOR1 - RIGHT + SENSOR2 - RIGHT + SENSOR3 - RIGHT

The processing at this stage compares the latest data obtained from the sensors with the previous contents, to identify the beginning and end of a movement of a finger or object on any of the sensors. Each time the completion of one of these movements is detected, it is checked whether the sequence of the last sensor activations corresponds to that programmed for any gesture; If so, the command to be issued is determined, and it sends the corresponding request to the next stage, discussed above, of interaction with the operating system.

If movements in the sensor are detected that are not interpreted as gestures, but of navigation or cursor movement, the order is sent to move the mouse cursor on the screen to the stage of interaction with the operating system.

The invention may comprise a smoothing algorithm with which to solve a problem This may occur because the readings offered by a user's finger movement sensors on them may not be uniform and introduce spurious values. The actual offset values of a current and previous iteration are taken as input to produce uniform and constant final values. Figure 3 illustrates this process. It begins by comparing the displacement in the current iteration with the displacement in the previous iteration 31, for this it is checked if they have the same sense 32, in the case that it is not so it is concluded that the smoothed displacement is the same as the current displacement 33 and the process is over.

If the displacements do not have the same direction, it is checked whether the magnitude of the previous displacement is greater than the magnitude of the current displacement 34, if not, it is concluded that the smoothed displacement is the same as the current displacement 33 and the process. In the event that the magnitude of the preceding displacement is greater than the magnitude of the current displacement, it is checked if the magnitude of the current displacement is greater than a certain number that acts as threshold 35, if not, it is concluded that the smoothed displacement is the same as the current offset 33 and the process is terminated.

In the event that the magnitude of the current displacement is greater than the threshold, a correction is calculated that is half of the difference between the magnitudes of the current displacement and the previous 36, then the current displacement value is modified by a magnitude equal to the calculated correction 37 and subsequently it is checked if the correction has changed the direction of the offset 38. If the direction has changed, it is concluded that the smoothed offset is equal to zero 40, however, if the correction has not changed the direction of the displacement, said correction 39 is applied to the current displacement and the process is terminated.

An acceleration algorithm can also be included to solve the problem derived from the difference in resolution between the sensor and a typical computer screen. If the movement of the fingers on the sensors is translated directly to the screen, several passes are necessary to move the pointer from one side of the screen to the other. Therefore, an algorithm that multiplies the amplitude of the movement gradually is used in one embodiment of the invention. This means that for small displacements, which require greater precision, no acceleration is applied and instead, as the displacement is greater, the pointer displacement is greatly amplified, the more the greater the displacement. Figure 4 illustrates this process.

To allow fine control of the magnitude of the acceleration as a function of the amplitude of the displacement, the preferred embodiment of the invention uses an acceleration function defined by sections, which fits approximately a polynomial function, which gives the best result the higher its degree. However, the user has the possibility to modify the definition of sections to implement any function, and thus adjust the acceleration behavior to the one that is most comfortable and practical.

The acceleration algorithm consists of a list of acceleration-threshold pairs, and for each offset value obtained from the sensor, the smallest threshold that is greater than or equal to that offset is used. The result of the algorithm returns the input value multiplied by the acceleration value linked to that threshold found as we see in Figure 4. The first is to obtain the current displacement 41, then a first threshold 42 is set and then enters a loop in which the current displacement is compared with a threshold 43 that changes to a greater threshold 46 as long as the current displacement is greater than said threshold and as long as a maximum threshold is not exceeded 45. When the displacement it is not greater than the threshold or when the maximum threshold is reached, the loop is exited and it is concluded that the final displacement is equal to the current displacement multiplied by the acceleration associated with threshold 44 and the process is terminated.

In addition to this acceleration by sections, the invention may also include global acceleration parameters, which affect all displacements equally. This acceleration adjusts the relative movement of the cursor on the screen to its resolution, so that increasing the resolution of the desktop (for example, switching to 1280x1024 from 1024 x 768) does not translate into the need to swipe your finger over the navigation sensor when we move the cursor from side to side of the screen. For this acceleration, there are two values, called vertical scale and horizontal scale, which are multiplied respectively by all the displacements Ay and Δ ^χ , increasing or decreasing them. Similarly to how it happens with the acceleration values by sections, these two values can be changed dynamically by the user.

The summary of the overall procedure followed by one of the embodiments of the invention to calculate the horizontal and vertical displacements of the mouse cursor would be this:

1. Obtain the values of the sensor raw displacements (actual, not calculated) of the current and previous iteration.

2. Apply the corresponding acceleration to each of these values. 3. Pass these two values as arguments to the smoothing function to obtain a uniform displacement, both for the displacements on the horizontal axis Δχ and for the displacements on the vertical axis Ay.

4. Apply the value of the horizontal and vertical scale to the result respectively.

5. Send the values obtained for the execution of the relevant events during the Interaction stage with the operating system.

Additionally, the invention may also comprise an algorithm for detecting the pulsations made on any of the sensors. This algorithm analyzes the values of the squal and motion signals that are delivered by the sensor, looking for the appearance of characteristic patterns that identify the presence of a pulse, and are described below.

The squal signal (Surface Quality) is a signal generated by the optical sensor that delivers, in real time, an assessment of the quality of the tracking that the sensor is making of a possible surface located in its visual field, and consists, for each image captured, in the number of microscopic irregularities that are visible to the sensor in said image. A high squal value indicates the detection of many irregularities, which implies a good quality in the detection of surface displacement, while a low squal value indicates that few irregularities are detected, and the monitoring will be poor; in the most extreme case squal is 0, indicating that the sensor does not detect any surface, which usually occurs when there is no surface on the sensor, or there is but is very removed from it, at a distance much greater than the focal length of the lens or lens system.

For its part, the motion signal is a signal generated by the sensor to indicate if it has detected displacements Δχ and / or Ay on the surface since the last time they were read on the sensor, through its serial communications bus, the values of these displacements; a value of 0 in this signal indicates that no movement has been detected since the last reading.

The present invention additionally proposes a pulse detection algorithm that takes into account the following properties, determined by the analysis of traces of actual use of squal and motion signals:

1. In general, the readings of the sensor data during a pulse deliver only a few iterations in which the motion value is non-zero, while during a movement the number of iterations with non-zero motion is much greater, even when squal has value changes of a magnitude comparable to those of a pulsation.

2. In order for the use of motion to adequately discriminate whether or not the squal variation is part of a movement, it is not enough to analyze motion at the time the squal changes, but it is necessary to extend this analysis to a certain interval of time around of the squal variation.

For example, taking these properties into account, the pulse detection algorithm used in an embodiment of the invention uses the following rules:

1. A pulse is delimited by an activation movement (lowering of the finger on the sensor), followed after a certain time not exceeding a certain limit by a deactivation movement (raising of the finger), delimited respectively by a certain increase and a certain decrease of the squal signal.

2. To correctly discriminate these certain increases and decreases in squal the algorithm averages its values in a certain number of iterations, for example the last 25 iterations, and verifies that the change in squal has occurred in a short time interval (maximum three iterations), and that there has been no systematic activation of motion during a certain number of iterations before and after that change in squal; Only if both conditions are met, the algorithm considers the change in squal as a pulse.

In one of the embodiments of the invention, on the data coming from the optical sensors, the pulse detection algorithm, the click detection algorithm and the acceleration algorithm are executed simultaneously.

Regarding the stage of interaction with the operating system, you can see in more detail the relationship established between the pre-established gestures and the associated events. In one embodiment of the invention the mouse events that are handled comprise the horizontal movement of the mouse cursor; vertical scrolling of the mouse cursor; the amount of movement, also called scroll, of the mouse wheel; flags indicating a mouse status such as "left button pressed"; Event time frames and additional information that will often not be used but reserves a field for it.

Regarding the use of the invention for the sending by gestures of sequences of pulsations of keys to the operating system and computer applications, another embodiment of the invention uses the standardized mechanisms offered by the operating system for the generation of keyboard events, and also comprises: the codes corresponding to the keyboard events associated with each gesture that use them; time frames between consecutive presses and for the complete sequence; and additional information about the event, which will often not be used but for which a field is reserved.

The execution of all these events depends directly on the association established with the gestures preset in the invention, which can also be configured by the user. The movements of pointer movements are simpler since they only require the movement of the user's fingers on the sensors to move the cursor on the screen. However, the invention offers in one of its embodiments a large number of possibilities thanks to the detection of gestures and the multiple combinations that result from the use of 3 sensors.

The use of gestures can help to accelerate frequent operations, and they are extremely useful when they are easy to remember. This is achieved with the use of intuitive gestures, and with a limited number of gestures, since a too large amount overwhelms the user. In this way, a very short learning time is needed and their use is facilitated repeatedly. Following an embodiment of the invention, the operation of gesture detection is as follows: periodically new data from the sensors arrive at the processing module; if in an iteration movements appear in a certain sensor after a sufficiently long period of inactivity, it is considered that a new sensor activation has begun. In successive iterations the system continues to process the sensor data normally until, during a certain number of iterations, no movements are detected. When this occurs, it is considered that the activation of the sensor is over, and the displacement data on the x-axis, Δχ, and the displacement data on the y-axis, Ay, will be used to determine the direction and direction of movement performed by the finger, forming a scroll information. At this point, it is checked whether the movement made corresponds to a pre-established gesture, and two cases can occur:

1. The movement must not trigger any action, either because it does not correspond to any gesture or because it is part of a gesture that involves several sensors, and the others have not yet been activated.

2. It has been detected that the movement made in the sensor, and the movements produced in the other sensors, constitute a gesture, so that the associated command and the events associated with it will be executed by the part of the invention that interacts with the operating system used in the computer equipment.

Finally, all the variables involved will return to their initial default value, and the process will be repeated the next time movements are detected.

In one embodiment of the invention in which there are three aligned sensors, two modes of operation can be distinguished: a "normal mode" and a "gesture mode". In the "normal" mode the central sensor is used to move the cursor and make clicks that emulate the click on the left mouse button, while in the "gestures" mode the three sensors are used to perform gestures exclusively.

In one embodiment of the invention, the processing module has predetermined a series of gesture associations (sensor activations) to frequently used commands, which gives the user the possibility to start using the system from the first moment, without having than program it but without restricting its ability to subsequently alter, or completely eliminate, these assignments. The assignments used in the preferred embodiment of the invention can be seen in the following tables where on the one hand the movement to be performed on each sensor is indicated and on the other hand the command to which it would be equivalent (the user can modify it to his liking to configure it in a way different).

For "normal" mode:

Sensor activation Equivalent command

To the left on the left sensor Double click

Up on the left sensor Show the desktop

Down on the left 'Drag' (continuous press) of the button

left mouse

Movements in the central sensor Move the mouse cursor

Press on the central sensor Left click

Click on the right sensor Right click

To the left on the right sensor Select a block of text

Up on the right sensor Switch to gesture mode For the "gestures" mode:

It should be noted the advantages of this use of the invention in tasks such as dragging, which requires the continuous pressing of the left mouse button. When handling a conventional mouse on trawlers that require the mouse button to be held down, sometimes it happens that we unintentionally release the finger, or it simply causes discomfort to keep it pressed for a long time. This problem is solved in the proposed invention by giving the user the possibility of not only performing a normal click, but also activating a pulse that remains active, without the need for physical action with the finger, until it is released when making a specific gesture or another click. To initiate this sustained virtual pulsation, simply move a finger down on the left sensor, and the same action will be repeated (or the central sensor click will be pressed) to deactivate the pulsation.

The task of selecting a block of text also deserves to be highlighted. It is common to have to select a block of text that goes beyond the limits of the screen, which, with a conventional mouse, requires a scroll, or mouse wheel movement, while holding down the shift and The left mouse button. This action can be problematic at times, generating unwanted scroll movements, and causing frustration to the user. To simplify this action, the proposed invention offers a much simpler solution: once a click has been made at the beginning of the text to be selected, the user moves the mouse cursor to the end of the block to be selected, and then makes a move to the left on the right sensor; just enough to select all the text from where the initial click was made to where the mouse cursor is currently. This simplifies this operation and makes it much more practical.

To end with respect to gestures, an embodiment of the invention, apart from using a predetermined set of associations of gestures to commands, provides the user freedom to be able to program gestures definitions (that is, sensor activation sequences) and assignments of these gestures to commands at your leisure and convenience. The following describes the process that is followed to modify these gestures. Gesture customization allows you to associate an arbitrary path (temporary sequence of sensor activations) with a command, which can be composed of an arbitrary number of events (operating system actions such as a mouse button click or a mouse click). key), actions (set of events that occur at the same time, such as pressing the "control" key and the "c" key simultaneously) or groups of actions (commands that are executed in order); The sequence for this is:

1. Define the groups of actions that you want to associate with the gesture.

2. Program the trajectory, and assign it to the desired command or events.

Additionally, the application also allows you to define commands that consist of launching an application and / or opening a file.

Once the command, or group of actions associated with the gesture, is defined, the rest of the assignment process includes the steps of selecting a command (group of actions) and selecting a sequence of sensor activations.

To define the gesture, the sensors must be activated in the desired order; The gesture may involve a sensor, or several. The user will be asked to repeat the activation sequence a total of three times; This is done to be sure that the input data has been read correctly, as well as to allow the user to correct a movement in case it was wrong; what is achieved through a voting procedure, so that it is enough that the movements have been carried out identically twice for the gesture to be approved.

Optionally, the invention includes a graphical user interface (GUI), which is an application that the end user uses to control the start-up or shutdown of the complete system, configure it, or to obtain and graphically represent information about its operation and sensor data. The configuration may comprise the number of sensors in the system, the acceleration parameters of the sensors, the values of the horizontal and vertical scales and the serial port through which communications with the microcontroller will be established.

Additionally, the invention can incorporate in one of its embodiments, by way of improvement, a strip with a slight roughness around the opening of the sensors, in the flat acetate surface on which the finger slides, which serves to that, when passing over the fingertip, the user has a tactile feedback of the position of the finger with Regarding the sensor window. This allows, without looking, to know if the finger is properly placed on top of the sensor, or even to locate the central sensor and differentiate it from the side sensors without having to take your eyes off the screen.

A final aspect of the invention in one of its embodiments is the possibility of being adapted for different disabilities with the need only to make minor modifications to the preferred embodiment. The combination of the use of optical sensors and their arrangement in a number and configuration specifically oriented to the spatio-temporal correlation of their data allows the design of valid input devices for people with disabilities of such nature that makes it impossible for anyone to use of the other existing devices in the market. For example:

- Person without fingers: This disability can occur, for example, in the case of a person who, as a result of prolonged exposure to severe cold, has suffered severe freezing of the extremities to have had to have his fingers amputated of both hands, remaining only with the hands and stumps of the fingers; Other examples of accidents that can lead to this disability would be a work accident in which a machine has crushed the fingers of both hands to its operator, or the loss of the usual fingers in the explosives manipulators who have been surprised by the outbreak of a load.

After such an amputation, this person has been essentially disabled to use mice or touchpads (even optical):

1. - The mouse can be moved only by hand, but the problem appears when pressing the buttons, lacking fingers.

2. - Similarly, it is possible that by hand you could move a cursor using a capacitive touchpad. If the surface of the touchpad (or trackpad) is large enough, the entire hand could be used to move the cursor, but again the problem appears at the time of the button presses, because even if they can be done tapping ("taps ") on the touchpad, probably the contact surface of an entire hand (or a finger stump) will not be correctly detected when identifying a touch.

3. - The use of an optical touchpad of the type known as Blackberry is also at least problematic. It can be assumed that the device correctly recognizes the surface of the hand to move the cursor, but what is undoubtedly much more complicated is to press the sensor with the hand to achieve the click, for the small size of the sensor compared to that of the hand. 4.- Finally, the touch screens can be operated with the hand, or with the stumps of the fingers, but the contact surface is so large that the object on the screen on which the limb should be placed must be very large, which drastically limits the possibilities of using applications not developed specifically for that person.

For this type of disability, however, the present invention in one of its embodiments comprises an input device with full and wide functionality. The invention uses a configuration of three sensors arranged in line, and the spatio-temporal correlation of the data from the three sensors; however, the specific distance that each pair of sensors must separate is not fixed, but depends solely on how the spatial sequence of sensor activation is interpreted and the time that elapses between these activations.

This means that the invention can be scaled in size so that it can be used with a fingerless hand exactly the same as it would be used with a single finger; the only difference would be that the distances to be moved by the hand will be, logically, greater than those to be moved by the finger.

It is easy to see that, using this scaled model of the invention, and combining it with the use of gestures, the disabled user has an input device with a functional and expressive capacity that cannot be achieved with any of the current alternatives. It is also interesting to note that the scaling of the invention only requires increasing the distances between sensors, but no change in the sensor and processing hardware (and, therefore, in the cost of the solution).

- Person without hands (or arms): It is a more extreme case of the previous disability, and corresponds to a person who, by accident, illness, or any other reason, has lost both hands (or, worse, both arms, to a distance more or less close to the shoulder).

With this disability we must discard any of the conventional input devices from the previous point, because without hands (or arms) it is not possible to operate mice, touchpads, trackpads or touch screens. However, if the person still has at least one foot it is still possible to employ an embodiment of the invention; a scaled version of the preferred embodiment, which is used by moving one foot on the surface containing the sensors. The way to use this scaled version (using the central sensor to move the cursor, and lateral displacements on the three sensors to activate gestures and pulsations) can remind a little of the way a musician uses the foot to operate the bass pedal of an organ

Note that in this text, the term "comprises" and its derivations (such as "understanding", etc.) are not to be understood in an exclusive sense, that is, these expressions should not be construed as excluding the possibility that what described and defined may include other elements, stages, etc.

Claims

1. - Optical navigation method for a pointing device on a monitor of a computer system characterized by comprising:

- if the displacement information detected by the optical sensors is not equal to that of any preset gesture, move a cursor on the computer system monitor according to the displacement information detected on the optical sensors.

2. - The method of claim 1 wherein the displacement data is detected by 3 optical sensors arranged in line on a computer keyboard, the data collected by the lateral optical sensors activate different functionalities and gestures.

3. - The method of any of the preceding claims wherein a click event is also detected by processing a tracking signal, and the displacement signals on a vertical axis and a horizontal axis, therefore a click is detected when at a certain interval A variation in the tracking signal coincides with the absence of displacement data detected by the optical sensors on the vertical and horizontal axis.

4. - The method of claim 3 wherein the tracking signal is squal

5. - The method of any of the preceding claims wherein the pre-established gestures comprise displacement information that emulates the rotation of the wheel of a computer mouse.

6. - The method of any of the preceding claims wherein the preset gestures comprise displacement information that emulates the known event of dragging a selection, usually made using a computer mouse.

7. - The method of any of the preceding claims wherein the pre-established gestures comprise displacement information produced by the event of launching an application.

8. - The method of any of the preceding claims wherein the pre-established gestures comprise displacement information produced by the event of opening a file.

9. - The method of any of the preceding claims wherein the preset gestures comprise displacement information that produces the event of minimizing all open applications.

10. - The method of any of the preceding claims wherein the preset gestures are configured by the user associating any event with a sequence of gestures to be performed on the optical sensors.

11. - The method of any of the preceding claims wherein the displacement information that the optical sensors detect from the user is produced by operating the user with the fingers on the optical sensors.

12. - The method of any of the preceding claims wherein the displacement information that the optical sensors detect from the user is produced by the user acting on the optical sensors with any part of his body or with an object.

13. - The method of claim 1 wherein an acceleration algorithm configured to gradually transmit the user's movement over the optical sensors is used for cursor movement.

14. - Appropriate optical navigation system to point and activate functions in a computer equipment; The system is characterized by comprising:

a plurality of optical sensors that detect the displacement information of a user acting on them and provide them to a microcontroller; a microcontroller that establishes communications between the plurality of optical sensors and a processing module to which it delivers the information of displacement detected by the plurality of optical sensors;

a processing module that:

or if the displacement information is not equal to the displacement information of any preset gesture, move a cursor on the computer system monitor according to the displacement information detected on the optical sensors.

15. - The system of claim 14 wherein the plurality of optical sensors is 3 optical sensors arranged in line on the keyboard of a computer, wherein the central optical sensor collects user travel information and the lateral optical sensors detect clicks that activate different functionalities and gestures.

16. - The system of claim 15 wherein the plurality of optical sensors is arranged immediately above the cursors of a computer keyboard.

17. - A computer program comprising program code means adapted to perform the steps of the method according to any of claims 1 to 13 when said program is executed in a general purpose processor, a digital signal processor, an FPGA, a ASIC, a microprocessor, a microcontroller, or any other form of programmable hardware.