SE502186C2 - Device for three-dimensional control - Google Patents

Abstract

A method and a device are adapted for three-dimensional control, especially of a display. The ranges of application include three-dimensional "mice" and touch screens controllable in three dimensions. A free control space (S) is located between a radiant means and a detecting means, of which at least one is two-dimensional. An object, such as a human finger, is moved freely within the control space (S) in three directions (x, y, z), thus throwing two or more shadows (S1, S2) on the detecting means (10). The current position of the object (12) in the control space (S) is established on the basis of the absolute and relative positions of the shadows (S1, S2) detected. If the distance between the object (12) and the detecting means (10) is altered, the spacing of the shadows (S1, S2) is likewise altered. Generated control information indicates the current position of the object in three dimensions.

Description

502 186 10 15 20 25 30 35 2 EP-A1-0 526 015 describes a computer mouse for three-dimensional control. The user holds up the mouse with his hand in the air in front of and at a distance from a monitor and moves the mouse freely in space in front of the screen. The user can thereby create three-dimensional images on the screen by correspondingly moving his hand in space. The mouse carries a transmitter directed at the screen, and on the edges of the screen are three receivers, which are located at a distance from each other and are directed towards the transmitter of the mouse. A microprocessor compiles information from both the transmitter and the receivers to calculate the mouse's position in space. Although it may work theoretically, this known computer mouse should not be practically usable, since the user for ergonomic reasons can not work with a mouse carried in the air for a long time. The accuracy is also likely to be relatively poor with this known solution.

In the above-mentioned EP-Al-0 526 015, a pointing device operating with a guide ball for three-dimensional control is mentioned as known technology. A guide ball projecting through a housing allows control in two dimensions, and a guide ball enclosing a movable ring allows control in the third dimension without the user having to move his hand away from the ball. However, this known solution has the disadvantage that a linear movement of the pointing device in the third dimension corresponds to a circular movement of said ring, which complicates the work.

Furthermore, transmitter / receiver devices are known arranged directly adjacent to the frame of a monitor for two-dimensional control, wherein a human finger, a pen or the like is held against the screen and thereby breaks one or more light beams which are emitted parallel to the monitor. . With the help of receivers arranged in a suitable manner, the xy coordinates of the finger are determined on the screen. An example of this is shown in EP-A3-0 121 840, which describes a device for determining the position of an object in the field of view of a monitor. The device comprises two point-shaped radiation sources arranged in each lower corner of the monitor frame, a retro-reflecting strip extending along the two side edges of the monitor and its top, and a one-dimensional detector arranged behind each of the radiation sources. The radiation from the two radiation sources covers the entire field of view, and if a finger is applied to the screen in the field of view, a shadow will appear on each of the one-dimensional detectors. This makes it possible to trigonometric calculate the xy coordinates of the finger on the screen.

However, this known solution can not be used for the application computer mouse, nor is it suitable for use for three-dimensional control, even if the document proposes such three-dimensional control by making the retro-reflective strip. wavy.

The object of the invention is to provide a device and a method for three-dimensional control, in particular but not exclusively for three-dimensional control of a monitor, written prior art. which does not have the disadvantages of the above solution according to the invention should thus not require the user to hold any object in the air.

The solution must also be useful both for remote control of a cursor or pointer, ie for the area of use pointing devices or positioners, and for control directly on a monitor, ie for the area of use of touch screens.

Another object of the invention is that the solution should have a simple and inexpensive construction and be insensitive to environmental disturbances.

Another object of the invention is that the solution should be such that the user can perform linear control commands or movements in three directions directly connected to the corresponding control directions on the screen. These and other objects are achieved with a device and a method according to the appended claims.

The device according to the invention is thus characterized in that it has at least two mutually spaced, radiation emitting points, hereinafter referred to as radiation points, a two-dimensionally position-sensitive detector surface which is located at a distance from and is irradiated by the radiation points, and means for determining the position in three dimensions of an object, which is freely movable in a control space between the radiation points and the detector surface and which object thereby casts two shadows belonging to each of the radiation points on the detector surface, which position determination is based on the absolute and relative positions on the detector surface, and for providing control information corresponding to the current position of the object.

A distinctive feature of the invention is that one and the same detector surface is used for the control in all three directions, which for example differs from the solution in the above-mentioned EP-A3-0 121 840. Thus, according to the invention, the position of one shadow on the detector surface for a first position of the object may coincide with the position of the second shadow on the detector surface of another position of the object.

According to the invention, the position of the object in a first direction (x) and in a second direction (y) in a plane (xy plane) parallel to the detector surface can be determined on the basis of the positions of the two shadows. According to the invention, the position of the object, ie the distance from the object, can be calculated on the basis of the mutual position of the shadows. in the third direction (z), the detector surface, When the object is relatively close to the radiation points and relatively far from the detector surface, the two shadows will be relatively far apart. As the object is moved closer to the detection surface 186, 5 tory surface, the shadows will move towards each other. Thus, the shadows will move in relation to each other in a predictable way if the object is moved from and towards the detector surface. However, how much the distance between the shadows changes for a certain change in the object's distance to the detector surface may depend on the absolute position of the shadows on the detector surface.

The invention can for instance be realized as a three-dimensional "computer mouse", which is suitably connected to the monitor but which, unlike a traditional computer mouse, is normally intended to stand still on a table surface or the like. The detector surface can rest directly on the table surface, and the radiation points can be connected to and arranged at a certain height above the detector surface. The user can perform the three-dimensional control by guiding his finger, a pen or other object in the free control space above the detector surface in three directions. The invention can also be realized in direct connection with a monitor for providing a three-dimensionally controllable touch screen. The radiation points and the detector surface can then be applied at two opposite edges of the screen. In this case, a control space with a certain thickness is obtained immediately in front of the screen and parallel to it. The user can then use his finger, a pointing device or the like to perform a three-dimensional control in the control space, whereby the movements that the user performs directly correspond to the same control direction on the screen.

For both of the above applications of the invention, no moving parts are required to achieve the three-dimensional control.

How the actual calculation of the control information can be performed will not be discussed in detail, since this can be performed in several ways, for example with a pre-programmed microprocessor, and depends on the geometrics of the invention. In general, however, electrical output signals from the detector surface are used as parameters for producing control information or control signals for the three-dimensional control. How this information is to be calculated in the individual case is also entirely dependent on the actual application of the invention. In most cases, however, the control information can be calculated with simple trigonometric relationships.

The detector surface can be built up of a plurality of separate detector elements distributed in two directions over the detector surface, and can for instance be built up of CCD elements. The detector surface can consist of a plane, the surface or of a curved surface. The number of radiation points and the number of shadows can be more than two, but in order for the determination of the object's distance from the detector surface to be possible, one must always know for sure from which radiation point a certain shadow originates. In the simplest case - two sources of radiation distributed across the insertion direction of the object in space, the shadows will always be on the same side of each other. Other embodiments of the invention, however, include means which make it possible to distinguish between the different shadows. This can be accomplished in several ways. One way is that the radiation points emit different radiation from each other, for example with different power, or modulation, frequency and that the detector surface is arranged to distinguish radiation from the different radiation points. Another way is that the radiation points emit their radiation in different, successive time windows, whereby the detection is synchronized with these time windows. of memory function may be needed.

Some type A third way is to arrange a radiation-breaking gap or the like in front of each radiation point and to distinguish the different shadows on the basis of the interference pattern on the detector surface. 10 15 20 25 30 35 5027185 7 A case where it may be necessary to be able to distinguish the different shadows from each other is if the object in a first position casts shadows originating from a first and a second radiation point, while the object in a second position, where the shadow from the first radiation point falls outside the detector surface, it casts shadows originating from the second radiation point and a third radiation point.

According to the invention, the radiation emitted from the radiation points can be generated by one and the same radiation source from which the radiation is suitably directed to the different radiation points. As an alternative, such a common source of radiation may be mobile (rotating and / or linearly reciprocating) to emit radiation from different points in different time windows. The radiation can be virtually any electromagnetic radiation, preferably invisible radiation, such as IR radiation, to prevent interference from backlighting.

These and other features, advantages and uses of the invention will become apparent from the following description of embodiments of the invention.

Fig. 1 is a schematic perspective view for illustrating the principle of the invention.

Fig. 2 is a schematic illustration of an arrangement in which the invention is realized as a desktop computer mouse.

Fig. 3 is a schematic plan view for explaining "dead zones" in a control space.

Figs. 4A and 4B are schematic plan views of an embodiment with four radiation points.

Fig. 5 is a schematic illustration of an arrangement in which the invention is realized in direct connection with a display for providing a three-dimensionally controllable touch screen. 502 186 10 15 20 25 30 35 8 Fig. 1, to which reference is now made, shows a planar, rectangular, two-dimensional detector surface 10 extending in the xy plane of the orthogonal xyz coordinate system plotted in the figure. The detector surface 10 is made up of a number of separate detector elements 11 distributed as an xy matrix over the entire surface. For the sake of simplicity, only a few of these elements 11 are schematically shown. The number of detector elements 11 and the size of the active surface of each of them depend on, among other things, the desired resolution and accuracy of the device. One way to realize the detector surface 10 is to use one or more CCD units. For the sake of simplicity, Fig. 1 also does not show the electrical connections to the detector surface 10 and the signal processing electronics required for evaluating a detection result and producing control information, as these parts can be realized by a person skilled in the art without further explanation.

Two separate radiation emitting points P1 and P2, called radiation points, are arranged at a distance from the detector surface 10 in the z-direction and are located at a mutual distance in the x-direction. Each radiation point P1, P2 irradiates the entire detector surface 10. The radiation can be substantially any electromagnetic radiation that can be detected by the detector elements 11, and as a preferred example, IR radiation is used to eliminate interference from visible backlight.

The space between the radiation points P1 and P2 and the detector surface 10 is completely free and is called the control space, generally denoted by S. This space is intended to receive an object 12, preferably a relatively narrow, elongated object, such as a finger, a pen, a special purpose designed pointing device or equivalent. In the example, it is assumed that the object 12 is elongate and that it is inserted in its longitudinal direction in the control space S substantially in the positive y-direction. The dimensions of the object 12 relative to the arrangement in general are such that the object 12 on the detector surface 10 throws two elongated shadows S1 and S2 belonging to the radiation points P1 and P2, respectively. The lengths of the shadows S1 and S2 in the y-direction are denoted y1 and y2, respectively, and their mutual distances in the x-direction are denoted Ax.

Based on the absolute and relative positions of the shadows S1 and S2 on the detector surface 10, the position of the object 12 in the control space S can be calculated in three dimensions (x, y, z). The position in the x-direction can be calculated by reading the positions of the shadows S1 and S2 in the x-direction, and if a shadow covers more than one detector element 11, for example, the edge of the shadow with a maximum x-value can be considered to represent the x-coordinate of the shadow.

The position of the object 12 in the y-direction can be calculated by reading the shadow lengths y1 and y2 if consideration of the position determination of the object in the x- or z-direction is taken into account.

The position of the object 12 in the z-direction, ie its perpendicular distance from the detector surface 10, is calculated by a calculation of the mutual distances of the shadows S1 and S2 in the x-direction.

For the arrangement shown schematically in Fig. 1, it will be appreciated that the left shadow S2 will always originate from the right radiation point P2 and vice versa, regardless of the position of the object 12 in the control space. Thus, in this case, no means are required to distinguish the shadows S1 and S2 from each other. It should be noted that the surface area which the shadow S2 covers on the detector surface 10 is irradiated by the radiation point P1 and that the surface area which the shadow S1 covers is irradiated by the radiation point P2. The shadows S1 and S2 are thus "half shadows", but they still give a clearly and unambiguously detectable change in irradiation level on the detector surface 10.

In order for the position of the object 12 to be calculated, the object 12 must thus cast at least two shadows on the detector surface 10. In the arrangement in Fig. 1, however, there are certain "dead zones" within which the object 12 cannot cast two shadows on the detector surface 10. Fig. 3, which illustrates the arrangement of Fig. 1, shows the xz plane, the dashed lines 15-18 being drawn between the respective radiation sources P1, P2 and the x-direction. If the object 12 were located within the two triangles T1, the boundary edges 19 of the tector surface, and T2, it would cast only a single shadow on the detector surface 10. If, on the other hand, the object 12 is within the shaded triangle T3, it will cast the to determine the position required two shadows S1 and S2 on the detector surface 10. As marked by dashed lines 21, 22 and 23 in Fig. 3, suitable means in the form of walls or the like can be provided to limit the possibilities of movement of the object 12 to an area. within which two shadows are always obtained.

Figs. 4A and 4B show an example of an arrangement with four radiation points P1-P4. In Fig. 4A, the object 12 is in a position A, and for this position A the object 12 casts two shadows S1 and S2 belonging to the two left radiation points P1 and P2, while the two right radiation points P3 and P4 do not cast any shadows on due to the limited extent of the detector surface to the left in Fig. 4A. In Fig. 4B, the object 12 is in a position B further to the right and slightly higher up compared to position A, and for this position B the object 12 casts two shadows S2 and S3 belonging to the two middle radiation sources P2 and P3, while they the two external radiation sources P1 and P4 do not cast any shadows due to the limited extent of the detector surface to the right and to the left in Fig. 4B. The two positions A and B are specially selected to illustrate the fact that the shadow S2 constitutes the left shadow (lowest x-coordinate) in position A, while the same shadow S2 constitutes the right shadow (maximum x-coordinate) in position B Thus, for this arrangement, there must be means which make it possible for each possible position of the object to identify to which radiation source a certain shadow belongs. As stated above, this can be accomplished in several ways. The radiation points P1-P4 can 10 15 20 25 30 35 In the free control space. 502 186 ll, for example, is turned on and off in sequence, so that each radiation point emits radiation only within a predetermined time window. It will be appreciated that such time windows should suitably be so short that the object in principle stands still for an entire sequence. Another variant is to move one and the same source of radiation between the four points. Furthermore, the radiation can be modulated in different ways from the different radiation points, whereby the detector surface must be designed so that it can be determined from its output detector signals which type of radiation has been received.

Fig. 2 shows an arrangement where the invention comprises partly a number of light sources P1-P3 which are supported via legs 22 realized as a "computer mouse" 20, base plate 21 whose upper side constitutes the detector surface 10, the bottom plate 21. The computer mouse 20 is intended to be mounted stationary on a substrate and to be connected via a line 23 to a personal computer 30 or the like for three-dimensional control of a screen pointer 31 on a computer screen 32. The arrangement in Fig. 1 can be controlled by means of a user's finger, which is moved in three dimensions in Instead of a finger, one can imagine other objects, such as a pencil. To achieve the same possibilities for "mouse click functions" as in a traditional computer mouse, such an object can be provided with one or more manually controllable function keys which the user can influence when the object has been brought into the desired position in the control space.

Fig. 5 shows an arrangement where the invention is realized as a three-dimensionally controllable touch screen 40, where a detector surface 10 with separate detector elements 11 is arranged substantially horizontally along the lower edge of the screen 40 and where a number of radiation points P1-P7 are distributed along the upper edge of the screen 40. Fig. 5 also shows a pointing device 50, which replaces the finger in the previous examples and which is provided with one or more function keys 51 which the user can operate to "click" on different function fields. in the monitor.

The embodiments of the invention described above can be modified in many ways within the scope of the appended claims.

The detector surface can be built up of several sub-surfaces, and it does not necessarily have to be flat.

The detector surface can be designed as a continuously position-sensitive surface in two dimensions, instead of a plurality of separate detector elements as above.

The object, which is moved in the control space, can have other shapes than the elongated shape shown in the figures.

The object can be connected to the device via something so that the object is always available at the device. gon form of universal joint, The position of the object in z-direction can be calculated based on the relative position of more than two shadows for better accuracy.

The distance between the radiation points can be varied, which gives different resolution.

The radiation points can be distributed in other directions than in the x-direction, for example only in the y-direction. For this alternative, it would be possible to obtain some shaded areas with "full shadow" (shadow from both points of radiation) and some shaded areas with "half shadow" (shadow from only one point of radiation), but the distinction between the shadows would be quite possible.

Regarding the extent of the radiation points, it applies that in practice these can have very varying extent which deviates from "point-shaped", and the only limitation is really that one obtains the necessary shadows on the detector surface.

The generation of control information based on the output signals of the detector surface can be varied depending on the desired properties of the three-dimensional control. Thus, one can choose a signal processing which for a certain range of motion of the object gives an exactly equal range of motion of the mouse pointer, or one can have a signal processing which depends on the current position of the object. O

Claims (19)

502 186 10 15 20 25 30 14 PATENT REQUIREMENTS Device for three-dimensional control, in particular for three-dimensional control of a computer monitor, characterized by: at least two mutually spaced, radiation emitting points (P1, P2), hereinafter referred to as radiation points, one in two dimensions position-sensitive detector surface (10) located at a distance from and irradiated by the radiation points (P1, P2), and means for determining the position in three dimensions of an object (12), which is freely movable in a control space (S) between the radiation points (P1, P2) and the detector surface (10) and which object (12) thereby casts on the detector surface (10) two shadows (S1, S2) belonging to each of the radiation points (P1, P2), which position determines - (S1, S2) relative positions on the detector surface 10, and for emitting ning is based on the absolute position information (x, y, z) of the shadows and the current position of the object (12). Device according to claim 1, characterized in that the detector surface (10) is built up of a plurality (11) distributed in two directions (x, y) over the detector surface (10). speech separate detector elements Device according to claim 2, characterized in that the detector surface (10) Device according to one of the preceding claims, characterized in that the radiation points (P1, P2) are located at a mutual distance at least in one direction and comprise a CCD unit. (x) parallel to the detector surface (10). Device according to claim 4 in combination with claim 2 or 3, characterized in that beam (P1, P2) are located at least in a direction (x) at mutual distances which are parallel to the detector surface (10). ) and parallel to one of the two directions soxv1s6p 15 (x, y) in which the separate detector elements (11) of the detector surface (10) are distributed. Device according to one of the preceding claims, in that the possibility of the object (12) in the control space (S) is limited in such a characteristic mode of movement that the object (12) for each possible position in the control space (S) always cast at least two shadows on the detector surface (10). Device according to one of the preceding claims, characterized in that the radiation emitted from the radiation points (P1, P2) is an IR radiation. Device according to one of the preceding claims, characterized in that the radiation points (P1, P2) emit mutually different radiation and that the detector surface (10) is arranged to distinguish radiation from the P2s). Device according to one of the preceding claims, (P1, P2) emits radiation in different, successive time windows. different radiation points (Pl, k e n n e t e c k n a d of that the radiation points Device according to one of the preceding claims, characterized by a radiation source common to the radiation points (p1, P2). 11. ll. Device according to any one of the preceding claims, characterized in that the device in addition to said two radiation points (P1, P2) comprises one or more ...), P2, P3, P4) at a mutual distance and irradiates the detector surface (10). additional radiation points (P3, P4, all radiation points (P1, being located Device according to one of the preceding claims, characterized in that the device constitutes a separate unit unit (20) of the computer mouse type, which is connectable to a computer control ball and corresponds to (30) dimensional control of a pointer or marker a monitor (32). Device according to one of the preceding claims, for triangular (31) edges, characterized in that, in order to provide a three-dimensionally controllable touch screen (40), the radiation points (P1, P2) are mounted at a first edge of a monitor and that the detector surface (10) is mounted at an opposite edge of the monitor, so that the guide space (S) in which the object (12) is freely movable is constituted by a space immediately in front of the monitor (40). ). Device according to any one of the preceding claims, wherein said object (50) forms part of the device itself, characterized in that the object (50) is provided with a manually actuatable activating means (51), which upon activation generates an activation signal which issued in conjunction with said control information. Method for three-dimensional control, in particular for three-dimensional control of a computer screen, characterized by the following measures: emitting radiation from at least two mutually spaced points (P1, P2), hereinafter referred to as radiation points, to move in three dimensions an object (12) in a control space (S) between the radiation points (P1, P2) and a two-dimensional position-sensitive detector surface (10), which is located at a distance from and is irradiated by the radiation points (P1, P2). ), so that the object (S) thereby on the detector surface (10) casts two shadows (S1, S2) belonging to each of the radiation points (P1, P2), that based on the absolute and re of the shadows (S1, S2) lative positions on the detector surface (10) produce control information corresponding to the current position (x, y, z) of the object (12) in three dimensions in the control space (S). Method according to claim 15, characterized in that an elongate object (12, 50) is used as said object (12), so that the two shadows (S1, S2) thereby consist of an elongate shadow starting from a edge of the detector surface (10). 10 so2t1ssA 17 Method according to claim 16, characterized in that, as said elongate object (12), it utilizes a finger of a user. A method according to any one of claims 15-17, characterized by using the control information for sensing three-dimensional control of a cursor (31) or pointer of a monitor (32). A method according to any one of claims 15-18, characterized by using the control information to provide a three-dimensional touch screen (40).