JPH06230897A - Position indicator - Google Patents

Abstract

PURPOSE:To move a cursor position without contacting. CONSTITUTION:This indicator is provided with a cursor control means 5 for moving the cursor position on a screen, an optical pattern projecting means 7 for projecting an optical pattern 6 which can be optionally moved and whose moving direction can be discriminated by an operator, an optical pattern movement detection means 8 for detecting the movement of the optical pattern within the projecting area of the optical pattern and a cursor moving position calculating means 9 for calculating a cursor moving direction and a moving distance based on the detected output of the optical pattern movement detection means 8. The cursor control means 5 is controlled by the calculated output of the cursor moving position calculating means 9 and the cursor position on the screen 1 is moved. Thus, a cursor can be moved without contacting a cursor manipulating device.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an instruction device for inputting coordinate information and cursor movement information to a computer or the like,
In particular, the present invention relates to a position pointing device that moves a display position of a cursor or the like displayed on a screen in a non-contact manner.

[0002]

2. Description of the Related Art Conventionally, as a pointing device for moving a so-called position indicator such as a cursor displayed on a display screen of a display device such as a computer or a word processor, a mouse, a light pen, a trackball, a joystick or a cursor is used. Move keys are widely used.

FIGS. 14A and 14B are explanatory views of a mechanical mouse which is the most widely used pointing device at present. FIG. 14A is a side view and FIG. This kind of mouse 131 has a ball 1 that freely rolls inside.
32 is provided. When the mouse 132 is manually moved on the desk top surface or the like while contacting the ball 132 with a stationary flat surface such as the desk top surface, the ball 132 is moved according to the movement amount and the moving direction of the mouse 131. Rotate.

The rotation amount and the rotation direction of the ball 132 are two orthogonal rollers 133 which are in contact with the ball.
And 136 and encoders 135 and 136 respectively coupled to
Detected by. Information on the relative movement of the mouse 131 with respect to the desk top surface can be obtained from the rotation amount and the rotation direction signal of the ball detected by the two encoders.

The relative movement information of the mouse 131 with respect to the desk top surface obtained in this way is used as position designation information for the computer. The prior art relating to this type of mechanical mouse is disclosed in JP-A-61-61
-282914 and Japanese Utility Model Laid-Open No. 61-78342 can be mentioned. In addition, as the mouse, an optical type is also known, and the main body that stores the light source and the photodetector is moved on a special underlay with a regular lattice pattern engraved on it, and a special underlay that is traversed when moving is used. The movement information is detected by counting the number of the upper lattices, and is disclosed in, for example, Japanese Patent Laid-Open Nos. 57-207929 and 57-207930.

[0006]

As is clear from the principle of the mechanical mouse described above, it is necessary to operate the mechanical mouse in the vicinity of the computer or its terminal and on a stationary plane. Therefore, in practice, there is a restriction that it cannot be used except on a flat surface such as a desk surface on which a computer or its terminal is installed.

Also in the optical mouse, as described above, the main body in which the light source and the photodetector are housed is moved on the dedicated underlay on which the regular lattice pattern is engraved, and the number of lattices on the dedicated underlay traverse during the movement. This is the same as the mechanical type in that the movement information is detected by counting the number of points, which can be used only in the vicinity of the computer, and requires a stationary object.

Not only the mouse but also all the conventional pointing devices have the point that they can be used only in the immediate vicinity of the computer and require a stationary object.
The stationary object corresponding to the top surface of the desk in the case of a mouse is the CRT in the case of a light pen, or the pointing device itself in the case of a trackball or joystick.
Therefore, there is a common restriction that it cannot be used except near the computer or the desk on which the terminal is mounted.

Because of the above reasons, for example, an output screen of a computer is dynamically projected on a so-called electronic overhead projector (hereinafter, abbreviated as electronic OHP), and a lecture, a presentation, a conference or the like is performed by using this. In the situation, the speaker (explainer) is the electronic O
In order to point a cursor on a computer output screen projected by HP, if a conventional pointing device is used, it is necessary to operate a mouse or the like toward the computer.

This means that it is difficult for the speaker to change his / her posture, direction and place while talking, and it is also difficult to talk toward the screen on which the electronic OHP is projected or the audience. Is hindering his natural behavior. This not only burdens the speakers, but also impedes the natural understanding of the audience and reduces the efficiency of the lectures and conferences.

On the other hand, the conventional pointing device must be operated by hand when pointing. For this reason, when pointing is performed while inputting with the keyboard, it is necessary to release at least one hand from the keyboard and operate a pointing device such as a mouse. in this way,
Removing the keyboard from the keyboard in the middle of typing greatly reduces the efficiency of keyboard input, especially when the so-called blind typing (touch typing) is typing without looking at the keyboard. However, there is also a problem that when a finger is returned from the pointing device to the keyboard, a typing error is likely to occur, and the user is required to visually check the keyboard, which increases fatigue.

An object of the present invention is to solve the above problems of the prior art and provide a pointing device which can be used in a contactless manner by operating in the air without directly using a stationary surface such as a desk or a contact operating device. By doing so, a position pointing device that enables efficient pointing in a free posture when using an electronic OHP and natural and efficient pointing without touching the keyboard during touch typing is possible. To provide.

[0013]

In order to achieve the above object, the present invention provides an information processing means including a cursor 2 for indicating a display position on a screen 1 as shown in the basic configuration diagram of FIG. Output image projecting means 4 for projecting the output image of No. 3 on the screen, and cursor control provided for the information processing means 3 for moving the position of the cursor projected on the screen to indicate a desired display position. Means 5, light pattern projection means 7 for projecting a light pattern 6 which can be arbitrarily moved by the operator and whose movement direction can be identified, and a light pattern for detecting the movement of this light pattern within the projection area of the light pattern. A movement detection means 8, a cursor movement position calculation means 9 for calculating the cursor movement direction and movement distance based on the detection output of the optical pattern movement detection means 8, and this cursor movement position. By moving the cursor position on the screen 1 by the operation output of the calculation means 9 controls the cursor control means 5, the screen 1 in a non-contact
It is characterized in that the display position of the upper cursor 2 is moved.

[0014]

When the operator moves the light pattern projection means 7 to move the light pattern projection area, the light pattern 6 in the light pattern movement detection means 8 moves. The light pattern movement detecting means 8 detects the movement of this light pattern and inputs movement information to the cursor movement position calculating means 9.

FIG. 2 is an explanatory diagram of a projected light pattern, and the light pattern has such a shape that the movement of the pattern can be detected in principle by crossing the light pattern movement detecting means (photodetector) 8. If so, it may be an appropriate one, for example, a two-dimensional random pattern of (a), a regular lattice pattern of (b), a shape close to the points of (c), a cross line of (d),
In addition, non-repetitive geometric patterns such as arcs and lattice patterns can be used.

FIG. 3 is an explanatory view of an example of a pattern that cannot be used. As shown in FIG. 3, there is a so-called ladder chart pattern in which bright and dark linear bands are arranged at equal intervals. Such a pattern cannot detect the movement in principle even if it moves in a direction parallel to the band of the ladder, and is not suitable for detecting the moving direction and the moving distance of the cursor.

As the method of projecting the light pattern 6, a method of simply projecting a shadow, a method of irradiating the diffraction grating with light to generate a diffraction pattern having a regular grating pattern, or a method of irradiating the holographic optical element with light is used. Any method that can generate a light pattern, such as a method of generating a desired pattern by using the above method, may be used. Further, a method of generating so-called laser speckle can also be used.

Further, the light pattern is not limited to the light intensity pattern, and any pattern due to a change in physical characteristics of light such as a pattern including a change in color or a pattern including a change in polarization state can be appropriately used. . As described above, the light pattern movement detecting means 8 for detecting the light pattern 1 generated and projected by the light pattern projecting means 7
Consists of a photodetector. This photodetector may be appropriately selected according to the type of physical characteristics of the light pattern.

When the type of physical characteristic of the light pattern is a change in light intensity, a general photodetector for detecting light intensity can be used as it is. Most photodetectors fall into this category, including photodiodes, CCD sensors, position detectors (PSDs), photomultipliers, photocells,
A sensor that can detect the light intensity, such as a pyroelectric sensor, can be used.

When the physical characteristics of the light pattern are other than the light intensity, the light pattern can be detected by installing an appropriate filter in front of the light intensity detector. For example, in the case of a light pattern due to a change in the polarization state, a polarization filter may be installed in front of the light intensity detector, and in the case of a pattern due to a color change, an appropriate color filter may be installed in front of the light intensity detector. Good.

The photodetector is not limited to the one described above, and any photodetector may be used as long as it can detect a change in the physical characteristics of the projected light pattern. In general, detection of light pattern movement is not possible with one detector and requires multiple detectors. Specifically, how many detectors need to be combined and arranged can be appropriately selected according to the shape of the pattern.

If a plurality of photodetectors are required spatially,
A plurality of photodetectors may be arranged as appropriate, but an element in which a photodetector array is built in from the beginning such as a one-dimensional or two-dimensional CCD sensor, or several photodetectors such as a split photodiode. It is also possible to use elements that are spatially arranged in a suitable shape. The pattern movement detection method is
Any method may be used as long as the movement of the target pattern can be detected, but it is preferable to select it appropriately according to the morphological type of the pattern.

FIGS. 4 (a) and 4 (b) show how to detect the movement of an optical pattern.
It is an explanatory view of a method, for example, a CCD camera of a light pattern
Using a two-dimensional array sensor like
The light pattern is detected, and as shown in FIG.
₁The optical pattern signal I (t detected by₁, X, y),
The optical pattern signal I (t ₁
+ Δt, x, y) cross-correlation function CI (τ_x, Τ
_y) CI (τ_x, τ_y) = 1 / k∫∫I (t_1,x, y) ・ I (t₁+ Δ
t, x-τ_x, y- τ_y) When dxdy is calculated, the correlation peak is
Position of (τ_xp, Τ_yp), The time t₁And t₁Between + Δt
The distance and direction moved to can be obtained.

As described above, if the light pattern has a shape that can detect movement in principle, this method can always detect movement. Further, instead of using the two-dimensional array sensor, two non-parallel one-dimensional array sensors are used in combination, and the cross-correlation function is calculated one-dimensionally with respect to the output from each one-dimensional array sensor, The movement component in the direction along the array direction of the array of the one-dimensional array sensor may be obtained.

However, a two-dimensional array sensor or two 1
The use of a two-dimensional array sensor is also large in terms of equipment.
It is uneconomical to handle a large amount of data such as two one-dimensional data from one one-dimensional array sensor in terms of time and cost of data storage / transfer, and for such a large amount of data. Since it takes a lot of time and the power of a computer to calculate the correlation function by using the above, it is not always desirable if there is another more economical and convenient method.

An example of a detection method which is simpler and more economical than the method of combining the array sensor and the cross-correlation method will be described below. 5, 6 and 7 are the first of the simple detection methods described above.
As an example, it is an explanatory diagram of a method of projecting a light pattern having a regular lattice pattern and detecting the movement of the light pattern with at least two photodetectors per direction.

In the following description, one-dimensional movement will be described for the sake of simplicity, but when two-dimensional movement is detected, two one-dimensional movement detection mechanisms described below may be combined. As shown in FIG. 5, two sensors 80a and 80b in the X direction and two sensors 80c and 8 in the Y direction.
The light pattern 6 projected on the photodetector 80 consisting of 0d
When the regular grid pattern of 0 moves in the direction of the arrow (X direction) shown in the figure, the sensors 80a and 80b of the photodetector 80 produce outputs as shown in FIG.

The light pattern changes from sensor 80a to sensor 80.
When moving towards the b, the sensor 80 at a certain time t ₁
Since the portion of the light pattern projected on a is projected on the sensor 80b after a certain time Δt, the outputs of the sensors 80a and 80b are output only for a time corresponding to the distance between the sensors 80a and 80b and the moving speed of the light pattern. It becomes a shifted and similar shape.

The number of sensor output peaks or the number of zero crosses equivalent thereto is proportional to the amount of movement of the pattern.
In addition, the direction of movement can be known from which of the output of the sensor 80a and the output of the sensor 80b leads the phase. That is, the light pattern is moving from the side with the advanced phase to the side with the delayed phase.

FIG. 7 is a block diagram of a signal processing system for calculating the movement position of the cursor by detecting the movement of the light pattern from the output of the photodetector, and it shows two sensors 80a, 80 in the x direction.
The detection outputs of the two sensors 80c and 80d in the b and y directions are the number of peaks (or the number of zero crosses) of the above-mentioned sensor output signal through the digital converters 70x and 70y each including an amplifier, a noise cut filter, and a binarization circuit. Of the phase difference between the output of the sensor 80a and the output of the sensor 80b (positive / negative) is detected.
Input to 0z.

The result calculated by the arithmetic processing unit 70z is input to the cursor control means 5 of the information processing apparatus 3 such as a computer and moves the cursor on the display screen. Further, as a second example of the above-mentioned simple detection method, a random pattern such as laser speckle may be projected on the photodetector having the same configuration as in FIG. 5, instead of the regular lattice pattern.

This method is based on the "movement information detecting method of random space pattern" already filed by the present applicant (JP-A-1-
No. 287468). The number of peaks of the output of the sensor or the number of zero crosses equivalent to it is proportional to the amount of movement of the pattern when viewed as an average,
There is a random error due to the randomness of the light pattern.

The property that the direction of movement of the light pattern can be known by which of the output of the sensor 80a and the output of the sensor 80b leads the phase is the same as in the case of a regular lattice pattern. In addition, the third of the above-mentioned simple detection methods
As an example, as shown in FIG. 8, a position detecting element (so-called PSD) 8 which is a sensor for detecting a place where light is applied
1 is used, the light pattern 6 is sufficiently small with respect to the aperture 81D of the photodetector 81 instead of the repetitive pattern such as the above-mentioned grating.
It is also possible to project 1 and obtain a signal indicating the moving direction and moving distance of the light pattern directly from the light detecting means. Since the position detecting element such as PSD directly outputs the information of the position where the light is applied, it is possible to reduce the circuit scale by using this.

It should be noted that at least two one-dimensional ones or two-dimensional ones can be used as the position detecting elements such as PSD. Examples of the position detecting element such as PSD include a one-dimensional position detecting element S3270 and a two-dimensional position detecting element S1 manufactured by Hamamatsu Photonics KK
869 or the like can be used. In addition, when a two-dimensional PSD is used as the light pattern, the PSD, such as a point, a small circle, or a small rectangle, is projected when projected onto the PSD aperture.
Any spot having a spot smaller than the opening of

Further, in order to project the light pattern as a small spot onto the aperture of the PSD, the image of the light pattern projection means 71 is formed by using the image forming optical system (imaging lens 90) as shown in FIG. It is desirable to form an image on the PSD light receiving portion (photodetector 81). Furthermore, when one-dimensional PSD is used, a sufficiently thin line may be used as the light pattern.
When detecting a two-dimensional position, two one-dimensional PSDs and two lines may be combined.

Further, in order to reduce noise due to the influence of visible ambient light such as indoor illumination light, and to make the projected image such as the grating for the pointing device invisible to the naked eye, so as not to disturb the OHP projected image. The configuration of
A light source for projecting a light pattern (semiconductor laser 721) having a wavelength other than visible light is used, and a band-pass filter (band-pass color filter 821) that transmits only the wavelength of the light source for projection to the detector of the light pattern movement detection unit. ) Is used.

In order to minimize the light output of the light pattern projecting means and to further eliminate noise due to the influence of ambient light such as indoor illumination light, the light output of the light pattern projecting means is time-modulated to obtain the light pattern. It is desirable to adopt a so-called optical synchronization detection (lock-in detection) system in which the movement detection means demodulates to separate the noise light component and the pattern light component.

The present invention significantly improves the degree of freedom in using a pointing device by enabling pointing in the air, and can be used in all fields requiring the degree of freedom in pointing.
It goes without saying that the application is not limited to pointing for electronic OHP and pointing for touch typing.

[0039]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the position pointing device according to the present invention will be described in detail below with reference to the drawings. FIG. 10 is a schematic diagram for explaining the main configuration of one embodiment of the position pointing device according to the present invention, in which 62 is a projected light pattern, 621 is an enlarged view of the projected light pattern, 72 is a light pattern projection means, 7
Reference numeral 21 is a semiconductor laser, 722 is a projection lens, 723 is a computer hologram for pattern generation, 724 is a semiconductor laser drive circuit, 82 is a split photodiode constituting a photodetector, 821 is a bandpass color filter, and 822 is a photodetector circuit. (Electric circuit for divided photodiode), 90 is a cursor movement position calculation means.

FIG. 11 shows the position pointing device of FIG.
FIG. 11 is an explanatory diagram of an overall view applied as a P pointing pointer, in which the same reference numerals as those in FIG. 10 correspond to the same portions, 1 is an electronic OHP screen which projects a projected image 21 of an electronic OHP, and 2 is a projected image in the projected image. A displayed cursor, 30 is a computer, 40 is an electronic OHP, 50 is an explainer, 90
Is a cursor movement position calculation device.

In FIGS. 10 and 11, the electron OHP is used.
By projecting the position-indicating light pattern 62 on the screen 1 or in an area including this screen, the person who explains the electronic OHP projection image 21 (pointing device = position-indicating device operator) moves the cursor 2 in the projection image. The contents are explained to the audience 51 while moving.

It should be noted that noise due to the influence of visible ambient light such as indoor illumination light is reduced, and the image of the projected light pattern such as the grating for the position pointing device is made invisible to the naked eye.
In order not to obstruct the projected image, as shown in FIG. 10, a light source for projecting an optical pattern (semiconductor laser 7
21) that has a wavelength other than visible light is used, and a bandpass filter (bandpass color filter 8) that transmits only the wavelength of the projection light source to the detector 82 of the light pattern movement detection unit is used.
21) is used.

As shown in this embodiment, the computer 3
It is used as a pointer when the 0 output screen is projected on the electronic OHP screen 1 by the electronic OHP 40. The beam light from the semiconductor laser 721 turned on by the semiconductor laser drive circuit 724 passes through the projection lens 722 and the computer hologram 723 for generating a light pattern to become a light intensity pattern of a regular lattice pattern 621, and thus an electronic OHP screen. It is projected in the vicinity of 1.

The projected light pattern 62 moves on the surface of the photodetector by changing the posture of the pattern projection means 72 held by the explainer 50. This movement of the light pattern is detected by the divided photodiode 82 provided in the vicinity of the electronic OHP screen 1, and the movement information is passed through the cursor movement position calculation device 90 and the computer 30.
It is input to the cursor control means provided in the control unit 2 and controls the position of the cursor 2 on the output screen (electronic OHP screen 1).

The cursor 2 is moved by manually changing the posture of the housing of the light pattern projection means 72 to move the projection pattern in a desired direction. Computer hologram 7
23 is a semiconductor laser 721 and a projection lens 722.
It is designed and manufactured so that a desired pattern can be projected by combining with. In order to make the projected light pattern grating 621 for the pointer invisible to human eyes, the semiconductor laser 721 uses infrared light emission.

In order to eliminate the influence of ambient light such as room lighting, a bandpass color filter 821 that transmits only the wavelength of the laser light may be placed in front of the split photodiode 82. Further, in order to reduce the influence of noise of ambient light, the semiconductor laser drive circuit 724 and the divided photodiode electric circuit 822 may be set to perform optical synchronization detection. For optical synchronization detection, for example, a light modulation photo IC'S4285-40 'manufactured by Hamamatsu Photonics KK or an electric circuit having a function equivalent to this can be used.

As the projected light pattern, speckle which is a two-dimensional random pattern may be used instead of the regular lattice pattern as described above. In order to generate speckle, instead of the computer generated hologram 723 of FIG. 10, an object such as frosted glass having rough and random irregularities as compared with the wavelength of light may be used. FIG. 12 is a schematic diagram for explaining a main part configuration of another embodiment of the position pointing device according to the present invention, in which 120 is a display of a personal computer or word processor, 121 is its display screen, 122 is a keyboard, 1
Reference numeral 23 is an operator's hand, 124 is a light pattern projection means attached to the operator's fingers, and 125 is a light detection means installed on the display side, which is applied to a computer for blind touch or a cursor position pointing device for a word processor. It was done.

In the figure, the structure of this embodiment has exactly the same function as the above embodiment, but the light pattern projection means 124 is miniaturized so that it can be attached to a finger. The operator does not remove the finger from the keyboard 122,
By changing the posture of the pattern projection unit, it is possible to move the cursor position and execute the pointing operation.

FIG. 13 is a schematic view for explaining still another embodiment of the position pointing device according to the present invention, which is applied to a cursor position pointing device for a computer or word processor for blind touch as in FIG. And
The light pattern projection means 124 is attached to the operator's face or head, and by changing the direction of the face, the position of the cursor 2 displayed on the display screen of the display 120 can be moved to perform a pointing operation. It is a thing.

[0050]

As described above, according to the present invention,
It is possible to perform a pointing operation by moving the cursor position in a contactless manner by operating in the air without directly using a stationary desk surface or a contact operation device.
It is possible to provide a position pointing device that enables a natural and efficient pointing operation without releasing the keyboard from the keyboard during efficient pointing in a free posture when using a touchpad or during touch typing.

[Brief description of drawings]

FIG. 1 is a schematic diagram illustrating a basic configuration of the present invention.

FIG. 2 is an explanatory diagram of an example of a light pattern that can be used in the present invention.

FIG. 3 is an explanatory diagram of an example of a light pattern that cannot be used in the present invention.

FIG. 4 is an explanatory diagram of an optical pattern movement detection method according to the present invention.

FIG. 5 is an explanatory diagram of an example of a sensor of a detection method that is simpler and more economical than a method that combines an array sensor and a cross-correlation method.

FIG. 6 is an explanatory diagram of an example of a sensor output of a detection method that is simpler and more economical than a method that combines an array sensor and a cross-correlation method.

FIG. 7 is an explanatory diagram of an example of a signal processing circuit of a detection method that is simpler and more economical than a method that combines an array sensor and a cross-correlation method.

FIG. 8 is an explanatory diagram of a simple optical pattern movement detection method using PSD.

FIG. 9 is an explanatory diagram of a configuration example for projecting a light pattern as a small spot on an opening of a PSD.

FIG. 10 is a schematic diagram illustrating a main part configuration of an embodiment of a position pointing device according to the present invention.

FIG. 11 is an explanatory diagram of an overall view in which the position pointing device of FIG. 10 is applied as a pointing pointer for electronic OHP.

FIG. 12 is a schematic diagram for explaining a main part configuration of another embodiment of the position pointing device according to the present invention.

FIG. 13 is a schematic view for explaining still another embodiment of the position pointing device according to the present invention.

FIG. 14 is an explanatory diagram of a mechanical mouse that is the most widely used pointing device.

[Explanation of symbols]

1 ... Screen, 2 ... Cursor, 3 ...
.Information processing means, 4 ... Output image displaying means, 5 ...
..Cursor control means, 6 ... Light pattern, 7 ...
..Light pattern projection means, 8 ... Light pattern movement detection means, 9 ... Cursor movement position calculation means

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification number Reference number within the agency FI Technical display location G09G 5/08 C 8121-5G (72) Inventor Keiji Shimizu 2274 Hongo, Ebina-shi, Kanagawa Fuji Xerox Incorporated (72) Inventor Noriyuki Nehashi 2274 Hongo, Ebina, Kanagawa Fuji Xerox Co., Ltd. (72) Inventor Kimihiro Wakabayashi 2274 Hongo, Ebina, Kanagawa Fuji Xerox Co., Ltd.

Claims (1)

[Claims] 1. An output image projecting means for projecting an output image of an information processing means including a cursor indicating a display position on a screen on the screen, and a position of the cursor projected on the screen. Cursor control means provided in the information processing means for indicating a desired display position, a light pattern projection means for projecting a light pattern which can be arbitrarily moved by the operator and whose moving direction can be identified, and the light pattern Light pattern movement detecting means for detecting movement of the light pattern within the projection area, cursor movement position calculating means for calculating the cursor movement direction and movement distance based on the detection output of the light pattern movement detecting means, and the cursor. By moving the cursor position on the screen by controlling the cursor control means with the calculation output of the movement position calculation means, A position pointing device which moves the display position of the cursor on the screen by contact.