JP2008041111A - Pointing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a non-mechanical pointing device which is inexpensive, highly reliable, used by a user as is the case with the conventional mouse. <P>SOLUTION: The pointing device 1 is attached to an opening 13 on the bottom and near an opening inside a housing and includes illumination light sources 2, 3 for illuminating the surface shape of an operation surface and an optical motion detection circuit 12 which generates a motion instruction signal indicating a motion to the illuminated surface shape visible via the openings in the directions of first and second axes. The optical motion detection circuit includes a plurality of optical detectors 10, each of which has output, a memory including a reference frame with a digitized output value of an optical detector and a digitized sample frame of the optical detector obtained next to the reference frame and a comparison frame is further correlated to the sample frame to confirm motions of the directions of the first and the second axes. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a pointing device used in a computer and its display device.

Hand-operated pointing devices used in computers and their display devices are almost common. Among the various devices, the most popular is the conventional (mechanical) mouse. Conventional mice typically have a bottom surface with three or more downwardly projecting pads made of a low friction material that lifts the bottom surface a small distance from the operating surface of the cooperating mouse pad. At the center of the bottom of the mouse is a hole through which the lower part of a rubber surfaced steel ball (hereinafter simply referred to as a rubber ball) comes out, and the ball is moved downward by gravity during operation. Pulled toward the top of the mouse pad. The mouse pad is usually a closed cell foam rubber pad covered with a suitable structure.
The low friction pad is slippery on the structure and allows the rubber ball to rotate without slipping when the mouse is moved. Inside the mouse is a roller or wheel that contacts the ball at its equator (a great circle parallel to the bottom of the mouse) and converts rotation into an electrical signal. The outer housing of the mouse is formed to have a “front and back” axis (in the same direction as the user's forearm) and a “left and right” axis perpendicular thereto when covered by the user's hand. The internal wheel in contact with the ball's equator is only responsible for the rotation of the ball caused by the movement component in the longitudinal direction of the mouse and the rotation of the other wheel caused by the movement component in the lateral direction. Arranged to respond. The resulting rotation of the wheel or contact roller generates an electrical signal representative of such a motion component. (For example, F / B representing front and rear and L / R representing left or right) These electrical signals F / B and L / R are coupled to a computer and software is responsive to the signals, The display position of the pointer (cursor) is changed by Δx and Δy according to the movement of the mouse. The user moves the mouse so that the displayed pointer is at a desired location or position as necessary. When the pointer on the screen points to a target object or place, one of the buttons or buttons of the mouse is activated by the finger of the hand operating the mouse. This activation acts as an instruction to take some action, and its action is defined by software in the computer.
US Pat. No. 4,799,055

  Unfortunately, the normal types of mice described above have a number of drawbacks. In particular, the deterioration of the mouse ball or its surface, the deterioration or damage of the surface of the mouse pad, and the deterioration of the rotation performance of the contact roller (for example, (a) due to dirt or lint dirt, (b) wear (C) According to both (a) and (b)). All of these can cause irregular or complete failure when operating the mouse at will. Such episodes may cause the user to complain that the cursor on the screen moves in all other directions, but cannot move the cursor downwards, for example. Therefore, the industry has responded by allowing the mouse ball to be removed for ease of replacement and cleaning of the recess in which the ball fits. It was also the first purpose of introducing the mouse pad to prevent the mouse ball from getting dirty. However, some users are very fed up with the mouse when such a countermeasure is useless. The exchange of mice and mouse pads is an active business.

  The root reason for all such problems is the rather subtle compromise between how a traditional mouse is almost mechanical in construction and operation and how it generates and transmits mechanical forces. It depends heavily on (compromise).

  There have been some previous attempts to use optical methods as an alternative to mechanical methods. It included the use of an optical detector that responds to mouse movement on a specially marked mouse pad and responds to movement of a specially striped mouse ball. U.S. Pat. No. 4,799,055 describes an optical mouse that does not require a pre-marked surface. (With the disclosed two one-pixel wide photosensor linear arrays at right angles in the x and y directions and its state machine motion detection mechanism, the optical mouse is incorporated with the technology of incorporated patents But our view is that the incorporated, offset and correlated array [pixel pattern in the area] technology of the incorporated patent is fairly high performance and hard to break). To date, despite the large number of users dissatisfied with mechanical mice, none of these early optical techniques have been widely accepted as a satisfactory alternative to conventional mechanical mice. Therefore, it would be desirable to have a non-mechanical mouse that can be implemented from a manufacturing perspective, is relatively inexpensive, reliable, and can be used by the user as an operating equivalent essentially of a conventional mouse. . This need can be met by a new type of optical mouse with a familiar “feel” and no undesirable behavior. In addition, the new optical mouse's operation should not be dependent on cooperation with the mouse pad and can be operated on almost any surface, whether in a special situation or not.

  The solution to the problem of replacing a conventional mechanical mouse with its optical counterpart is to link various specific spatial surface shapes of the operating surface under the mouse directly to the array of pixels, just as in human vision. The motion is detected optically by imaging. In general, the operating surface can be almost any flat surface, and in particular, the operating surface need not be a mouse pad. The operation surface under the imaging mechanism is irradiated from the side surface with, for example, an infrared (IR) light emitting diode (LED). Surprisingly, a wide variety of surfaces create a large collection of highlights and shadows when illuminated at the proper angle of incidence. The angle is generally as low as, for example, about 5-20 degrees and is referred to herein as the “grazing” angle of incidence. Paper, wood, formica and painted surfaces can all be used successfully. The only thing that doesn't work well is smooth glass (unless you have a fingerprint). The reason these surfaces can be used successfully is because they have fine textures that in some cases cannot be perceived by human senses alone.

  The IR light reflected from the microtextured surface is focused on an appropriate array of photodetectors (eg 16 × 16 or 24 × 24). The LED can continuously emit a constant or variable amount of illumination servo controlled to maximize performance aspects (eg, the dynamic range of a photodetector cooperating with the operating surface albedo). . Alternatively, the charge accumulation mechanism coupled to the photodetector can be “closed” (by a current shut switch), and the LED switches on and off to provide an average amount of light and control exposure It is done. Turning off the LEDs can save power, which is an important consideration in battery operating environments. The individual photodetector responses are digitized to the appropriate resolution (ie 6 or 8 bits) and stored as a frame at the corresponding location in the memory array. Thus, the mouse of the present invention comprises an “eye”, and the present invention further comprises the eye to “see” movement by making comparisons with successive frames.

  The size of the image projected on the photodetector is preferably a slight enlargement of the original surface shape, for example 2 to 4 times. However, if the photodetector is sufficiently small, it need not be enlarged and is desirable. The size of the photodetectors and their spacing is such that one or several photodetectors are adjacent to one image surface shape and not vice versa. Thus, the pixel size represented by the individual photodetectors corresponds to a spatial area on the operating surface that is substantially smaller than the size of the typical spatial surface shape on the operating surface, which is the cloth covering the mouse pad. It may be an element of a fine surface shape of a fiber strand, a paper or cardboard fiber, a microscopic change in the painted surface, or an embossing on a plastic laminate. The overall size of the array of photodetectors is preferably large enough to receive images of several surface shapes. In this way, when the mouse moves, an image of such a spatial surface shape generates a pixel information conversion pattern. The number of photodetectors in the array and the rate at which the contents are digitized and captured are related to how fast the Magic Eye mouse can be moved and tracked on the operating surface. Tracking is accomplished by comparing a newly captured sample frame with a previously captured reference frame to ascertain the direction and magnitude of motion. One method that can be performed is to shift the entire contents of one of the frames by a distance of one pixel (corresponding to the photodetector) in each of the eight possible directions by successive one-pixel offset trial shifts. (E.g. one side, one side one down, one down, one up, one side up, one side in the other direction, etc.). Up to 8 trials are made, but don't forget that nothing moved, so a 9th trial "null shift" is also needed. After each trial shift, the parts of the frame that overlap each other are subtracted on a pixel basis on a pixel basis, and the resulting differences are summed (preferably after squaring) to determine the similarity within that overlap region. Construct a measure of (correlation). Of course, larger trial shifts are possible (e.g., two laterally one down), but at some point the benefits are compromised by the complexity involved and the frame rate is sufficiently increased with a small trial shift. Only is preferred. The trial shift with the smallest difference (maximum correlation) can be obtained as an indication of motion between the two frames. That is, this provides raw F / B and L / R. Raw movement information is evaluated or accumulated and display pointer movement information (Δx and ΔY) with convenient granularity is provided at an appropriate information exchange rate.

  The actual algorithm described in the incorporated patent (and used by the Magic Eye Mouse) is an improved and enhanced version of the aforementioned algorithm. For example, assume that the photodetector is a 16 × 16 array. By storing the digitized value of the photodetector output appearing at a certain time t0, the reference frame can first be obtained. At a later time t1, a sample frame is acquired and another set of digitized values is stored. We want to correlate a new collection of 9 comparison frames (null, 1 side, 1 side up, etc.) with a reference frame version representing "where was the last". Note that the comparison frame is a temporary shift of the sample frame, and when it is shifted, the comparison frame does not overlap exactly with the reference frame. In other words, one edge or two adjacent edges will not match. The position of a pixel along a non-matching edge does not impair the corresponding correlation (ie its particular shift), but everything else. Also, the positions of those other pixels are real numbers of pixels that make the signal-to-noise ratio very good. For “nearest neighbor” (ie, limited to null, one side, one up / down, combinations thereof), the correlation causes the squared difference of all pixel locations that have spatial correspondence. Nine “correlation values” are generated that can be derived from the sum (ie, pixel locations in one frame that are actually paired with pixel locations in other frames. Unmatched edges are paired as such. do not become).

  A method of shifting and a method of obtaining a correlation value will be briefly described. Shifting is accomplished by addressing the offset into a memory that can output the entire row and column of the array at one time. Dedicated arithmetic circuitry is connected to the memory array containing the reference frame to shift and the memory array containing the sample frame. The formulation of correlation values for a particular trial shift (closest or nearest neighbor collection member) is achieved very quickly. The best mechanical analogy is to imagine a transparent (reference) film of transparent and dark patterns arranged like a checkerboard, except that the correlation values are probably random. Now imagine that a second (sample) film with the same universal pattern overlaps the first film except that it is a negative image (the dark and transparent areas are opposite). The pair is then aligned and illuminated. As the reference film is moved relative to the sample film, the amount of light that passes through this combination varies with the degree to which the images match. The position that allows the least amount of light to pass is the best correlation. If the negative image pattern of the reference film is a square or two shifted from the image of the sample film, the position through which the least amount of light passes will match the shift. Describe the minimum light passing displacement, and for a magic eye mouse, consider that the mouse has moved that much, considering the position with the best correlation. That is what actually happens in an integrated circuit (IC) having a photodetector, memory and arithmetic circuitry configured to implement the image correlation and tracking techniques described herein.

  It is desirable if a given reference frame can be reused with successive sample frames. At the same time, if each new collection of 9 (or 25) correlation values (collection at ti, ti + 1, etc.) resulting from a new image (next sample frame) at the photodetector does not contain sufficient correlation Don't be. In a hand-held mouse, typically several consecutive collections of comparison frames can be obtained from the (16 × 16) reference frame acquired at t0. This can be done by maintaining the most recent motion direction and displacement data (equal to known speed and time interval from previous measurements). This allows for “prediction” how the set of pixels in the reference frame will shift (forever) !, so that in the next sample frame the “nearest neighbor” will be correlated. You can expect to make it. Shifts that deal with this prediction invalidate or remove some of the reference frames, reduce the size of the reference frames, and reduce the statistical quality of the correlation. A new reference frame is acquired when the edge of the shifted and reduced reference frame begins to approach the center of what was the original reference frame. This mode of operation is called “prediction” and is a 5 × 5 extended “nearest nabor” (null, 2 horizontal / 1 up, 1 horizontal / 2 up, 1 horizontal / Upward, two sideways, one sideways, ...) can also be used for comparison frames. The advantage of the prediction is that it speeds up the tracking process by streamlining the internal correlation procedure (eliminating the comparison of two arbitrarily associated 16 × 16 data arrays) and the time allocated to the acquired reference frame. Is to reduce the proportion of

  The magic eye mouse of the present invention can be provided with another button for stopping the generation of a motion signal to the computer in addition to the normal button generally provided by the mouse, without interfering with the positioning of the pointer on the screen. The mouse can be physically rearranged on the operation surface. This may be needed when there is no room for the operator to physically move the mouse but the screen pointer still needs to be moved. This occurs, for example, in a UNIX system that uses a display system known as a “single logical screen” (SLS), where as many as four monitors are each arranged to display a portion of the entire “screen”. there is a possibility. When these four monitors are arranged in a horizontal row, the left-right distance required for a single corresponding maximum mouse movement is much wider than what is normally allowed. For example, the usual strategy an operator takes for extended rightward displacement is to simply lift the mouse on the right side of the operating surface (mouse pad or one corner of a cluttered desk) and lower it to the left Continue to move to the right. What is needed is that during this strategy, the motion indication signal is not subject to false behavior, so that the pointer on the screen behaves as expected. The function of the “Hold” button is either by the proximity sensor below the mouse that determines that the mouse is not touching the operating surface, or by noting that all or most of the image is “dark” This is done automatically (it is actually a bit complicated and will be explained in more detail in the next paragraph). In the absence of the hold function, (a) the field of view tilts when the mouse is lifted, or (b) two differently spaced spatial surface shape frames imaged at significantly different times during removal or replacement, An inconvenient error when obtained when representing a small distance between two frames of the same surface shape can cause a slight tilt of the image during removal and replacement of the mouse. A convenient location for the actual hold button is near the bottom along the side of the mouse where the thumb and opposite ring finger grab and lift the mouse. In addition, the natural increase in the grasping force used to lift the mouse links the hold function. The hold function may include an optional short delay based on release of the hold button, detection of proper approach, or return of a valid digitized value. During that delay, there is time for the lighting control servo loop or internal automatic gain controller to settle and a new reference frame is acquired before motion detection resumes.

  Second, this action of the pixels in the image “darkens”. Of course, the infrared light from the illumination LED does not reach the photodetector with the same quality as before, because the reflective surface is too far away or simply not in the field of view. However, if the magic eye mouse is turned over, or as a result, the lower side is exposed to a strong light environment, the output of the photodetector may be at any level. What is important is that the level is constant or nearly constant. The main reason for the level being constant is that the image is no longer focused and all image surface shapes are blurred and spread over the entire collection of photodetectors. Therefore, the photodetection device becomes constant at a certain average level. This is in sharp contrast to the case where there is a focused image. When focused, the correlation between frames (recall one side, one side down, etc.) indicates a distinct phenomenon.

  Assume that the spatial surface shape is accurately mapped and tracked on the photodetector by the lens system and that the mouse has moved in the exact amount and direction required for the surface shape to travel from detector to detector. Also, for simplicity, assume that there is only one surface shape and that the image is the size of a photodetector. Thus, all the photodetectors except one are at almost the same level, and one detector that is not at that level is at a substantially different level depending on the surface shape. Under these very idealized conditions, it is clear that the correlation behaves very well, with eight “large” differences and one small difference in the system (otherwise a fairly flat surface sink). (Hall) uses 9 trials in the nearest neighbor (and recall that there may be no movement) [Note: wise readers will have a “big” difference in this relatively unnatural example However, one will notice that it actually corresponds to or is due to only one pixel and is probably not worthy of calling it “large” and cancels the initial shift film analogy. The light that passes through the film in this example is only for one pixel in the surface shape. A more normal image with a significantly different collection of pixels will increase the difference from a truly “big” difference. ]

  Such highly idealized conditions are not the usual case. It means that the image of the spatial surface shape that follows is both larger and smaller than the size of the photodetector, and the movement of the mouse is continuous, and such an image can be transferred to multiple detectors at once. It is standard to follow a path that allows entry. Some detectors receive only partial images, for example, and some detectors perform analog addition of both bright and dark portions. As a result, the sink hole may expand (in terms of the number of photodetectors associated with it) and the sink hole depth may be correspondingly reduced. This situation can be shown by imagining that a heavy ball rolls along a tight but very stretchable membrane. This film is associated with a discrete integer Cartesian coordinate system. How much will the membrane expand at any integer coordinate position when the ball rolls? First imagine that the ball is very small in diameter but very heavy, and then imagine that the ball is large in diameter but the same weight. The analogy may be inaccurate but serves to illustrate the concept of the “sink hole” described above. A common case is that a substantially flat surface with a well-defined sink hole becomes a large recess or bowl shape.

  We refer to a surface generated or described with various correlation values as a “correlation surface” and is most interested in its surface shape at various times.

  In the present specification, these will be described in order to achieve two purposes. First, the movement of the shape of the recess in the correlation plane when the magic eye mouse moves allows interpolation to a finer granularity than just the size / interval of the photodetector. This is illustrated by the findings that the magic eye mouse (pointing device) of the present invention can do and leave at that time. Full details of the interpolation are described in the incorporated patent. A more detailed discussion of interpolation is considered unnecessary. Second, this is the real reason for the discussion in the previous section, but when the magic eye mouse is lifted, there will be no correlation surface depression, and the correlation will be approximately equal (ie, a “flat” correlation surface). It is an observation of the replacement. When this happens, you can quite reliably explain that the magic eye mouse is floating in the air, and then automatically activate the hold function until the appropriate recess ("bowl") reappears. be able to.

  Another way to activate or initiate the hold function is that the magic eye mouse is moved faster than a certain threshold speed (and possibly more than the available physical space in which the mouse is moving). There is also a sudden retrace movement in the strategy to move the screen pointer far away). Beyond the speed threshold, the motion indication signal, usually associated with movement, is suppressed until the speed is slower than the appropriate level.

  The present invention provides an optical mouse that is inexpensive and highly reliable and can be used by a user in the same manner as a conventional mouse.

  Referring now to FIG. 1, a simplified cross-sectional view of a prior art imaging / navigation device 1 of the type described in the incorporated patent is shown. The LED 2 may be an infrared LED, and emits light irradiated by the lens 3 (which may be an integral part of the LED package) from the hole 13 of the bottom surface 6 to the region 4 which is a part of the operation surface 5. . The average incident angle is preferably in the range of 5 to 20 degrees. Although not shown for the sake of clarity, the hole 13 is transparent to the light from the LED 2 and has a window to prevent dirt and dust from entering the inside of the magic eye mouse. Also good. The operation surface 5 may be a special object such as a mouse pad or, more generally, the surface of almost anything except smooth glass. Examples of suitable materials include, but are not limited to, paper, cloth, laminated plastic tops, painted surfaces, ground glass (smooth side down), desk pads, real wood, fake wood, etc. . In general, any microtextured surface having a surface shape with a size in the range of 5 to 500 microns.

  Illumination of the fine textured surface is most effective when performed from the side so that highlight and shadow patterns generated by irregularities in the height of the surface are emphasized. Incident angles suitable for illumination range from about 5 to 20 degrees. It may be a very smooth or flat surface (eg, polished and polished) whose reflectivity simply changes with compositional changes (on a microscale). In such a case (and assuming it can be guaranteed), the illumination incident angle may approach 90 ° because there is no reason to create a shadow. However, a surface with such a smooth but fine detail is not what we normally think of when we say “any surface”, and it is more preferable and finer when it has a grazing angle of incident illumination. A magic eye mouse that is intended to be used with "any surface" that is textured to the best works.

  The image of the illuminated area 4 is projected through the optical window 9 of the integrated circuit package part 8a onto the array 10 of photodetectors. This is done by the lens 7. Moreover, the package part 8a can also make them unnecessary by combining the independent window 9 and the lens 7 into one same element. The photodetector may comprise, for example, a square array of 12-24 detectors on one side, each detector being a phototransistor with a photoelectric area of 45 × 45 microns and a center-to-center spacing of 60 microns. is there. The phototransistor then charges a capacitor whose voltage is digitized and stored in memory. Array 10 is created on a portion of integrated circuit die 12 that is affixed to package portion 8b with adhesive 11. Details of how the integrated circuit is held in place, the shape or configuration of the lens, and how to attach the lens are not shown, but it is clear that they can be done in a conventional manner. In addition, it is obvious that the approximate level of illuminance in the region 4 can be controlled by adjusting the intensity of light emitted from the LED 2 while paying attention to the output level of the photodetector. This may be continuous control, pulse width modulation, or a combination of both.

  The reader again recalls that the details of the motion detection operation are fully described in the incorporated patent (also briefly described in the summary of the invention). Therefore, it is not necessary to repeat the description here.

  Reference is now made to FIG. 2, which is a bottom view of a mouse 14 constructed in accordance with the present invention. Simply put, the bottom view of this magic eye mouse 14 is very similar to a particular conventional mouse by Hewlett-Packard Company, namely C1413A. The major difference is that there is a protective lens or window 16 that is transparent to infrared light where the ball should be. This is the transparent window in which the hole 13 described in the description of FIG. 1 is omitted. There is also no normal rotating ring that acts as a removable holder that allows access to the ball for cleaning or replacement. What is shown in the figure is a connection cable 17 having a lower surface 15 (corresponding to 6 in FIG. 1) of the mouse 14, a low friction sliding portion 19, and a deformation relaxation portion 18. Of course, the magic eye mouse 14 of the present invention may be a cordless mouse and may include an optical or wireless communication link with a computer.

  Referring now to FIG. 3, a side perspective view of a mouse 14 constructed in accordance with one aspect of the present invention is shown. This aspect of the invention is a hold function. The hold function is a mode of operation of the magic eye mouse that stops generating motion information or signals to the computer when it is determined that the mouse is not close enough to the operation surface following the spatial surface shape. is there. This allows the magic eye mouse to be picked up, moved and moved back down, or so-called “swiped” across the operating surface.

  In particular, the magic eye mouse 14 of FIG. 3 has at least one hold located near the bottom surface 15 of the side skirt 20 so that it is under the right thumb or left ring finger depending on which hand the operator uses. -It has a button 24. There may be another symmetrically arranged button that contacts the left thumb or right ring finger on the other side (not shown).

  The mouse 14 has a surface 21 that fits comfortably in the palm of the hand and first and second “normal” mouse buttons 22 and 23 that are operated with the toes and middle fingers, as is conventional. These work as usual.

  Button 24 is activated by a natural increase in the grip force required to pick up mouse 14 during a swipe. When one or both of these buttons are pressed, the hold function is activated. While the hold is maintained, transmission of the motion signal to the computer is interrupted. When the hold ends (button is released), a new motion signal is sent to the computer after obtaining a new reference frame. This has the advantage of allowing a swipe and allowing the user to force start the hold function.

  In addition, the hold function can be automatically activated by the operation of an independent proximity sensor on the bottom surface of the mouse. This is shown in FIG. 4 where the shouldered opening 26 in the base 6 receives a shouldered plunger 25 that is held down by the lever arm of the switch 28 above. The switch 28 is activated by the movement of the plunger 25 when the plunger greatly moves in the direction of the arrow 27, and the hold function is activated. The exact function of the independent proximity sensor is a matter of choice, but it can be as simple as the microswitch 28 that is actuated by the weight of the mouse by means of the plunger 25, or other non-mechanical methods.

  Yet another way to automatically activate and deactivate the hold function is to examine the nature of the digitized data of the photodetector array 10. If the output of the photodetector becomes sufficiently constant, it can be assumed that an image having a change is not projected on the photodetector array 10. This uniformity appears by creating a flat or nearly flat correlation surface. Rather than detecting uniform levels separately (usually using non-existing hardware), it is preferable to examine the shape of the correlation surface (the surface required for other reasons). The most likely cause of the flat correlation surface is that the mouse was taken up. This mode of operation requires that the depth of field be fairly shallow so that there is no undue delay in activating the hold function. Such a delay can create small artifacts during the movement of the screen pointer. This may include slight unintentional screen pointer movement due to mouse tilt as the mouse is lifted or moved. As long as the hold function is activated (manually or automatically), as long as a new reference frame is forcibly acquired before the generation of motion signals is resumed, how many appear to be correct small motions in the wrong direction There should be no danger of fake instructions from the combination of such new and old data. However, in the case of mere uniform level detection (eg of a sample frame), it is difficult to avoid optical effects (bright light source reflections) that disrupt the algorithm during still pending motion. In some cases. It should be understood that the shape of the correlation surface is a more reliable indicator. As previously mentioned, recall that so-called steering of the screen pointer is an incrementally driven servo operation performed by a human. If the screen pointer isn't there yet, just keep moving the mouse as needed. Small perturbations during the swipe are not fatal and may not be very noticeable depending on the particular application being run.

  Referring now to FIG. 5, there is shown a flow chart 29 that illustrates aspects of magic eye mouse operation including hold / prediction characteristics. Suppose that there is a start condition or position 30 from which step 31 “get reference frame” is reached. This refers to illuminating LED 2 and storing a digitized collection of photodetector values in an array of memory (not shown). The next step 32 is “Obtain a sample frame”. This refers to the same action except that the data is stored in a separate memory array and can reflect the movement of the mouse relative to where it was when step 31 was executed. In step 33 "Calculate Correlation Value", 9 (or 25) pieces with some kind of heavy-duty dedicated arithmetic hardware supported by an automatic address translation mechanism and a very wide path from the memory array. The correlation value of is quickly calculated. In step 34 “Is there a suitable dent in the correlation surface”, the nature of the correlation surface indicated by the collection of correlation values calculated in step 33 is examined. We want to know if the correlation surface is shaped like a bowl, and if so, we want to know how much water collects.

  If the correlation surface is of the appropriate bowl shape, pass 36 leads to optional step 37 "Hold button pressed", which will be described in detail in the next section. Otherwise, if a flat correlation surface or “unsuitable bowl shape”, proceed along path 35 to optional step 42 “delay”. There are several possible causes for the exit from the qualifier 34. For example, the speed is extremely high, the surface shape suddenly disappears on the operation surface, or the mouse floats in the air. In the absence of an explicit HOLD button, the present invention achieves proper magic eye mouse behavior by suppressing motion signals to the computer during the air movement of the swipe action through the exit path 35.

  If the magic eye mouse has a HOLD button, there is an optional qualifier 37 where the status of the HOLD button 24 (whether it has been pressed) is determined. The case where the button is pressed is treated in the qualifier 34 as if it were an inappropriate bowl. That is, the path 38 leading to the optional step 42 is selected.

  Optional step 42 provides an effective delay in several ways. First, if there is a swipe while it is in progress, it takes some time and the battery power can be saved by not imaging during that time. It is also assumed that the action of the delay is a little more complicated than stopping the finger movement on the flowchart. Step 31 “Obtain reference frame” is affected by the delay in step 42 and assumes that the lighting level control operation is initiated by the delay. Thereby, the time of readjustment of illumination level etc. is made. Regardless of whether there is a “delay” in optional step 42, pass 43 returns to step 31 and another motion detection cycle begins.

  To resume, pass 39 leads to step 40 "Predict reference frame shift". As described above, in order to obtain the displacement required for prediction, it is generally not necessary to acquire and maintain information on the actual speed and time interval in the x and Y directions. You can imagine the measurement environment that is needed, but this is not one of them. Instead, the prediction lag can be obtained as the amount of movement corresponding to the correlation in the previous step 34.

  The next step 44 is “output Δ × and ΔY”. Now record how much the mouse has moved since the last measurement cycle. The amount of deviation necessary to obtain the correlation is the desired amount. These values can be determined by noting which comparison frames are actually correlated (assuming no interpolation). These “raw” Δx and ΔY motion values can be accumulated into an execution value that is sent to the computer at a rate slower than the rate at which the raw value of step 44 is generated.

  The qualifier 45 asks whether a "new reference frame is needed". If the answer is yes, path 46 leads to step 48 "store the current sample frame in the reference frame". (This sample frame reuse makes it easy to see that you don't have to maintain the actual speed and time interval of the prediction process. To get another new reference frame, a lot of things get complicated. (You will probably use the D = RT or distance equation for prediction.)

  A new reference frame is needed when there is a sufficient shift in the reference frame due to prediction and the reference frame does not overlap sufficiently with the comparison frame to obtain a positive correlation. Locations in the range of 3-5 deviations (do not retrace themselves) relate to 16x16 reference frame limitations.

  If the answer of the qualifier 45 is NO and the reference frame does not need to be exchanged, the process proceeds to the same step 49 by the path 47 together with the path from the step 48. Step 49 "Move reference frame" actually moves the value permanently in the memory array representing the reference frame. This movement is due to the predicted amount and the moved data is lost. After shifting the reference frame, the path 50 returns to step 32 “Get Sample Frame” and the next measurement cycle begins.

  Referring now to FIG. 6, there is shown a simplified flowchart portion 50 showing that step 44 of flowchart 29 of FIG. 5 has been replaced with steps 51-55. The purpose of doing this is similar to the various hold operation schemes already described and can be used in conjunction with or instead of that scheme. The general concept of the modification depicted in FIG. 6 is to skip step 55A and send no update information, or send Δx and ΔY zero even if not true (by optional step 55B). Is to deceive the computer. This is done whenever step 52 determines that the speed of the mouse movement has exceeded, for example, 3 to 6 inches per second (approximately 7.6 to 15.2 cm). For a given magic eye mouse, such a limitation can be as simple as a fixed number of pixel displacements within several measurement cycles, assuming that the measurement cycle is faster than normal mouse movements. It is expressed in This concept is based on the fact that normal mouse movements are probably the closest neighbor reference frame (5 × 5 neighbors) that is new to every measurement cycle between many (eg, 10-25) consecutive measurement cycles. Need not be moved as much as possible for the combined operation. In such a case, the magic eye mouse will operate on the hairy edge in hold mode with a NO answer of qualifier 34 and pass 35. According to the assumption, the correlation is lost at higher speeds), ie it is generally very rare to obtain a new reference frame. Of course, it can happen that the speed of the mouse is actually fast and the path 35 is used. That is natural. However, it may not be appropriate to use the technique of FIG. 6 if the speed of the measurement cycle is not very fast relative to the normally expected mouse movement.

  Step 51 represents after and before the actual communication of the values Δx and ΔY to the computer in the previous step 44. An example that requires attention to this difference is the internal accumulation of motion that has not yet been sent to the computer because the rate of the magic eye mouse's internal motion measurement cycle is higher than the rate of information exchange with the computer. In some systems, this accumulated information is used in the mouse rather than strictly keeping the computer informed. In such a case, with respect to the qualifier 52, the pass 53 (and bypassed step 55A) that needs to be performed does not inform the computer that there has been movement and deceives the computer without losing mouse functionality. It is desirable.

  If it is allowed to continue such accumulation during a quick retrace to mimic lifting the mouse, the computer can eventually win when the speed drops to normal speed, and the accumulation Is finally sent and the screen cursor can be snapped to the right place depending on how the entire system works. In such a case, another set of accumulations must be maintained along with those of the computer remaining in bypass step 55A.

  Of course, there may be no internal use by the mouse of the accumulated Δx and ΔY other than being sent to the computer. In such a case, nothing needs to be done other than leaving the accumulation in detour step 55A. Also, there may be no accumulation causing such anxiety in the mouse, for example, such accumulation is performed by software in the computer.

  Finally, refer to FIG. This figure is a plot 56 of a neighborhood (5 × 5) correlation surface 57 with appropriate recesses. Two horizontal axes 58 and 59 represent the X and Y axes of mouse movement. Units shown along the axis are pixels. In the planes of axes 58 and 59, smoothed and interpolated contour lines 60 are drawn to further illustrate the shape of the correlation surface directly above. The measure of correlation on the vertical axis 61 is essentially expressed in arbitrary units.

Embodiments of the present invention are summarized below.
1. A handheld pointing device (1, 14, 15) for a computer system, etc.,
A housing having a flat bottom surface (6) that moves relative to an operating surface (5) having an imageable surface shape;
The housing also has a top surface (21) configured to receive a human hand;
The housing also has a skirt (20) connected to the periphery of the flat bottom;
The housing has a first axis extending from the heel portion of the upper hand rest substantially toward the upper middle finger rest and a second axis perpendicular to the first axis, both axes being on the bottom surface Parallel to the bottom opening (13, 16),
An illumination light source (2, 3) that is mounted near the inner opening of the housing and illuminates the imageable surface shape of the operating surface;
A motion indication signal is mounted near the inner opening of the housing and indicates movement in the direction of the first axis and the second axis relative to the illuminated imageable surface shape visible through the opening. An optical motion detection circuit (12),
A plurality of optical detectors (10) each having an output;
A memory including a reference frame (31) of the digitized output value of the photodetector and a digitized sample frame (32) of the photodetector obtained next to the reference frame; A plurality of comparison frames of the moving version of the reference frame (49) are correlated with the sample frame to confirm movement in the direction of the first axis and the second axis (44) The handheld pointing device (1 , 14, 15).
2. The device of claim 1, wherein the existing reference frame is moved (40) by an amount corresponding to a previous correlation with the comparison frame.
3. The device of claim 1, wherein an existing sample frame is periodically acquired as a new reference frame (48).

4). A handheld pointing device (1, 14, 15) such as a computer system,
A housing having a flat bottom surface (16) that moves relative to an operating surface (5) having an imageable surface shape;
The housing also has a top surface (21) configured to receive a human hand;
The housing also has a skirt (20) connected to the periphery of the flat bottom;
The housing has a first axis extending from the heel portion of the upper hand rest substantially toward the upper middle finger rest and a second axis perpendicular to the first axis, both axes being on the bottom surface Parallel to the bottom opening (13, 16),
An illumination light source (2, 3) that is mounted near the inner opening of the housing and illuminates the imageable surface shape of the operating surface;
A motion indication signal is mounted near the inner opening of the housing and indicates movement in the direction of the first axis and the second axis relative to the illuminated imageable surface shape visible through the opening. An optical motion detection circuit (12),
Proximity detection that detects that the lower surface is more than the selected distance from the operation surface and is coupled to an optical motion detection circuit to suppress the generation of motion instruction signals when the lower surface is further than the selected distance from the operation surface Vessel (24, 34/27, 28),
Including device.
5. A device according to claim 4, wherein the proximity detector comprises at least one switch (24) located at a position below the right thumb or left ring finger of the hand grabbing the pointing device on the skirt.
6). 5. A device according to claim 4, wherein the proximity detector comprises at least one switch (24) arranged at a position below the left thumb or right ring finger of the hand grabbing the pointing device on the skirt.
7). The optical motion detection circuit includes a plurality of photodetectors (10) each having an output, and a memory including a digitized output value of the photodetector and a proximity detector are converted into digitized values in the memory. The device of claim 4 comprising a combined arithmetic comparison circuit.
8). A device according to claim 4, wherein the proximity detector includes a pressure-actuated switch (27, 28) disposed near the lower surface.
9. An optical motion detection circuit includes a plurality of photodetectors (10) each having an output, a digitized photodetector output value reference frame (31), and a digitized result obtained next to the reference frame A new reference frame and a next comparison frame at the end of suppression of motion indication signal generation and before resumption of motion indication signal generation (35). 38) The device according to 4 above, obtained in 38).
10. 10. The device of claim 9, wherein the new reference frame is obtained after a selected amount of delay (42) beyond the point when the proximity detector no longer detects that the bottom surface is a selected distance from the operating surface. .

11. A pointing device (1, 14, 15) such as a computer system,
A housing having a flat bottom surface (6) that moves relative to an operating surface (5) having an imageable surface shape;
The housing also has a top surface (21) configured to receive a human hand;
The housing also has a skirt (20) connected to the periphery of the flat bottom;
The housing has a first axis extending from the heel portion of the upper hand rest substantially in the direction of the upper middle finger rest and a second axis perpendicular to the first axis, both axes being on the bottom surface Parallel to
Bottom openings (13, 16);
An illumination light source (2, 3) that is mounted near the inner opening of the housing and illuminates the imageable surface shape of the operating surface;
A motion indication signal is mounted near the inner opening of the housing and indicates movement in the direction of the first axis and the second axis relative to the illuminated imageable surface shape visible through the opening. An optical motion detection circuit (12);
A detector coupled to the motion detection circuit to detect that the movement of the pointing device within the time period exceeds a selected limit (52) and to suppress the generation of a motion indication signal in response thereto (53) When,
Including device.
12 An optical motion detection circuit includes a plurality of photodetectors (10) each having an output, a digitized photodetector output value reference frame (31), and a digitized result obtained next to the reference frame And a memory containing a comparison frame (32) of the photodetector output value, so that the detector no longer detects that the new reference frame and the next comparison frame have exceeded the limit of movement of the pointing device. 12. The device according to claim 11, which is obtained after and before resuming suppression of generation of the motion instruction signal.

1 is a simplified pictorial cut side view of a prior art imaging / navigation device. FIG. It is a bottom view of a mouse constructed according to the present invention. 1 is a side perspective view of a mouse constructed in accordance with one aspect of the present invention. FIG. FIG. 4 is a simplified cut-away side view at the base of the mouse of FIGS. 2 and 3 and used to automatically activate the hold function. 6 is a simplified flowchart illustrating aspects of magic eye mouse internal operation associated with the operation of the hold function when used with a function called prediction. FIG. 6 is a simplified partial diagram illustrating a speed detection method that modifies the flowchart of FIG. 5 and activates the hold function. FIG. 6 is a perspective view of a plotted correlation surface with appropriate recesses.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,14,15 Pointing device 2,3 Illumination light source 5 Operation surface 6 Bottom surface 10 Photo detector 12 Detection circuit 13, 16 Aperture 24, 34/27, 28 Proximity detector 31 Reference frame 32 Sample frame

Claims (4)

In a handheld pointing device for a computer system,
A housing having a bottom surface, wherein an opening passes through the bottom surface, the housing is movable with respect to the operation surface, and the housing has a top surface formed to receive a human hand; The bottom surface and the top surface are opposed to each other and are isolated to form the interior of the housing, and the housing extends in a direction intersecting with each other along the operation surface. A housing consisting of defining an axis;
An illumination source mounted within the housing and illuminates the operating surface through the opening, the illumination source being located opposite the opening and in a range of approximately 5 microns to 500 microns. Illuminating a portion of the operating surface having an irregularly-shaped surface forming a fine texture having a size shape from a single point at an incident angle in the range of approximately 5 ° to 20 ° Thus, the surface that extends outward from the operation surface and has an irregular height to block the illumination is highlighted, and the surface that extends toward the inside of the operation surface has an irregular height. A shadow is generated on the illumination, and the illumination that causes the shadow is blocked by an adjacent surface with irregularly illuminated height, and the highlight and the shadow are Translational opening An illumination source consisting of forming a pattern of images that change with motion;
A plurality of optical detectors mounted in the housing and arranged two-dimensionally, and an optical motion detection circuit including a memory for storing output information from the plurality of optical detectors;
The plurality of photodetectors detect an optical image by light from the operation surface irradiated by the illumination source through the opening,
The memory sequentially stores information of the optical image detected by the plurality of photodetectors at different times as a reference frame and a sample frame,
The optical movement detection circuit generates a movement signal indicating movement of the housing in a plane formed by the first axis and the second axis on the operation surface;
The reference frame is stored at a location of a reference memory array corresponding to each of the plurality of photodetectors, and the sample frame is located at a location of a sample memory array corresponding to each of the plurality of photodetectors. The other reference frame for each of a plurality of comparison frames stored and defined as frames each of which is a trial movement of one of the reference frame or sample frame in a predetermined plurality of directions, respectively, by a predetermined distance, respectively Or determining a correlation with a sample frame, wherein the correlation is calculated based on a value at a memory array location corresponding to an overlap between the comparison frame and the other reference frame or sample frame; The motion signal is a signal indicating the direction and magnitude of the trial movement when the correlation is maximized. It is generated as,
The optical motion detection circuit automatically blocks generation of motion signals along the first axis and the second axis in response to the correlation obtained for each of the plurality of comparison frames. Automatic holding means, whereby the correlations calculated based on the values at the position of the memory array corresponding to the overlapping portion are substantially equal to each other, so that the housing is separated from the operating surface. The movement signal is configured to prevent false behavior of the movement signal; otherwise, the movement is performed assuming that the bottom surface of the housing is disposed in close proximity to the imageable portion of the operation surface. A hand-held pointing device that automatically initiates detection.   The first axis is defined as an axis extending from a heel portion of the upper surface hand rest of the housing toward the upper middle finger base, and the second axis is perpendicular to the first axis. The hand-held pointing device according to claim 1, characterized in that it is defined as an extending axis.   The handheld pointing device according to claim 1, wherein the illumination light from the illumination source is configured to pass through the opening and partially irradiate the operation surface within the opening. .   The hand-held pointing device according to claim 1, wherein the optical image by the light from the operation surface is an image depending on a spatial surface shape of the operation surface.

Images