New! View global litigation for patent families

WO2000070551A1 - Stylus pen for writing on the monitor - Google Patents

Stylus pen for writing on the monitor

Info

Publication number
WO2000070551A1
WO2000070551A1 PCT/US2000/012995 US0012995W WO2000070551A1 WO 2000070551 A1 WO2000070551 A1 WO 2000070551A1 US 0012995 W US0012995 W US 0012995W WO 2000070551 A1 WO2000070551 A1 WO 2000070551A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
stylus
pen
location
screen
display
Prior art date
Application number
PCT/US2000/012995
Other languages
French (fr)
Inventor
Keith A. Vanness
George K. Vanness
William Marshall
Original Assignee
Cbd Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G06F3/0386Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry for light pen
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03542Light pens for emitting or receiving light

Abstract

A system, method, and apparatus for enabling a user to draw high resolution graphical images onto a monitor or display screen are presented. Graphical location indicia corresponding to display screen locations are displayed on the monitor or display device. When the tip of a stylus pen is placed onto the monitor or display device, a location indicator displayed at display location thereunder is optically detected by the stylus pen. The stylus pen then transmits corresponding electronic signals to a subsystem which determines a pixel location corresponding to detected location indicator. A tracking algorithm is also presented wherein vector indicia are projected in a localized region surrounding the location of the stylus pen. The tracking algorithm permits fast tracking of the stylus pen at rates exceeding the refresh speed, as well as minimizing the obstruction to the graphical images displayed on the monitor or display device.

Description

STYLUS PEN FOR WRITING ON THE MONITOR

BACKGROUND OF THE PRESENT INVENTION

Field of the Invention

The present invention relates generally to computer graphics, and more particularly to inputting high resolution images into a computer system by moving a stylus pen over a display screen.

Description of Related Art

There has been a need for accurate computer pens, known as light pens or stylus pens, which would allow users to input high resolution images, such as handwriting, into computer systems. At present, slow, low resolution computer pens are available. Ideally, a stylus pen used in connection with a graphical user interface should make operating a computer system more intuitive and user- friendly. The goal of prior art stylus pens was to allow a user to input a graphical image into a graphical user interface by placing the stylus pen over the display device and drawing the graphical image, similar to using a writing pen, but the prior art stylus pens could not keep pace with a user's movements were inaccurate, and provided poor graphic resolution.

Stylus pens in general have a configuration similar to that of writing pens. They are used to effectively write on the computer monitor screen. For decades, engineers have tried to develop a display level input device, i.e., a location/touch sensitive display screen. The key factor concentrated on in prior art has been how to get the display screen to register the stylus location rather than have the stylus determine its location on the screen. A brief analysis of existing technology demonstrates that some success has been found by switching the direction of the stylus location determination. Specifically, the stylus pen can be used to determine its location on the display surface, rather than trying to get the display surface to register the location of the stylus pen. To this extent, prior stylus pens rely on detecting the passage of the display monitor's electron beam as the display is refreshed. The timing of the electron beam detection is used to determine the stylus pen's display location.

Prior stylus pens operate using a variety of techniques to produce a graphical effect on the display screen. Such techniques include or require (1) a specially designed display screen, and/or (2) the tracking of the vertical synchronization pulse, to name a few.

U.S. Patent 3,543,240 issued to Miller, et. al, discloses a stylus pen operating with a remote display which includes a set of counters. The counters are reset by computer instruction and count certain events. When the stylus pen detects a passing electron beam, an electronic pulse is produced by the stylus pen's circuitry causing the counters to stop. The values of the counters can then be used to determine the pixel location of the stylus pen. This technique has proven over time to be slow and inaccurate.

U.S. Patent 3,949,391 issued to Benjamin, discloses a plasma panel stylus pen system, wherein the synchronization electron beam is first coarsely detected when the stylus pen touches the screen and thereafter more finely detected. The foregoing, however, requires a specialized display device having circuitry which only scans the area about the coarsely detected region to determine the location of the stylus pen.

U.S. Patent 5,187,467 issued to Myers, discloses a stylus pen system capable of detecting horizontal and vertical synchronization signals. Here counters are utilized to determine where the stylus pen's location is based on the detection of a synchronization pulse. U.S. Patent 3,337,860 issued to O'Hara, U.S. Patent 3,413,540 issued to Haring, U.S. Patent 5,227,622 issued to Suzuki, U.S. Patent 4,620,107 issued to Frame, and U.S. Patent 4,675,665 issued to Halliwell each tend to disclose systems in which the location of the stylus pen is determined by detection of the passing of and synchronization with the scanning electron beam as it moves across the monitor's screen.

Each of the above stylus pen systems are disadvantageous because of reliance on detection of the passing of and synchronization with the monitor's scanning electron beam. This makes the system inherently slow and inaccurate due to the difficulty in catching and synchronization with a single pulse of the passing electron beam. Furthermore, the prior art stylus pen systems have a wide field of view that encompasses several horizontal lines of the electron beam raster. Therefore, the detected pulse may actually be a series of discontinuous pulses or may have an inaccuracy of two to three raster lines. Additionally, the size of the field of view varies depending on the type of phosphor used within the display screen, the distance between the pen and the phosphor, and the intensity of the electron beam's light. Furthermore, these prior art systems generally do not work with liquid crystal displays (LCDs) or plasma displays common to laptop computer systems. Reliance on the passing electron beam limits the pixel location sampling rate of the stylus pen to the screen refresh rate. Additionally, detection of the passing electron beam is more suited for monochrome CRT displays as opposed to the present day more common, color displays .

Therefore, there is a need for a method and system which determines the location of a stylus pen independently from the electron beam and which does not require special circuitry built into the display screen's surface. There is also a need to provide a system wherein a conventional display device and a general purpose computer system can be easily adapted for use with the stylus pen. There is also a need to efficiently track the movement of the stylus pen to produce a high resolution mechanism for producing graphics on a display screen via a stylus pen.

SUMMARY OF THE INVENTION

An embodiment of the present invention provides a stylus pen that can be used to effectively produce graphics on a computer's display when the stylus pen is touched to the display screen. Such graphics are produced via the stylus pen reading a "hidden" or inlayed pixel pattern embedded in the display screen's picture or graphics.

An embodiment of the present invention determines the location of a stylus pen independently of a monitor's scanning electron beam.

An embodiment of the present invention provides a system wherein a conventional display device and a general purpose computer system can be easily adapted for use with the stylus pen.

An embodiment of the present invention efficiently tracks movement of the stylus pen at a rate that can keep pace with a user's normal hand movements.

An embodiment of the present invention displays location indicia or an inlayed pixel pattern on the display device's screen. The location indicia are readable by the stylus pen. In one embodiment, the location indicia is provided by a subsystem, or circuitry which can be connected between the display device and a general purpose computer system or built into the general purpose computer. The subsystem receives graphic images for the display device and may interleave the graphic images with the location indicia. The graphic images, interleaved with the location indicia are then displayed on the display device. When the stylus pen is directly over an area on the display device, the stylus pen reads the interleaved pixel location indicia, and provides the location indicia to the subsystem. The subsystem then determines the location and/or movement of the stylus pen. The determined location of the stylus pen is then provided to the general purpose computer system, via a mouse port or other suitable mechanism or data port.

In an alternate embodiment, a computer system with the stylus pen is disclosed without a subsystem externally connected between the stylus pen and the general purpose computer. The computer system transmits graphic images interleaved with the location indicia to the display device. The stylus pen can read the location indicia from the screen and transmit the read location indicia from the screen to the computer system. The computer system then calculates the location and/or movement of the stylus pen.

Additionally, a method for locating and tracking the movement of the stylus pen is presented. The status of the stylus pen is continuously polled. When the status of the stylus pen changes from disengaged to engaged (i.e., the stylus pen is placed m contact with the display device's screen) , the location indicia is displayed m an array format of the display device. The stylus pen can optically read the location indicia. The read pixel location indicia is used to determine the location of the stylus pen. After determining the location of the stylus pen, a direction box including localized vector indicia about a smaller localized area surrounding the determined stylus pen contact location is displayed on the display device. As the stylus pen is moved across the screen, the localized vector indicia is read, movement vectors describing the movement of the stylus pen are determined, and the direction box is repositioned to follow the movement of the stylus pen. Pen-like graphics can be placed on the screen to provide high resolution "writing" on the screen as the stylus moves across the screen.

BRIEF DESCRIPTION OF THE DRAWINGS

Various objects and advantages of this invention will become apparent and more readily appreciated from the following description of the presently preferred exemplary embodiments, taken in conjunction with the accompanying drawings, of which:

FIGURE 1 is a block diagram of a system configured in accordance with an exemplary embodiment of the present invention;

FIGURE 2 is a block diagram of an exemplary stylus pen in accordance with the present invention;

FIGURE 3 is a diagram of a monitor displaying pixel location indicia in accordance with the present invention;

FIGURE 4 is a diagram of a monitor displaying localized vector indicia about a location of a stylus pen in accordance with the present invention; FIGURE 5 is a block diagram of an exemplary subsystem in accordance the principals of the present invention;

FIGURE 6 is a flow diagram describing the operation of the subsystem in accordance with the present invention; and

FIGURE 7 is a block diagram of a computer system in accordance with another exemplary embodiment of the present invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE PRESENT INVENTION

An exemplary stylus pen m accordance with the present invention implements a digital solution for whiteboard or freehand drawing technology which allows the stylus pen to be used for a wide variety of uses. The stylus pen can be used to allow freehand input directly into graphical operating environments and applications during business conferences. Graphic artists can use an exemplary stylus pen apparatus m a freehand manner directly with drawing applications. Persons with carpal tunnel syndrome, or who are otherwise unable to type, may use the exemplary stylus pen instead of a keyboard.

By designing the exemplary stylus pen to operate as a peripheral, the stylus pen can be flexibly used with general purpose computer systems. The exemplary peripheral design permits the stylus pen to be used with a wide variety of commercially available monitors. The stylus pen can also be used with television sets configured to operate as a computer display, as well as backlit monitors and plasma panels. The exemplary peripheral design also permits the stylus pen to be used either m conjunction with a mouse or as a substitute therefore. The foregoing may be achieved by integrating the exemplary stylus pen with a device enabling the monitor to act as a transmitter. An optical sensor m the stylus pen acts as a receiver.

Referring now to FIGURE 1, there is illustrated a block diagram of an exemplary computer system, referred to generally by the numeral 100, configured in accordance with a first embodiment of the present invention. The system 100 includes a general purpose computer system 105 with a mouse port 110, a keyboard port 115, and a monitor port 120. The computer system 105 performs general computer processing tasks and can be operated m a stand-alone mode or be connected to additional computers to form a computer network. The general purpose computer 105 receives user inputs through the mouse port 110 and the keyboard port 115 and transmits outputs from the monitor port 120.

The computer system 105 is configured to receive inputs, and to transmit outputs using electronic data signals. Peripheral devices are connectable to the computer system 105 to serve as an interface for inputs to be received from, and outputs to be communicated to users. Peripheral devices include, for example, a keyboard 125, a mouse 130, and a monitor 135. The keyboard port 115 is connectable to keyboard 125, permitting the user to provide text-based inputs. The mouse port 110 is connectable to a motion based user input device, commonly known as a mouse 130 (although not shown connected) . The monitor port 120 is connectable to a monitor 135, wherein outputs are graphically displayed to the user via a display screen 136. The monitor 135 is also used to assist the user in providing inputs, by displaying graphical user interfaces (GUI). A graphical user interface prompts the user to provide inputs to the computer system 100 in a user-friendly, intuitive manner. For example, a GUI is often used to display a number of items associated with various user inputs. The user provides an input by pointing to the item associated with the input and pressing a button on the mouse 130. The foregoing process is known as pointing and clicking.

Another example of a peripheral device is a stylus pen 140. The stylus pen 140 allows the user to input high resolution graphical images into a GUI by placing the stylus pen 140 over the monitor 135 and over a particular monitor displayed pixel (s), and drawing a graphical image directly (seemingly) on the screen. The stylus pen 140 itself, however, does not directly write to the monitor 135. When the stylus pen is placed against the monitor's display screen 136, the stylus pen 140 detects a location indicator in the form of an illuminated pixel. The location indicator is found within an array of location indicia that are optically displayed at the particular pixel location (s) on the display screen 136. The recognized location indicator can be referred to as the pixel location of the stylus pen. Thus, the pixel location of the stylus pen 140 is ascertained and provided to the computer system 105 from the detected location indicator. The computer system 105 can then provide the pixel location of the stylus pen 140 to various software programs.

Virtually everything displayed on the screen 136 of conventional monitors, such as monitor 135, must be provided by an external monitor driver source due to conventional monitors being passive devices. The computer system 105 provides a stream of graphical image data for display on the monitor 135. The aforementioned location indicia must also be provided to the monitor 135. In the exemplary embodiment shown in FIGURE 1, the aforementioned location indicia is provided by a subsystem 145. The subsystem 145 is connected to the monitor 135 and outputs the location indicia thereto.

Referring now to FIGURE 2, there is illustrated a block diagram of an exemplary stylus pen 140 in accordance with the present invention. The stylus pen is placed in contact with the monitor 135 so that it can optically view the pixels on the screen 136. When the location indicia is displayed on the monitor 135, the location indicia associated with the particular pixel over which the stylus pen 140 is located is detected by an optical sensor in the stylus pen 140. The optical sensor converts the location indicia to a data signal and transmits (via a direct connection or a cordless interface) the data signal to the subsystem 145. The subsystem 145 can then determine the pixel location of the stylus pen 140. The stylus pen 140 includes a fiber optic bundle 205 mounted into a swivel tip 210. The fiber optic bundle 205 provides an optical route for received light to an optical sensor 215. The tip of the fiber optic bundle 205 is mounted such that it has a flat surface 225 that stays flush with the display screen via the swivel tip 210. The combination of fiber optic bundles 205, the flat face 225 and the swivel tip 210 allows the optical sensor 215 to optically read the location indicator displayed at a particular pixel under the tip of the fiber optic bundle 205 with maximum clarity from most writing angles.

The stylus pen 140 also includes a micro-switch 230 disposed in a manner such that when the tip of the fiber optic bundle 205 is placed m contact with the monitor 135, the mechanical pressure from the contact causes the micro- switch 230 to be depressed. Depressing the micro-switch 230 causes the stylus pen 140 to signal the subsystem 140 that the stylus pen 140 is m an engaged mode. Additionally, where the fiber optic bundle 105 is moved away from the monitor 135, the stylus pen 140 signals the subsystem 145 that stylus pen is m a disengaged mode.

In another embodiment of the present invention, a light intensity is calculated using the optical sensor 215 instead of using the micro-switch 230. By utilizing a light intensity reading to place stylus pen m engage or disengage mode, a user need only place the tip of the stylus pen on the screen to engage it. Furthermore, by eliminating the micro-switch 230, the number of circuits in the stylus pen 140 is decreased and the micro-switch will not become a critical failure point m the system due to wear. Furthermore, by utilizing light intensity, the user can accomplish a substantially similar result by touching the monitor screen or having the tip of the stylus pen close to the monitor screen. When the fiber optic bundle 205 is placed in contact with the screen 136 of the monitor 135 directly over a location pixel, the optical sensor 210 detects the light wavelength (or color) of the light radiated from the pixel, and transmits a corresponding electronic signal to the subsystem 145. The pixel location of the stylus pen 140 is then determined by the subsystem 145 by ascertaining the pixel location associated with the color corresponding to the electronic signal. The stylus pen 140 also includes two buttons 240a, 240b, which correspond to the right and left buttons, respectively as found on a standard two- button mouse.

The full matrix mode will now be described. When the stylus pen 140 first makes contact with the screen 136 of the monitor 135, the graphical image displayed on the screen 136 is replaced by full matrix mode location indicia detectable by the stylus pen 140 and identifiable by a location determination algorithm which will be described below. Referring now to FIGURE 3, there is illustrated a diagram of the monitor screen 136 displaying the location indicia during the full matrix mode. The image displayed on the monitor screen 136 is composed of a two-dimensional array of pixels 305. Each pixel 305 displays a predetermined color for a predetermined time period. The time period can be equal to or related to the refresh period of the monitor 135.

Each area 305 of the display screen 136 is defined by its own color pattern. An area may be a group of pixels or a single screen pixel. In full matrix mode the display screen 136 is defined by a fixed number of rows 315 and columns 310. Each row 315 and column 310 is then "labeled" with its own unique color. An equation is created to define a color to fill each area or pixel 305. The color is from either the row 315 or column in which the area 305 resides. An exemplary equation for an 800 x 600 resolution display screen using 100 colors for columns is now described:

Define the colors for the columns 310 (Xcolor) and colors for the rows 315 (Ycolor) in a sequential fasion.

Note that in this example the height of the display screen is 0.75 times the width of the display screen, thus there will be three quarters the number of colors used for rows as for columns. The number of colors for columns (Ncolors) is used to calculate a color pattern.

Thus, Xcolor = ( 0 to 99) Ycolor = (100 to 174)

The total number of pixels = pixel resolution for the rows on the display. Example : For 800 x 600 resolution display

Total pixels = 800

Next, a color for each pixel column 310 is calculated. The result will be an integer value that can be used to cross reference a color table: Xcolor = [ (Pixel (X) /Total Pixels) x Ncolors] Truncated

Example: Xcolor (X=300) =[( 300/800) x 100] Truncated

Next, a color for each pixel row is calculated. The result will be an integer value that can be used to cross reference a color table: YColor=[ (Pixel (Y) /Total Pixels) x Ncolors] Truncated

+ Ncolors

Example: Ycolor (Y=233) =[ (233/800) xlOO] Truncated + 100 = 129

Next, it is decided which color is used for each pixel 305: If Pixel (X) + Pixel (Y) is odd, then fill the pixel with Xcolor for that pixel location. If Pixel (X) + Pixel (Y) is even, then fill the pixel with Ycolor for that pixel location.

Example: For Pixel (X=300, Y=233) fill with Xcolor, which is color #37

To determine the screen location of the stylus when it is first touched to the screen, the location indicia is flashed on the display screen so that the optics of the stylus can read the color (Xcolor or Ycolor) of the location. The stylus, via the optical sensor, is polled so that the detected color can be determined. The color is represented as the Xcolor number so that, via the above equations, the location of the stylus can be determined. The equations that follow disclose an exemplary technique for determining the location of the stylus based on the color read by the stylus optics.

First, cross reference the selected color table to determine Xcolor and Ycolor. (Note: If a color is detected that is not in the selected color table, the system will log an error) :

If the color returned is between 0 and Ncolors, then it is Xcolor.

If the color returned is between (Ncolors+1) and(l .74*Ncolors) , then it is Ycolor Example : Colors returned from sensor: Color

#129 and Color #37

Therefore Xcolor=37 and Ycolor=129

Reverse calculate the pixel reference position for the pixel column from Xcolor:

ReferencePixell (X) = [Xcolor* (TotalPixels) / (Ncolors) ] Example : Xcolor = 37

ReferencePixel = [37*800/100] = 296

Reverse calculate the pixel reference position for the pixel row from Ycolor: ReferencePixel (Y) = [Ycolor-Ncolors) * (Total

Pixels) / (Ncolors) ]

Example : Ycolor = 129

Reference Pιxel=[ (129-100) *800/100] =232

In the example shown, the pixel position read was (300 x 233), and the reference position returned was (296 x 232) . This is extremely close to the actual stylus position. The series of equations and calculations shown are capable of reaching double pixel resolution through the simple use of more colors. Once the stylus location on the display screen is determined, then a moving overlay pattern is positioned on the screen about the stylus pen's location. The moving overlay can be substantially any shape, but preferably is square. Once the moving overlay pattern is displayed on the display screen tracking mode is initiated.

Referring to FIGURE 4, the tracking mode will now be described. In order to track movement of the stylus across the screen, tracking mode is used. The computer system utilizes the tracking mode at a speed which allows enough stylus pen positions to be gathered for effective use in a writing or drawing application.

Tracking mode is used to track the stylus pen 140 as it is being moved across the screen 136 from the initial point of contact with the screen 136 to other positions on the screen. An overlay pattern 405 (exploded view) is displayed on top of the graphic images 415 displayed on the screen 136. The overlay pattern 405 must be both visible to the user and allow the user to see the graphic images 415 that are being 'drawn on' by the stylus 140.

An exemplary overlay pattern 405 is 20x20 pixels but could be larger than 100 x 100 pixels in size. The overlay pattern 405 comprises vector indicators 404 in every other pixel location within the overlay pattern 405. The vector indicators 404 are each assigned a different color and may create a checkerboard pattern over graphic image 415. The graphic image portion that is underneath the overlay pattern 405 is converted to a black and white (gray scale) so that the graphic image portions pixels are not confused with the vector indicators 404. Note that the pattern does not have to be checkerboard but could be octagons, pentagons, triangles or substantially any repeating or interlocking pattern. It is further noted that the patterns could incorporate as few as one-third to one-eighth of all the pixels in the overlay pattern, but preferably one-half the pixels provides the greatest stylus tracking accuracy and resolution at normal stylus movement speeds. In other words, the portion of the original image that is underneath the overlay pattern 405 is converted to gray scale, while a checkerboard pattern of half the pixels in the overlay pattern are each displayed as a different color.

In FIGURE 4, each numbered (or Xed) location in the overlay pattern 405 represents a uniquely colored vector indicator 404. The center location represents the known stylus pen position 410. The stylus pen position 410 is centered in the overlay pattern 405 on refresh of the screen 136. As the stylus pen 140 is moved across and in contact with the screen, the stylus pen 140, via the optical sensor, reads a localized indicator 404 that is displayed under the stylus pen's location. The color of the localized indicator 404 is determined thereby providing information to the computer system which can be used to determine new stylus pen position information and pen movement vector information.

Upon a screen refresh, the moving overlay pattern is recentered about the last determined position of the stylus pen. Furthermore, a graphic trail following the stylus can be created as the pen moves across the screen. The graphic trail simulates writing on the display screen 136. In a preferred embodiment, the stylus pen's optics can read and interpret the vector indicators 404 at a rate that exceeds the refresh rate of the screen. Thus, the graphic trail behind the pen can be updated with a plurality of stylus pen locations on a screen refresh. Furthermore, the trail behind the stylus pen (see FIGURES 1 and 7) can be a high resolution trail of one or more pixels m width. The overlay pattern 405 is preferably large enough so that the stylus pen 140 is not likely to be moved outside of the overlay pattern 405 by a user for the duration of a refresh period. The preferred embodiment can sense three to five or more vector indicators per refresh cycle, which allows the exemplary embodiment to take 300 to 500 or more stylus location reads per second. Since one exemplary application of the present exemplary invention is for "brainstormmg tools" like white boards, or for artistic tools like freehand drawing, it is important to allow a user to move the stylus pen on the screen at speeds that are comfortable to the user and at speeds which will allow the user to view an accurate depiction of what they are drawing on the screen instantly.

The stylus pen can be tracked anywhere on the screen with minimal obstruction to graphical images being displayed on the screen. The location of the pen is determined at a rate that far exceeds the refresh rate of the screen. That is, the pen location is determined at a rate that is three to five times more often than the refresh rate. The speed of stylus pen location calculations is limited by the speed of the host computer system 105 and/or the integrated circuit speeds found m the computer's graphic circuitry, not the speed of the screen refresh rate.

Referring back to FIGURE 2, the optical sensor 215 m the stylus pen must be able to discriminate discrete colors in a large color spectrum produced by a color display monitor. An exemplary computer system utilizing 8 bit (256 color) technology is sufficient to meet the design requirements for matrix mode and tracking mode. CCD or CMOS optical sensors have been determined to be able to provide the required optical sensing capability m an exemplary stylus pen. It is understood that other types of optical sensors could be used without departing from the spirit of the invention. Furthermore, it is understood that CCD and CMOS sensors could also be utilized with color technology at 10 bit (1024 colors) or 24 + bit (true color) . As the color technology advances and pixel size decreases, the vector indicators may each comprise a plurality of pixels instead of a single pixel each. Referring now to FIGURE 5, there is illustrated a block diagram of the subsystem 145 m accordance with an embodiment of the present invention. The subsystem 145 includes a central processor 505 which executes instructions stored m non-volatile memory 510, such as Programmable Read-Only Memory (PROM) . The central processor 505 receives inputs from the stylus pen 140, the mouse 130, and graphical images from the monitor port 110 of the computer system 105, and provides outputs to the monitor 135 and the mouse port 110 of the computer system 105. The central processor 505 receives the input from the stylus pen 140 via the stylus port 525. The inputs received from the stylus pen 140 include a status signal describing the mode of the stylus pen, the electronic signal provided by the optical sensor 215, and signals corresponding to the buttons 240a, 240b. When the stylus input port 525 receives the engaged signal from the stylus pen 140, the central processor 505 retrieves the electronic signal provided by the optical sensor 215 through the stylus port 525. The central processor 505 then determines the pixel location and/or offset vector based on the signal provided from the stylus pen's 140 optical interpretation of the location indicator's color or the localized vector indicia's color.

The pixel location and/or vector indicator location of the stylus pen 140 can be determined m a number of ways that would be understood by one skilled m the art. In one scheme, the electronic signal is compared to determine the associated vector indicator location within the overlay pattern. The indicator location is then offset by the center of overlay patterns location. The non-volatile memory 510 can be rewritten or reinitialized so that on initialization a sampling of the screen's color variation capabilities can be loaded or reloaded into the look-up table.

The location of the stylus pen 140 may be determined by transmitting the location or vector indicator information from the subsystem to the mouse port 110 of the general purpose computer system 105. In order to provide inputs to the computer 105 from the subsystem 145, a mouse/stylus driver 530 is placed between the central processor 505 and the mouse port 110. The mouse/stylus driver 530 enables the system to convert an absolute location to a mouse motion signal that is transmitted to the mouse port 110 of the computer system 105. A user may use the computer system 105 to perform functions with or without using the stylus pen 140. If the stylus pen 140 is not being used, the user may use the mouse 130 instead. In order to use the mouse 130 with the exemplary computer system 105, the subsystem 145 is connected to and receives signals from the mouse 130 via the auxiliary mouse port 535. The mouse driver 530 includes decision logic that can selectively pass the signals received from the mouse 130 or the stylus pen 140 to the mouse port 120. When the stylus pen 140 is disengaged, the inputs of the auxiliary mouse port 535 are provided to the computer system 105. When the stylus pen 140 is engaged, the mouse 130 is disengaged and information data which will move the cursor to the detected location of the stylus pen is provided to the computer system 105. The mouse driver 530 will determine that the stylus pen is engaged when the micro-switch 230 is depressed. The central processor's 505 control of the monitor 135 is effectuated by a screen driver 540. The screen driver 540 is based upon a standard screen driver as can be found in general purpose computer systems 105. The screen driver 540 allows the subsystem 145 to interrupt the graphical images for display on the monitor 135 and overlay it with the location indicia or the overlay pattern comprising vector indicators that are detectable by the stylus pen 140. The central processor 505 outputs the images to be displayed to the screen driver 540. The screen driver 540 then causes the monitor 135 to display the prescribed images.

The screen driver 540 can be provided with preferably one of three patterns by the central processor 505 - (1) unmodified graphical images from the computer system 105; (2) the location indicia (full matrix mode); or (3) the moving overlay pattern interposed with or overlaying the graphical images received from the computer system 105. The graphical images are received from the computer system 105 via a screen input port 545. The screen input port 545 is connectable to the display port 120 of the computer system 105. When the central processor 505 displays the graphical images from the computer system 105 for display, the central processor 505 outputs the contents of the screen input port 545 to the screen driver 540.

The location indicia are stored in the non-volatile memory 510. When the central processor 505 selects the location indicia for display (due to the stylus pen 140 being engaged) on the monitor 135, the central processor 505 accesses the location indicia from the non-volatile memory 510 and outputs the pixel location indicia to the screen driver 540 such that it is interposed with or overlays the image already displayed on the screen. The localized vector indicators are also stored in the non-volatile memory 510. When the central processor 505 selects the moving overlay pattern for display (tracking mode), the localized vector indicators are interposed with the graphical images provided by the computer system 105 in an area about the stylus pen's position. The central processor provides the graphical images from the screen input port 545 and accesses the vector indicia from the non-volatile memory 510. The localized vector indicators are interposed with an overlay of the graphical images about the pixel location of the stylus pen 140 at the most recent refresh by the central processor 505 and output to the screen driver 540. Note, that the stylus pen 140 can read the location indicia or localized vector indicators at a rate that exceeds the screen's refresh rate.

Referring now to FIGURE 6, there is illustrated a flow diagram describing exemplary operation of the subsystem in accordance with the present invention. The subsystem constantly polls (step 605) the status of the stylus pen to determine whether the stylus pen is engaged (step 610) . When the stylus pen is not engaged, i.e. disengaged during step 610, the subsystem forwards the graphical images received from the computer system to the monitor (step 615) . While the stylus pen is in a disengaged mode, steps 605 through 615 are repeated and the stream of graphical images received from the computer system are displayed on the monitor. Furthermore, the mouse 130 is fully operational when the stylus pen 140 is disengaged.

However, when the stylus pen becomes engaged during step 610, the subsystem temporarily halts displaying the graphical images received from the computer system and instead displays the pixel location indicia of the matrix mode (step 620) . The location indicia are displayed until the subsystem receives a signal from the stylus pen detecting the location indicia at the pixel location of the stylus pen (step 625) . In one exemplary embodiment, the location indicia may be interposed with the graphical images received from the computer system. When the subsystem receives a detection signal from the stylus pen identifying one of the location indicia for the pixel location of the stylus pen, the subsystem determines the pixel location of the stylus pen (step 630) and forwards it to the computer system (step 635) . The pixel location of the stylus pen 140 is communicated to the computer system 105 by calculating a vector from the last known mouse location to the pixel location of the stylus pen 140.

After determining the initial pixel location of the stylus pen during step 630, the pen can be moved across the display screen. The location of the stylus pen can be tracked by displaying a localized overlay pattern about the initial location of the stylus pen. At step 640, the graphical image received from the computer system is interposed or overlayed with the overlay pattern which comprises vector indicators. The overlay pattern is preferably centered about the pixel location of the stylus pen and displayed on the monitor. The subsystem then waits until a 'next' detection signal is received from the stylus pen (step 650) . When the next detection signal is received from the stylus pen, the subsystem determines a two dimensional offset vector from the detection signal (step 655) and forwards the two dimensional offset vector to the computer system (step 660) via the mouse port. The subsystem then updates the pixel location of the stylus pen (step 665) by adding the offset vector determined during step 655 to the last pixel location of the stylus pen. It is noted that a plurality of detection signals may have been received and recorded between each screen refresh. At step 670, the subsystem polls the status of the stylus pen to determine if the stylus pen is still engaged (step 675) . If the stylus pen is engaged, a determination is made as to whether the refresh period has expired (step 680) . If the refresh period has expired, the moving overlay pattern is interposed or overlayed with the graphical image received from the computer system 105 and positioned about the pixel location of the stylus pen 140 determined during step 665. Steps 650 to 680 are repeated until the stylus pen becomes disengaged during step 675. When the stylus pen becomes disengaged during step 675, the subsystem returns to the disengaged mode at step 605. The foregoing exemplary system and associated exemplary methods advantageously permits a user to easily adapt a general purpose computer system 105 and conventional monitor 135 to practice the present invention by simply connecting the subsystem 145 as shown m FIGURE 1. However, recent advances m microelectronics technology have provided for portable computers known m the art as laptop computers. A laptop computer usually includes a computer system integrated with a display screen. However, because the display screen is integrated with the computer system, a laptop computer is not as easily adaptable for use with the exemplary subsystem 145. Furthermore, the configuration and communication protocols of the display screen on many laptop computers are proprietary, as opposed to standardized, adding further complexity. Referring now to FIGURE 7, there is illustrated a computer system 705 configured m accordance with another embodiment of the present invention. The computer system 705 includes a stylus pen port 710 for receiving signals from a connectable stylus pen 140, an integrated display device 715, and a central processing unit 720 for performing processing functions. The stylus pen port 710 can be a dedicated port created by the computer manufacturer or a standard serial port. It is understood that the integrated display device can be substantially any type of color display device. The signals received by the stylus pen port 710 are forwarded to the central processing unit 720. The central processing unit 720 responds to the inputs received from the stylus pen port 710 by causing location indicia 305 and/or movable overlay patterns 405 to be displayed onto the integrated display device 715 in accordance with the flow chart of FIGURE 6 by executing sets of computer readable instructions. The sets of instructions can be stored in, for example, RAM 725, ROM, or any data storage or memory device. Until required by the central processing unit 720, the set of instructions may be stored m a variety of computer readable memory devices, for example m a hard disc drive 730, or m removable memory such as an optical disc 735 for eventual use m an optical disc drive 740, or a floppy disc 745 for eventual use in a floppy disc drive 750. Although the invention has been described with a certain degree of particularity, it should be recognized that elements thereof may be altered by persons skilled m the art without departing from the spirit and scope of the invention. Therefore, the invention is limited only by the following claims and their equivalents.

Claims

WHAT IS CLAIMED IS:
1. A computer system that allows a user to effectively write on a display screen with a stylus, said computer system comprising: a general purpose computer; a subsystem connected to said general purpose computer; a display screen for receiving graphics data from said general purpose computer and for displaying said graphics on said display screen, said display screen further receiving location indicators from said subsystem for display on said display screen; and a stylus adapted to optically read at least one said location indicator displayed on said display screen, said optically read location indicator being provided to said general purpose computer in the form of data so that a location of said stylus pen on said display screen can be calculated.
2. The computer system of claim 1, wherein said location indicators are predetermined colored locations displayed on said display screen.
3. The computer system of claim 1, wherein said location indicators are displayed at predetermined colored pixel locations on said display screen.
4. The computer system of claim 1, wherein said stylus optically reads a plurality of location indicators during single refresh period of said display screen.
5. The computer system of claim 1, further comprising a peripheral point and click device that provides point and click information to said personal computer.
6. The computer system of claim 5, wherein said point and click device is inoperative when said stylus is reading said location indicators on said display screen.
7. A method of determining a position of a stylus on a display screen comprising the steps of: touching a stylus to said display screen; displaying an array of color coded locations on said display screen; optically reading a color coded location, of said array of color coded locations, that is located under said stylus; and determining a first location of the stylus on said display screen based on the optically read color coded location.
8. The method of claim 7, wherein each said color coded location is a pixel location on said display screen.
9. The method of claim 7, wherein after the step of determining, the method further comprises the steps of: positioning an overlay pattern on said display screen at said first location such that said overlay pattern is substantially centered about said first location; moving said stylus across said display screen; optically reading a plurality of color coded vector locations within said overlay pattern; determining the locations of said moving stylus pen based on said optically read color coded vector locations; and repositioning said overlay pattern at one of said plurality of said optically read color coded vector locations.
10. The method of claim 9, wherein is said step of repositioning is performed once every screen refresh cycle.
11. The method of claim 9, further comprising a step of displaying a trail effectively behind said stylus, said trail comprising at least said optically read color coded vector locations.
12. The method of claim 9, wherein said step of determining said locations further comprises the step of substantially calculating a movement speed of said stylus.
13. The method of claim 9, wherein said step of determining said locations further comprises the step of substantially calculating a movement vector of said stylus.
14. An apparatus for determining positions of a stylus on a display screen, said apparatus comprising: a stylus having an optical sensor for reading and discriminating between colors displayed on a display screen; an article of manufacture for implementing a stylus tracking procedure, said article of manufacture comprising: at least one computer readable medium; processor instructions contained on said at least one computer readable medium, said processor instructions configured to be readable by said at least one processor and thereby cause said at least one processor to operate to: poll a status of said stylus to determine whether said stylus is touching said display screen; display an array of colors on said display screen; receive an optically interpreted first detection signal from said stylus; and determine the position of said stylus on said display screen based on said first detection signal.
15. The apparatus for determining positions of a stylus on a display screen of claim 14, wherein said processor instructions further cause said at least one processor to operate to: overlay an overlay pattern substantially about said determined positions of said stylus, said overlay pattern comprising a plurality of vector indicators; receive optically interpreted vector indicator signals from said stylus as said stylus is moved across said display screen; tracking a plurality of stylus positions based on said received optically interpreted vector indicators; and positioning a new overlay pattern on said display screen about determined stylus positions.
16. The apparatus for determining positions of a stylus on a display screen of claim 14, wherein said array of colors is display such that a plurality of the pixels of said display screen are different colors arranged in said array of colors.
17. The apparatus for determining positions of a stylus on a display screen of claim 15, wherein said overlay pattern is smaller than 100 pixels by 100 pixels.
18. The apparatus for determining positions of a stylus on a display screen of claim 15, wherein said overlay pattern comprises a plurality of colors positioned in predetermined locations about said overlay pattern.
19. The apparatus for determining positions of a stylus on a display screen of claim 18, wherein said plurality of colors are positioned in a checkerboard pattern.
20. The apparatus for determining positions of a stylus on a display screen of claim 15, wherein said new overlay patterned is positioned on said display screen at a stylus position determined once every screen refresh cycle.
21. A method for inputting a high resolution image into a personal computer for display onto a display device, the method comprising the steps of: polling a status associated with a stylus pen; displaying a matrix, when said status is engaged, comprising a first plurality of colors, wherein each of said plurality of colors corresponds to a location on the display device; optically reading a particular one of the first plurality of colors with said stylus pen and calculating a location corresponding to the particular one of the plurality of colors; and displaying a direction box comprising a second plurality of colors, wherein each of said second plurality of colors corresponds to at least one of a direction associated with a vector and a position optically reading with said stylus pen a particular one of the second plurality of colors and calculating at least one of a vector associated with the particular one of the second plurality of colors and a position of the particular one of the second plurality of colors.
PCT/US2000/012995 1999-05-14 2000-05-11 Stylus pen for writing on the monitor WO2000070551A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US31186999 true 1999-05-14 1999-05-14
US09/311,869 1999-05-14

Publications (1)

Publication Number Publication Date
WO2000070551A1 true true WO2000070551A1 (en) 2000-11-23

Family

ID=23208861

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/012995 WO2000070551A1 (en) 1999-05-14 2000-05-11 Stylus pen for writing on the monitor

Country Status (1)

Country Link
WO (1) WO2000070551A1 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001061453A2 (en) * 2000-02-16 2001-08-23 Telefonaktiebolaget Lm Ericsson (Publ) Method and system for using an electronic reading device on non-paper devices
US6611259B1 (en) 2000-02-16 2003-08-26 Telefonaktiebolaget Lm Ericsson (Publ) System and method for operating an electronic reading device user interface
US6693623B1 (en) 2000-02-16 2004-02-17 Telefonaktiebolaget Lm Ericsson (Publ) Measuring applications for an electronic reading device
US6738053B1 (en) 2000-02-16 2004-05-18 Telefonaktiebolaget Lm Ericsson (Publ) Predefined electronic pen applications in specially formatted paper
GB2397188A (en) * 2003-01-08 2004-07-14 Jonathan Richard Swift Dedicated art picture display
US6813396B1 (en) 2000-02-16 2004-11-02 Telefonatiebolaget L.M. Ericsson (Publ) Method for sharing information between electronic reading devices
US6839623B1 (en) 2000-02-16 2005-01-04 Telefonaktiebolaget Lm Ericsson (Publ) Positioning applications for an electronic reading device
US6885878B1 (en) 2000-02-16 2005-04-26 Telefonaktiebolaget L M Ericsson (Publ) Method and system for using an electronic reading device as a general application input and navigation interface
US6952497B1 (en) 2000-02-16 2005-10-04 Telefonaktiebolaget L M Ericsson (Publ) Method and system for electronically recording transactions and performing security function
US8436837B2 (en) 2010-02-25 2013-05-07 Hewlett-Packard Development Company, L.P. Stylus input system
US8619172B2 (en) 2010-03-31 2013-12-31 Hewlett-Packard Development Company, L.P. Rotatable camera assembly

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1998035336A1 (en) * 1997-02-09 1998-08-13 Roni Raviv Display pointing device

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1998035336A1 (en) * 1997-02-09 1998-08-13 Roni Raviv Display pointing device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"PEN INPUT DEVICE", IBM TECHNICAL DISCLOSURE BULLETIN,US,IBM CORP. NEW YORK, vol. 33, no. 10A, 1 March 1991 (1991-03-01), pages 248 - 251, XP000110032, ISSN: 0018-8689 *
"PROGRAMMABLE LIQUID CRYSTAL DISPLAY MOUSE PADS", IBM TECHNICAL DISCLOSURE BULLETIN,US,IBM CORP. NEW YORK, vol. 32, no. 3B, 1 August 1989 (1989-08-01), pages 211 - 213, XP000029875, ISSN: 0018-8689 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001061453A2 (en) * 2000-02-16 2001-08-23 Telefonaktiebolaget Lm Ericsson (Publ) Method and system for using an electronic reading device on non-paper devices
WO2001061453A3 (en) * 2000-02-16 2002-01-10 Ericsson Telefon Ab L M Method and system for using an electronic reading device on non-paper devices
US6611259B1 (en) 2000-02-16 2003-08-26 Telefonaktiebolaget Lm Ericsson (Publ) System and method for operating an electronic reading device user interface
US6693623B1 (en) 2000-02-16 2004-02-17 Telefonaktiebolaget Lm Ericsson (Publ) Measuring applications for an electronic reading device
US6738053B1 (en) 2000-02-16 2004-05-18 Telefonaktiebolaget Lm Ericsson (Publ) Predefined electronic pen applications in specially formatted paper
US6813396B1 (en) 2000-02-16 2004-11-02 Telefonatiebolaget L.M. Ericsson (Publ) Method for sharing information between electronic reading devices
US6839623B1 (en) 2000-02-16 2005-01-04 Telefonaktiebolaget Lm Ericsson (Publ) Positioning applications for an electronic reading device
US6885878B1 (en) 2000-02-16 2005-04-26 Telefonaktiebolaget L M Ericsson (Publ) Method and system for using an electronic reading device as a general application input and navigation interface
US6952497B1 (en) 2000-02-16 2005-10-04 Telefonaktiebolaget L M Ericsson (Publ) Method and system for electronically recording transactions and performing security function
GB2397188A (en) * 2003-01-08 2004-07-14 Jonathan Richard Swift Dedicated art picture display
US8436837B2 (en) 2010-02-25 2013-05-07 Hewlett-Packard Development Company, L.P. Stylus input system
US8619172B2 (en) 2010-03-31 2013-12-31 Hewlett-Packard Development Company, L.P. Rotatable camera assembly

Similar Documents

Publication Publication Date Title
Zhang et al. Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper
US4963858A (en) Changeable input ratio mouse
US6952198B2 (en) System and method for communication with enhanced optical pointer
US5473344A (en) 3-D cursor positioning device
US5852434A (en) Absolute optical position determination
US6624824B1 (en) Tilt-scrolling on the sunpad
US7203911B2 (en) Altering a display on a viewing device based upon a user proximity to the viewing device
US6151015A (en) Pen like computer pointing device
US20090179854A1 (en) Dynamic input graphic display
US6043805A (en) Controlling method for inputting messages to a computer
US4787051A (en) Inertial mouse system
US5767842A (en) Method and device for optical input of commands or data
US20110187647A1 (en) Method and apparatus for virtual keyboard interactions from secondary surfaces
US20110227947A1 (en) Multi-Touch User Interface Interaction
US5686942A (en) Remote computer input system which detects point source on operator
US20080192017A1 (en) Automatic Projection Calibration
US20090109175A1 (en) Method and apparatus for user interface of input devices
US7355583B2 (en) Motion-based text input
US6359616B1 (en) Coordinate input apparatus
US20090115725A1 (en) Input device and method of operation thereof
US7257255B2 (en) Capturing hand motion
US4369439A (en) Cursor position controller for a display device
US20100058251A1 (en) Omnidirectional gesture detection
US20100001963A1 (en) Multi-touch touchscreen incorporating pen tracking
US20070146325A1 (en) Computer input device enabling three degrees of freedom and related input and feedback methods

Legal Events

Date Code Title Description
AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP