WO1997000713A1

WO1997000713A1 - Method and apparatus for controlling images with a centrally located displacement control device

Info

Publication number: WO1997000713A1
Application number: PCT/US1996/010875
Authority: WO
Inventors: Andrew L. Carter
Original assignee: Apple Computer, Inc.
Priority date: 1995-06-23
Filing date: 1996-06-24
Publication date: 1997-01-09
Also published as: AU6394296A

Abstract

An input control device is coupled to a host system. A displacement control device is centrally located in the input control device and controls the location of a cursor displayed over video images, scrolling of the video images and adjusting of the point of view of the images, the images being generated by the execution of an application program by a dedicated multimedia game system or a personal computer. The cursor can be used to point to a particular portion of the image (i.e. a target image), the portion being as small as a pixel. The input control device has a body which has both a concave and a convex edge which taper at the ends to form cusps of a crescent shape. The body has a hollow portion at or near the crests of the concave and convex edges. In one embodiment, a trackball assembly is employed as the displacement control device and is disposed within the hollow space so that a portion of its trackball extends up through the top surface of the body and is accessible to an operator. The operator rotates the trackball and the resulting displacement is translated by a microcontroller, coupled to the trackball device and located within the hollow portion, into digital information representing the magnitude and direction of the displacement. The digital displacement information generated by the microcontroller is then transmitted to the host system over a databus coupling the microcontroller to the host system. The host system converts the digital displacement information into signals which drive the video display to control the displayed images. A second embodiment employs a track pad as the displacement control device. Both embodiments also have two actuator buttons located at the crest of the convex edge, three actuator buttons located at the crest of the concave edge, four actuator buttons arranged in an arcuate pattern to the right of the trackball and four actuator buttons arranged in a cross pattern to the left of the trackball. The buttons can be programmed to identify, select, pull-down, drag or fire upon the target video images pointed to by the cursor, as well as to scroll or change point-of view based non-displacement switch activation.

Description

METHOD AND APPARATUS FOR CONTROLLING IMAGES WITH A CENTRALLY LOCATED DISPLACEMENT CONTROL DEVICE

1. FIELD OF THE INVENTION: The present invention relates to video game input controllers, and more particularly to hand-held video game input controllers that are optimized for target acquisition and selection, scrolling and point-of-view image control and graphics input.

2. BACKGROUND OF THE RELATED ART:

When video games first appeared on the market, they were typically implemented in the form of large arcade-style machines. These arcade-style video game machines typically included a video screen for displaying the video game, a pointing device by which the location of a cursor displayed on the screen was manipulated to acquire targets, and a number of select or actuator buttons which were used to select or fire upon the targets pointed to by the cursor.

The pointing devices used by such arcade style machines to manipulate the location of the cursor to acquire targets could be divided into two major categories. Devices falling within the first of such categories are sometimes known as Fitts' law devices. Fitts' law devices translate a literal spatial displacement of the control feature of the device, relative to some instantaneous datum, into a displacement of the cursor location on the video screen. Fitts' law devices can be further divided into relative reporting devices and absolute reporting devices.

For relative reporting devices, there is no beginning or ending to the range of motion of the sensor. Each displacement reported to the screen is relative to the last position of the cursor on the screen. Trackballs and mice are examples of inherent relative reporting devices. For relative reporting devices such as a trackball, displacement of the ball is encoded in device increments which are then translated by the host system into abstract digital increments. The host system then applies the appropriate scaling and acceleration factors to produce the movement of the cursor on the screen. A mouse converts a relative spatial displacement along a desk top or mouse pad into a displacement of the cursor on the screen. One or more selector buttons are located on the mouse so that the operator can acquire and select the target using the same hand. Trackballs are sometimes embedded in computer keyboards and permit the operator to acquire a target with typically

1 BSTITUTE SKuET (RULE 26) one finger of one hand while selecting the target, either with a different finger of the same hand or a finger of the other hand, using predesignated keys to select the target. As with the arcade games that employ trackballs, because the keyboard rests on a support surface such as a desk top, the operator has the freedom to operate the trackball with one hand while selecting the target with the other.

In the case of absolute reporting devices, displacement of the control feature from a datum position (usually the center neutral position) is translated into a movement of the cursor. The reported position of the control feature is typically limited to a rectangular area having defined borders or limitations to the working area. The reported position is made either in literal spatial dimensions or some abstract machine representation of distance. Examples of absolute reporting devices include track pads, graphics tablets and displacement joysticks. Absolute reporting devices are better suited for certain tasks, such as free hand drawing or writing, while relative reporting devices are better suited for target acquisition tasks.

Graphic tablets typically sense an absolute location which is pointed to by a pen and which is converted to an absolute location on the screen. The target can be selected by applying pressure to the pen which activates a select switch. Similarly, track pads sense the location of contact between its surface and, for example, a user's finger. The track pad surface forms a two- dimensional distribution of elements which are sensitive to changes in local capacitance which is temporarily altered by the proximity of the user's finger tip. A transducer senses this disturbance in the local capacitance and produces a signal defining the location of the contact in two dimensions.

Displacement joysticks, which are inherently absolute reporting devices, suffer from disadvantages in that the cursor range on the screen will be limited to the range of motion of the joystick in relation to its initial center- neutral position; the cursor will always snap back to its original position. Thus, an absolute reporting joystick is not an optimal solution to the function of target acquisition. Devices which are inherently absolute reporting can be operated in modes defined by machine software to report displacements relative to the last prior position, which eliminates some of their disadvantages. Trackballs have sometimes been implemented in arcade-style video game machines. The trackball is typically recessed into the console of the video game machine such that only a portion of the trackball extended above the surface of the console. Rotation of the trackball by a hand of the user is mechanically transferred to two rollers, the displacement of each being translated into electrical signals representing a displacement of the cursor in both the x and y directions. The trackballs employed in arcade machines are most often located skewed either to the right or left (typically the right) so that they can be operated by one hand while the other hand is used to activate select or fire buttons. Because arcade-style machines are large and thus stationary, the player is not required to maintain the position of the controls and is therefore free to operate the trackball with one hand and the selector or fire buttons with the other. The second broad category of pointing devices are those which do not respond to a literal displacement of the control feature of the pointing device, but rather respond to closure of a switch or the creation of a force by the control feature of the pointing device. In arcade-style video games, these pointing devices are often implemented as simple cursor control keys or joysticks. The cursor control keys are typically implemented in a cross-like formation such that when they are activated, left and right buttons cause the cursor to move in the left or right direction, respectively, and top and bottom buttons cause the cursor to move up and down respectively. The movement of the cursor is maintained at a particular rate as long as the button is depressed.

Switch-type joysticks operate much the same way as cursor control buttons. A single rocker plate closes one (and sometimes two) of four switches for up, right, down and left, the rocker plate being actuated through displacement of the joystick from its center neutral position. The video game machine then interprets the switch activation as a direction and rate of cursor movement just as with the cursor control buttons previously discussed. Sometimes the host system to which the pointing device is interfaced converts switch activation duration into an accelerated rate of movement of the cursor on the screen based on the elapsed time of the switch activated. Another type of non-displacement (i.e., non-Fitts' law) pointing device is called a rate joystick. Rate joysticks sense an implied force and convert the direction of the implied force into a direction and convert the magnitude of the applied force into a rate of movement of the cursor on the screen.

With the advent of computer systems that are dedicated to graphics applications or that employ graphical user interfaces (GUIs), a number of displacement devices are currently used as cursor control devices for such systems. Pointing devices such as mice, trackballs, track pads, and graphic tablets have become widespread for these applications. This is because graphics applications and GUIs, like video games, require the user to acquire a target such as an icon (or even a single pixel) on the screen using a cursor. The user must then select the target by activating a selector button. It is well- known in the art that humans have a much easier time mastering target acquisition using a relative reporting displacement control device to control cursor position than a device which causes cursor movement based on a rate, such as a digital switch-operated joystick where the rate of cursor movement is derived from a force on the stick or the position of the stick. Operators using non-displacement devices often suffer the frustrating problem of over or undershooting the target, particularly when the target is a single pixel.

Over the past several years, videogames have by and large moved from arcades into the home through the introduction of relatively inexpensive dedicated video game players and personal computers. Today, consumers are able to purchase a dedicated, microprocessor-based game-playing system which is capable of playing compatible video game cartridges using the picture tube of a standard television set for a display. Personal computers are easily adapted to play video games as they already have the necessary components, including a display screen, and the typical input control devices discussed above. Various input control devices have been designed for use with dedicated game players or to supplement conventional input control devices in the case of personal computers. These devices enable users to select modes, to control cursor position for acquiring targets, and to select or shoot targets acquired by one or more pointing devices of the input control device.

Nearly every commercially available input control device employs a switch operated joystick as a cursor control device, along with various numbers of mode and/or select and/or fire buttons. Joysticks have been favored because they can be operated while resting the input control device on a surface. Further, fire buttons can be incorporated into the joystick (such as at the tip of the joystick) so that the same hand that manipulates the joystick to acquire a target can also select or fire on the acquired target. This solves the problem of holding the input control device in one hand while the player acquires and selects /fires on targets with the other.

Despite the fact that a displacement device which is relative reporting (particularly a trackball) is the most optimal solution to the function of target acquisition, a function which is pervasive of video game playing, no one to Applicant's knowledge has overcome the difficulties of providing a solution for home video game players which employs such a device in an input control device intended for hand-held operation. This is true despite the fact that such devices had been previously provided in arcade-style games and also in the keyboards of some personal and laptop computers.

Thus, there is a need for a hand-held video game input device which provides the game player the option to operate a displacement device for purposes of controlling the cursor in the context of target acquisition, for scrolling the screen or for adjusting viewpoint, while permitting the user to operate firing buttons with the same hand as is used to operate the trackball, and still further freeing the other hand to operate other input control functions even while the input control device is not resting on a supporting surface.

SUMMARY OF THE INVENTION The present invention has application to multimedia and computer game programs executed by dedicated multimedia game systems and personal computers (i.e. host systems). A central processing unit (CPU) is coupled to a mass storage device, such as for example, a read-only memory game cartridge, CD-ROM, floppy disk or hard disk drive. The mass storage device stores an application game or multimedia program which is executed by the CPU. Execution of the application program results in the display of video images on video display device also coupled to the CPU. The video display device can be, for example, a cathode-ray tube (CRT) or a standard television set. Along with the video images, a character such as a cursor is generated on the screen for purposes of pointing to a particular portion of an image, the portion being as small as a pixel. An input control device is coupled to the host system and permits a user to control various aspects of the images displayed. The input control device comprises a displacement-type control device which can control the location of the cursor on the screen, the point-of-view presented by the image, or scrolling motion of the image in various directions. The input control device also has one or more buttons which can be programmed in accordance with a particular application program to perform various functions when actuated, such as identifying, selecting or firing upon a portion of the displayed image pointed to by the cursor or selecting different modes of operation.

In a first embodiment, the input control device has a body which has both a concave and a convex edge which taper at the ends to form the cusps of a crescent shape. The body has a hollow portion at or near the crests of the concave and convex edges. A trackball is employed as a displacement control device and is disposed within the hollow space so that a portion of the trackball extends up through the top surface of the body and is accessible to an operator. The operator rotates the trackball and the displacement resulting from the rotation is translated by a microcontroller, coupled to the trackball device and located within the hollow portion, into digital information representing the magnitude and direction of the displacement. The digital displacement information generated by the microcontroller is then transmitted to the host system over a databus coupling the microcontroller to the host system. The host system converts the digital displacement information into cursor location information, point-of-view information and/or scrolling information for driving the display. The first embodiment of the input control device also has two actuator buttons located at the crest of the convex edge, three actuator buttons located at the crest of the concave edge, four actuator buttons arranged in an arcuate pattern to the right of the trackball and four actuator buttons arranged in a cross pattern to the left of the trackball.

In a second embodiment, a track pad is employed as a displacement control device. The four actuator buttons to the right of the track pad are arranged in a diamond configuration. In both embodiments, the body is configured such that it can rest on the cusps of the crescent and an extension of the bottom of the body at the rear of the body to provide stable operation while resting on a flat surface in the manner of a tripod.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a view from above of a first preferred embodiment of the invention.

Figure 2 is a view of the bottom of the first preferred embodiment of the invention.

Figure 3 is a view from under the front of the first preferred embodiment of the invention. Figure 4 is a cross-sectional view from the side of the first preferred embodiment of the invention.

Figure 5 is a profile view of the first preferred embodiment of the invention.

Figure 6 is a top view of a second preferred embodiment of the invention.

Figure 7 is a block diagram representation of the control circuitry embodied in the apparatus of the present invention. DETAILED DESCRIPTION OF THE INVENTION The first embodiment of the apparatus of the present invention will now be described in detail with reference to Figures 1 through 5. Figure 1 shows a view from above the video game input controller 10 of the present invention. Controller 10 has a body which is crescent-shaped, formed by a top portion 23 and a bottom portion 31 (Figures 2 and 3). The body of input controller 10 is preferably made of high impact ABS plastic, but other suitable materials may also be used. The crescent-shaped body has a convex edge 13 and a concave edge 15. The convex 13 and concave 15 edges crest together at the center of the body and taper to form the two cusps 26 and 28. The body of input controller 10 is formed such that there is a hollow space 70 (Figure 4) between the top piece 23 and the bottom piece 31, the depth of which is greatest between the crests of the convex 13 and concave 15 edges.

A trackball 14 is centrally disposed in the body such that it slightly protrudes through top portion 23 for access by an operator. The trackball 14 is rotatably secured in a trackball assembly (99, Figure 7), which is securely disposed inside the hollow space 70 of the body. Embedded in the surface of top piece 23 and to the left of centrally located trackball 14 is a cross-configured set of four actuator buttons 12. The four actuator buttons 12 are preferably coupled to a rocker plate, the rocker plate in turn capable of activating one or more switches depending upon which of the buttons 12 are actuated. On the right side of centrally located trackball 14 is a set of four arcuately configured actuator buttons 16, also which preferably activate switches. The actuator buttons 16 are preferably coded by raised dots 17, to aid the operator in identifying the specific buttons through touch. Actuator buttons 16 can also be visually coded using distinct colors or symbols for each button. As illustrated in Figure 4, three actuator buttons 18 are mounted in the surface of the body of the input controller 10 along the concave edge 15 of the crescent. As illustrated in Figure 2, two actuator buttons 20 are mounted along the convex edge 13 of the crescent-shaped body of input controller 10. Those of skill in the art will recognize that any number of switch types can be employed for actuation by the various buttons. For example, the buttons can be made to produce a distinctive click when engaged to provide both audible and tactile feedback to the operator. A microcontroller circuit (90, Figure 7) along with the connectors necessary to interface the trackball assembly 99, buttons 16, buttons 18, buttons 20 and buttons 12 to the microcontroller circuit 90 are securely housed in the hollow space 70 of the body as illustrated in Figure 4. Cable 24 interfaces input controller 10 to a dedicated game-playing system or personal computer (not shown). Cable strain relief 21 receives and secures cable 24 to the body of input controller 10. Cable 24 then interfaces to an output port of microcontroller circuit 90 within input controller 10. Details of the trackball assembly 99 and microcontroller circuit 90 will be presented in more detail below.

An area of bottom portion 31 extends downwardly to produce the hollow space 70, and forms a stabilizing point 22 that creates a tripod-like arrangement with cusps 26, 28. Thus, the input controller 10 can be supported by a flat surface in a stable manner, with cusps 26, 28 and stabilizing point 22 in contact with the flat surface.

Figure 6 illustrates a second preferred embodiment 50 of the invention. In the second preferred embodiment, a track pad 54 is employed in place of trackball 14. The track pad is securely embedded in the surface of top portion 63 in a location which is central to the controller 10 analagously to trackball 14 in the first preferred embodiment. Details of the trackpad 54 will be presented in greater detail below. Also in this embodiment, actuator buttons 56 have been arranged in a diamond shape. Additionally, actuator buttons 60 have been made larger and more elongate.

Figure 7 illustrates the interconnection of the trackball assembly 99 and buttons 18/58, buttons 16/56, buttons 12/52 and buttons 20/60, to microcontroller 90. Also illustrated is a track pad assembly 100 associated with the second preferred embodiment and which can be substituted for trackball assembly 99.

Trackball assembly 99 can be any implementation of a Fitts' law trackball device. The embodiment illustrated in Figure 7 shows trackball 14 in mechanical contact with X and Y rollers 91 and 93 respectively. The physical displacement of rollers 91 and 93 is transformed by X and Y encoding circuits 97 and 95 respectively into electrical signals which are proportionate to the displacement of the rollers caused by the rotation of trackball 14. Output of encoding circuits 97 and 95 are then input into circuitry programmed to perform X & Y RESOLVING AND ENCODING represented by Block 92. The programmed circuitry of Block 92 converts the electrical signals provided by encoding circuits 97 and 95 into binary data representing the displacement of the trackball 14 since the last or previous report. Track pad assembly 100 is similar to trackball assembly 99. Preferably, the track 54 pad senses disturbances in an array of elements which are sensitive to changes in capacitance created by contact or proximity between its surface and an electrically conductive object, for example, the operator's finger. The change in capacitance of element(s) of the track pad is sensed and transformed by X an Y encoding circuits 107 and 107 respectively into electrical signals representing position in the X and Y directions of two-dimensional space. These signals are, like those of the trackball assembly 99, input into X and Y RESOLVING AND ENCODING CIRCUITRY 92 and encoded as binary data representing change in X and Y positions since the previous report. Those of skill in the art will recognize that any number of variations of the trackball assembly 99 and track pad assembly 100 can be made without departing from the intended scope of the invention. For example, the trackball 14 can have different diameters or be constructed with either a tacky or smooth surface. Track pad 54 can have varying dimensions and the mode by which contact location is sensed and determined can be of any known implementation.

Each of the buttons of input controller 10 actuate switches which are coupled to circuitry programmed to perform the function of BUTTONS STATE DETECΗON represented by Block 96. The programmed circuitry represented by Block 96 senses activation of one or more of the switches by the operator actuating buttons to which the switches are coupled. The switches coupled to buttons can be directly connected as shown in Figure 7, or they can be connected using a grid or matrix configuration. Circuitry programmed to perform the function of CENTRAL CONTROL represented by Block 94 receives the outputs from Blocks 92 and 96 and provides that information to circuitry programmed to perform the function of COMMUNICAΗON represented by Block 98. The programmed circuitry represented by Block 98 implements the communication protocol between input controller 10 and the host system.

The communication protocol used in the preferred embodiment is known as the Apple Desktop Bus (ADB). The binary data representing displacement of the trackball 14 (or representing the change in contact position of the finger on the track pad 54) is transmitted over the ADB to the host system where it is converted into a new cursor location on the video screen, a scrolling motion or a shift in viewpoint of the displayed image. Likewise, status data denoting which of the buttons (if any) are actuated (i.e. which switches are activated) is also transmitted to the host system and processed in accordance with the function assigned to the buttons by the program. The specification of the ADB protocol is available from Apple Computer Corp., Cupertino, California. Those of skill in the art will recognize that any number of communications protocols are available which will suffice for the intended purpose without exceeding the intended scope of the invention. Other such protocols include the Universal Serial Bus developed by Intel Corporation and the access.bus developed by Digital Equipment Corporation.

Microcontroller circuit 90 is preferably a PIC16C57 EPROM-Based Microcontroller available from Microchip Technology, Inc., Chandler,

Arizona. Those of skill in the art will recognize that the functions performed by microcontroller circuit 90 are quite similar to well-known techniques for response control devices, and that such functions can be accomplished using a variety of commercially available embedded controller solutions. For a detailed background concerning the 16C57, reference can be made to the

Microchip Data Book, 1994 Edition, published by Microchip Technology, Inc. and which is incorporated herein by this reference.

The application program being executed by the host system defines the purpose of each of the actuator buttons. In the preferred embodiment, buttons 18/58 are programmed to select various modes of operation, buttons 12/52 are programmed to control cursor location, possibly to scroll the displayed image and to change the point-of-view of the displayed image based on non- displacement principles as an alternative non-fitts' law, to the trackball or track pad; buttons 16/56 are programmed to select specific portions of the image (as small as a pixel) pointed to by the cursor; and buttons 20/60 are typically programmed for selecting or firing upon images or pixels of images pointed to by the cursor.

The unique design of input controller 10 permits an operator ambidextrous access to the trackball 14 or track pad 54 while holding the input controller 10 at one of the cusps 26/66 or 28/68 with the other hand. Further, the operator can operate the trackball with the thumb of the same hand as the fingers of which can be used to actuate the firing buttons 20/60. Finally, the input controller 10 permits an operator to brace the input controller 10 against the operator's body so that, for example, the operator can use the thumb of his or her left hand to operate the trackball 14, the index and middle fingers of his or her left hand to actuate firing buttons 20/60, while freeing the right hand to actuate selector buttons 16/56. The crescent shape of the body and the location of the trackball 14 (or track pad 54) facilitate the operator's simultaneous access to the trackball 14 (or track pad 54) and the various buttons of the input controller 10. SUMMARY As described in this Specification, the present invention may be implemented in one of a number of embodiments, each of which provides a Fitts' law (i.e. displacement) cursor control device in the form of trackball or track pad to manipulate the position of a cursor displayed on the video screen, or to control scrolling or point-of-view functions for images displayed on the video screen, of a host system being used to execute a multimedia or computer game application by either a dedicated game player, or alternatively, a personal or other digital computer. It will further be appreciated that the embodiments illustrated herein may be used to control a displayed cursor or images for any number of purposes, including pointing to targets such as portions of images, acquiring or selecting the target images to which the cursor is pointing, moving or dragging the selected target images or portions of images, firing upon the targets, scrolling the images left, right, up or down, and /or adjusting the point-of-view of the image. Those of skill in the art will recognize that the use of the term "acquiring" in no way implies simply targets to be fired upon, but rather is intended to mean any portion of any image to which the cursor is directed, including those to be selected. Further, the act of acquiring is intended to include simply pointing (e.g. overlapping) the target image or pixel, as well as actual selection of the target.

Although the present invention has been described with reference to a few exemplary Figures 1 through 7, it will be apparent that many alternatives, modifications and variations may be made in light of the foregoing description. For example, the body of the input controller can be comprised of one or several pieces. Further, the crescent-shape can be modified to shorten or lengthen the cusps, to increase a decrease the arcs of the curve or even to bend the cusps to be substantially perpendicular to the curved section.

Claims

I Claim:

1. An input control device for a host system, said input control device comprising a host system, including a host processor connected to a display device, a body having a top, bottom, front, back, and left and right sides, said body designed to be capable of contacting a user's right and left hands simultaneously, a displacement control means positioned in said body so that a user can conveniently reach the displacement control means with either said right or said left hand, and a connection between said displacement control means and said host system whereby displacement input information may be communicated to said host system.

2. The input control device of claim 27 wherein said displacement control means is located approximately equidistant between said right side and said left side of said body.

3. The input control device of claim 28 further comprising a position indicia on said display device, control circuitry for said position indicia, a circuit to translate control input of the displacement control means into signals or commands which can be used by said control circuitry for said position indicia to modify the position of said position indicia on said display device in accordance with said control input.

4. A method of controlling a position indicia on a display system, said method comprising providing a host system, including a host processor connected to a display device, with a position indicia on said display system, providing a body having a top, bottom, front, back, and left and right sides, said body designed to be capable of contacting a user's right and left hands simultaneously, providing a displacement control means positioned in said body so that a user can conveniently reach the displacement control means with either said right or said left hand, providing a connection between said displacement control means and said host system whereby displacement input information may be communicated to said host system, moving said displacement control means and, in a specified relationship thereto, controlling the position of said position indicia on said display system.

5. The method of controlling a position indicia of claim 30 wherein said displacement control means is located approximately equidistant between said right side and said left side of said body.

6. The method of controlling a position indicia of claim 31 further comprising a user providing input to said displacement control means by using the user's right hand.