US20040085368A1 - Method and apparatus for providing visual feedback during manipulation of text on a computer screen - Google Patents
Method and apparatus for providing visual feedback during manipulation of text on a computer screen Download PDFInfo
- Publication number
- US20040085368A1 US20040085368A1 US10/697,845 US69784503A US2004085368A1 US 20040085368 A1 US20040085368 A1 US 20040085368A1 US 69784503 A US69784503 A US 69784503A US 2004085368 A1 US2004085368 A1 US 2004085368A1
- Authority
- US
- United States
- Prior art keywords
- text
- cursor
- display device
- selected text
- source location
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0486—Drag-and-drop
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/174—Form filling; Merging
Definitions
- the present invention pertains to a method and apparatus for providing user feedback in a computer system. More particularly, the present invention pertains to a method and apparatus for providing visual feedback to a computer user while manipulating text on a computer screen.
- Graphical user interfaces typically include a user interface window, more commonly referred to as a desktop window. While operating within the desktop window computer users are able to manipulate graphical objects, i.e. icons, using a cursor control device such as a mouse or trackball. Icons can be moved simply by pointing, clicking and dragging. During these operations the computer user receives visual feedback that enhances the feeling of physical control over the desktop window and the icons therein. For example, selected icons are highlighted and, while the mouse button is held down, the selected icon moves with the cursor.
- the present invention provides a method and apparatus providing visual feedback to a computer user while dragging selected text.
- the processing logic of the present invention creates a text object from the selected, highlighted text when the computer user initiates a drag. Simultaneously, the selected text is visually de-emphasized.
- the processing logic snaps the text object to the cursor so that the text object follows the cursor without obscuring the insertion point. The computer user is then free to move about text onscreen in search of an insertion point. Finally, when the computer user selects the insertion point, the selected text is visually zoomed from the source location to the insertion point. As a result, the computer user has a feeling of physical control while moving selected text within a document.
- FIG. 1 is an illustration of a computer system.
- FIG. 2 is a flow diagram of a method of providing visual feedback while directly manipulating text.
- FIG. 3A illustrates the selection of text.
- FIG. 3B illustrates the change of cursor caused by a mouse down event.
- FIG. 3C illustrates the placement of the text object relative to the cursor during a drag.
- FIG. 3D illustrates the tracking of the text object during a drag.
- FIG. 3E illustrates the placement of the text object after a mouse up event.
- FIG. 4A illustrates a bounding rectangle at a source location.
- FIG. 4B illustrates a bounding rectangle at a destination location.
- FIG. 4C illustrates a zoom from a bounding rectangle at a source location to a bounding rectangle at a destination location.
- FIG. 5 is a flow diagram of an idle loop for changing the shape of the cursor.
- FIG. 1 illustrates a computer system 10 in which the present processing logic of the invention operates.
- Computer system 10 is illustrated as an Apple MacintoshTM computer system, however, those of ordinary skill in the art will understand that alternative computer systems may be used.
- Computer system 10 includes a monitor 12 for visually displaying information to a computer user, a central processing unit (CPU), and internal memory, enclosed by housing 14 .
- Keyboard 16 allows a computer user to provide input to the CPU.
- Computer system 10 also includes a cursor control device, such as mouse 18 , or a trackball, joystick, or special keys capable of signaling movement in a particular direction.
- Computer system 10 also includes a signal generation device for signaling active and inactive control states.
- the signal generation device may be realized using a switch or button 19 on or near the cursor control device. The up position of button 19 signals the inactive control state while the down position of button 19 signals the active control state.
- Computer system 10 includes an operating system capable of tracking the position of mouse 18 and cursor 20 .
- the operating system also detects the state of the signal generation device and reports the position of cursor 20 in each state of the signal generation device.
- the processing logic of the present invention is stored within the internal memory of computer system 10 , or on other machine readable media, and executed by the CPU.
- the processing logic can be separately compiled or loaded entity or incorporated as part of a larger word processing system. In either case, the processing logic of the present invention may be activated using techniques well known to those of ordinary skill in the art.
- the processing logic of the present invention provides visual feedback to users of computer system 10 during direct manipulation of text within a word processing application. That visual feedback includes de-emphasizing selected text at a source location, creating a text object resembling the selected text and “snapping” the text object to the pointer such that the text object travels with the pointer. Once a destination location is chosen, the selected text is visually “zoomed” from the source location to the destination location.
- FIG. 2 illustrates in flow diagram form the operation of the processing logic of the present invention.
- FIG. 3 illustrates the effect of the processing logic upon selected text 30 in relation to displayed text 32 .
- the processing logic performs two major tasks: determining whether visual feedback is required and providing that visual feedback.
- steps 50 - 58 the processing logic determines whether visual feedback is required.
- the computer user indicates the desire for visual feedback by pressing mouse button 19 down while cursor 20 is over selected text 30 , continuing to hold down mouse button 19 and moving cursor 20 from its position at the time of the mouse down event.
- the processing logic generates visual feedback during the manipulation of selected text 30 in steps 60 - 90 .
- the processing logic generates visual feedback for the computer user until after mouse button 19 is released; i.e., returns to the up position and the inactive state.
- Visual feedback while mouse button 19 is down includes generation of a text object, snapping the text object to the cursor, flashing the insertion caret at location under cursor and moving the text object with the cursor. Once the computer user releases mouse button 19 it returns to its up position, selected text 30 is visually zoomed from the source location to the destination location.
- the processing logic is not activated until after text has been selected from the text displayed on monitor 12 .
- selected text 30 “Once upon a time,” is indicated by reverse video, which distinguishes it from text 32 displayed on monitor 12 .
- step 50 the processing logic waits in step 50 for a mouse down event.
- the processing logic is also informed of the cursor position at the time of the mouse down event. Let us refer to this cursor position as the “mouse down position.” Afterward, the processing logic advances to step 52 .
- step 52 the processing logic determines whether the computer user may require visual feedback to accompany the manipulation of selected text 30 . This determination is made by comparing the mouse down position to the coordinates of selected text 30 . The user does not require visual feedback for text manipulation if the computer user pressed mouse button 19 down while cursor 20 was not over selected text 30 . In that case, the processing logic branches to step 54 to handle the mouse down event in a conventional and appropriate manner. On the other hand, visual feedback may be required if the mouse down position was over selected text 30 . The processing logic advances to step 56 to determine if so.
- step 56 the processing logic waits to see if the computer user requires visual feedback to accompany manipulation of selected text.
- the computer user does not require visual feedback if a mouse up event occurs before cursor 20 is moved from the mouse down position. Accordingly, the processing logic branches to step 54 to handle the mouse up as if a mouse down event occurred at the original mouse down location. However, in the absence of a mouse up event, the processing logic branches to step 58 .
- step 58 the processing logic determines the current position of cursor 20 . Let us refer to this position as the “current cursor position.” If the current cursor position is some arbitrary distance away from the mouse down position the computer user has indicated a desire to move, or pick-up, selected text 30 . In one embodiment, the current cursor position is chosen to be three pixels away from the mouse down position to indicate a desire to manipulate selected text 30 . The distance chosen may vary according to needs of the computer user.
- the processing logic responds to movement of cursor 20 while mouse button 19 is still down by branching to step 60 . On the other hand, if the cursor current position is still very close to the mouse down position, then the processing logic continues to await indication of whether or not visual feedback is desired by branching back up to step 56 .
- step 60 the processing logic begins providing visual feedback to the computer user by de-emphasizing selected text 30 .
- Selected text 30 can be de-emphasized in a number of ways that will be apparent to those skilled in the art.
- de-emphasis of selected text 30 is performed using a QuickDrawTM brand graphics processing routine, which transforms the white areas surrounding the text to light gray and black letters of the text to medium gray.
- FIG. 3C illustrates de-emphasized selected text 30 a according to this scheme. De-emphasis of selected text 30 achieved, the processing logic advances to step 62 .
- step 62 the processing logic creates text object 34 from selected text 30 .
- text object refers to a visual representation of selected text 30 , which may or may not include all of selected text 30 .
- Text object 34 need only include sufficient text to remind the computer user selected text 30 without unnecessarily obscuring much of the text 32 displayed on monitor 12 .
- Text object 34 may be created by generating a bit map of selected text 30 and limiting its size. Text object 34 may also be represented by a dotted outline of selected text 30 , similar to what computer users see when they drag icons within the desktop window. Such a representation of text object 34 requires less CPU time but does not inform the computer user of selected text 30 which is being moved.
- Text object 34 may also be emphasized as compared to displayed text 32 .
- text object 34 is emphasized using reverse video.
- Other methods of emphasizing text object 34 will be obvious to those skilled in the art. Therefore, the manner of creating and emphasizing text object 34 will not be described in detail herein.
- the processing logic branches to step 64 after its creation of text object 34 .
- the processing logic displays text object 34 at the source location during step 64 .
- source location refers to the initial location of selected text 30 . Stated another way, “source location” refers to the location that is the source of selected text 30 .
- text object 34 Because it is initially displayed at the source location, text object 34 obscures de-emphasized selected text 30 if they are both the same size. Once text object 34 is moved it no longer obscures selected text 30 .
- the processing logic also displays insertion caret 36 near cursor 20 in step 64 .
- insertion caret refers to any visual representation that indicates to the computer user the precise location in a body of text where selected text 30 may be inserted. According to common word processing conventions, insertion caret 36 is visually represented by a thin vertical line.
- cursor 20 may be displayed as an arrow. Processing logic for doing so will be described in detail herein below.
- insertion caret 36 is preferably displayed near tip 20 a of cursor 20 , as shown in FIGS. 3C and 3E. Initial display of insertion caret 36 and text object 34 begun, the processing logic proceeds to step 66 .
- the processing logic begins the process of snapping text object 34 to cursor 20 in step 66 .
- the processing logic does this by initializing an interpolation factor, which is used to snap text object 34 to cursor 20 in a discrete and finite number of steps. Text object 34 is gradually moved to cursor 20 to prevent computer users from becoming disoriented by text object 34 appearing in an unfamiliar location. This done, the processing logic enters the snapping and tracking logic by branching to step 68 .
- the loop of steps 68 - 90 snaps text object 34 to cursor 20 and, once snapped, ensures that text object follows the movement of cursor 20 on monitor 12 .
- the first step within the loop is determining whether the interpolation factor has reached its maximum value of one. If it has not, text object 34 has not yet snapped to cursor 20 . To continue snapping, the processing logic branches to step 70 . On the other hand, if text object 34 has snapped to cursor 20 , as indicated by the maximum value of the interpolation factor, the processing logic advances to step 72 .
- step 70 the interpolation factor is incremented.
- the amount by which the interpolation factor is incremented depends upon the choice of the number of steps for text object 34 to reach cursor 20 . For example, if the number of steps chosen for text object 34 to reach cursor 20 is four, then the interpolation factor is incremented by 0.25 each pass through step 70 . The number of steps chosen will depend upon the visual effect and response time desired.
- the processing logic branches to step 72 .
- the processing logic moves text object 34 in step 72 to a location determined by the interpolation factor and the current distance between the source location and cursor 20 .
- the interpolation factor has not yet reached its maximum value, text object 34 will be moved toward cursor 20 along an imaginary straight line between the source location and cursor 20 to a locating determined by the distance multiplied by the interpolation factor.
- text object 34 is moved to a location determined by a weighted average of the source location and the destination location using the interpolation factor.
- the interpolation factor reaches its maximum value, text object 34 snaps to cursor 20 .
- text object 34 follows the movement of cursor 20 providing a visual impression that text object 34 is attached to cursor 20 . This, in turn, gives the computer user a feeling of physical control.
- text object 34 is snapped to the lower left hand of cursor 20 , as shown in FIGS. 3C and 3D. This location prevents text object 34 from obscuring insertion caret 36 and focuses the computer user's attention on insertion caret 36 and cursor 20 while maintaining the strong visual impression that text object 34 is attached to cursor 20 . Those skilled in the art will appreciate that other locations near or on cursor 20 may also convey the same visual impression.
- step 72 After moving text object in step 72 , the processing logic advances to step 74 . There the processing logic moves insertion caret 36 if cursor 20 has moved. Insertion caret 36 is moved to a location between the two characters nearest to cursor 20 . This done the processing logic advances to step 76 .
- step 76 the processing logic compares the current location of insertion caret 36 to its previous location. If insertion caret 36 has not moved from its previous location, the processing logic advances to step 78 . There insertion caret 36 is “flashed,” or blinked on and off, indicating an idle state. On the other hand, if insertion caret 36 changed its location during step 74 then the processing logic advances to step 80 .
- step 80 the processing logic determines whether the computer user has selected an insertion point, or destination location, for selected text 30 .
- destination location refers to the position of cursor 20 at the time of a mouse up event.
- the processing logic makes this determination by discovering whether a mouse up event has occurred.
- a mouse up event necessarily accompanies selection of a destination location or insertion point.
- the location of cursor 20 at the time of the mouse up event determines the insertion point for selected text 30 .
- the processing logic responds to a mouse up event by branching to step 82 .
- the absence of a mouse up event indicates that the computer user has not yet selected the destination location.
- the processing logic responds to this state of affairs by branching back to the start of the tracking and snapping loop, step 68 .
- step 82 the processing logic begins the process of indicating visually to the computer user the movement of selected text 30 from the source location to the destination location. This involves three steps. First, the processing logic calculates bounding rectangles for selected text 30 at both the source location and the destination location. Second, the processing logic zooms from the first bounding rectangle at the source location to the second bounding rectangle at the destination location. Finally, selected text 30 is displayed at the destination location in the third and final step.
- step 82 the processing logic computes a bounding rectangle for selected text 30 at the source location. This step, and the second, are illustrated in FIG. 4 using selected text 31 ; i.e., “Jack and Jill.” Bounding rectangle 100 is defined by two points of selected text 31 : the point furthest to the top left and the point furthest to the bottom right. The dimensions of bounding rectangle 100 computed, the processing logic proceeds to step 84 . These points are chosen because the selected text may span vertically more than one line.
- step 84 offscreen the processing logic moves selected text 31 to the destination location.
- computer 10 maintains in internal memory a copy of the text file that text displayed monitor 12 is part of. This copy is said to be “offscreen.” Thus, it will be understood that the activities of step 84 do not alter the text displayed on monitor 12 .
- the processing logic performs an intelligent paste, which is also commonly referred to as intelligent cut and paste.
- Intelligent cut and paste is a set of editing features that account for the need for spaces between words.
- FIGS. 4A and 4B illustrate how intelligent cut and paste performs. In FIG.
- insertion caret 102 is the space immediately following the word “the.” Without intelligent cut and paste, after inserting selected text 31 displayed text 33 would not include a space between “Jack” and “the” unless selected text 32 includes a space before “Jack.” Stated another way, without intelligent cut and paste, after inserting selected text 31 the sixth line 104 of displayed text 33 would read as “assumed theJack.” However, using intelligent cut and paste, the sixth line 104 includes a space between “the” and “Jack,” regardless of whether selected text 31 includes a space before “Jack.” Thus, using intelligent cut and paste sixth line 104 reads as “assumed the Jack,” as illustrated in FIG. 4B.
- the processing logic emphasizes selected text 31 .
- selected text 31 may be emphasized visually in a great many ways, including displaying selected text 31 in reverse video.
- selected text 31 is also removed from the source location during step 78 depends upon whether the computer user has requested a cut and paste or a copy and paste. If a cut and paste operation was selected text 31 is removed from the source location. Otherwise, selected text 31 remains at the source location, but it will no longer be de-emphasized.
- bounding rectangle 104 is defined by two points of selected text 31 : the point of text furthest to the top left and the text point furthest to the bottom right. Given insertion point 102 , the dimensions of bounding rectangle 104 may vary from those of bounding rectangle 100 . This is the case in FIG. 4B.
- step 88 the processing logic zooms from selected text 31 at the source location to selected text 31 at the destination location.
- Zooming is a well known visual effect that animates the movement of an object between two points on monitor 12 .
- Methods for animating zooms are well known and will not be described in detail herein.
- the dimensions of bounding rectangles 100 and 104 are provided to a zoom processor. Given these, the zoom starts with a bounding rectangle 100 displayed near the source location. Two or more rectangles are displayed on monitor 12 at a time, each successive rectangle more closely approaching the dimensions and final location of bounding rectangle 104 .
- FIG. 4C attempts to illustrate this idea.
- intervening rectangle 106 differ slightly from those bounding rectangle 100 , just starting to approach those of bounding rectangle 104 .
- intervening rectangle 108 differ from those of intervening rectangle 106 , approaching more closely the dimensions of bounding rectangle 104 .
- step 90 The processing logic advances to step 90 after the zoom is complete. There the processing logic displays onscreen selected text 31 at the destination location. Following as it does the zoom, the effect of step 78 is that the zoom drops selected text 31 at the destination location.
- step 92 The processing logic branches to step 92 from step 90 to await activation again.
- FIG. 5 illustrates in flow diagram form a method for doing so as a background process.
- the processing logic begins in step 110 by determining whether cursor 20 is over selected text 30 . If cursor 20 is not, the processing logic branches to step 112 . There cursor 20 is displayed as an I-beam. On the other hand, if cursor 20 is over selected text 30 , the processing logic advances to step 114 . During this step the processing logic displays cursor 20 as an arrow, indicating that cursor 20 is over a draggable item.
- the visual feedback includes creation of a text object from selected text, tracking the insertion point as user moves mouse and zooming the selected text from its source location to its destination location.
Abstract
A method and apparatus providing visual feedback to a computer user while dragging selected text is described. As its first step, the processing logic creates a text object from the selected text once the computer user has initiated a drag. Simultaneously, the selected text is visually de-emphasized. Secondly, the processing logic snaps the text object to the cursor so that the text object follows the cursor without obscuring text at the insertion point. Finally, when the computer user selects the final insertion point, the selected text is visually zoomed from the source location to the final insertion point.
Description
- The present invention pertains to a method and apparatus for providing user feedback in a computer system. More particularly, the present invention pertains to a method and apparatus for providing visual feedback to a computer user while manipulating text on a computer screen.
- Many computer systems include graphical user interfaces. Graphical user interfaces typically include a user interface window, more commonly referred to as a desktop window. While operating within the desktop window computer users are able to manipulate graphical objects, i.e. icons, using a cursor control device such as a mouse or trackball. Icons can be moved simply by pointing, clicking and dragging. During these operations the computer user receives visual feedback that enhances the feeling of physical control over the desktop window and the icons therein. For example, selected icons are highlighted and, while the mouse button is held down, the selected icon moves with the cursor.
- Moving text within word processor files is not as easy. Highlighted text cannot be dragged about a document as if it were an icon. There is no way to “grab” the highlighted text. As a result, computer users do not have a feeling of physical control during cut, copy and paste operations.
- The present invention provides a method and apparatus providing visual feedback to a computer user while dragging selected text. As its first step, the processing logic of the present invention creates a text object from the selected, highlighted text when the computer user initiates a drag. Simultaneously, the selected text is visually de-emphasized. Secondly, after the cursor begins to move, the processing logic snaps the text object to the cursor so that the text object follows the cursor without obscuring the insertion point. The computer user is then free to move about text onscreen in search of an insertion point. Finally, when the computer user selects the insertion point, the selected text is visually zoomed from the source location to the insertion point. As a result, the computer user has a feeling of physical control while moving selected text within a document.
- Other objects, features, and advantages of the present invention will be apparent from the accompanying drawings and the detailed description that follows.
- The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings in which references indicate similar elements and in which:
- FIG. 1 is an illustration of a computer system.
- FIG. 2 is a flow diagram of a method of providing visual feedback while directly manipulating text.
- FIG. 3A illustrates the selection of text.
- FIG. 3B illustrates the change of cursor caused by a mouse down event.
- FIG. 3C illustrates the placement of the text object relative to the cursor during a drag.
- FIG. 3D illustrates the tracking of the text object during a drag.
- FIG. 3E illustrates the placement of the text object after a mouse up event.
- FIG. 4A illustrates a bounding rectangle at a source location.
- FIG. 4B illustrates a bounding rectangle at a destination location.
- FIG. 4C illustrates a zoom from a bounding rectangle at a source location to a bounding rectangle at a destination location.
- FIG. 5 is a flow diagram of an idle loop for changing the shape of the cursor.
- FIG. 1 illustrates a
computer system 10 in which the present processing logic of the invention operates.Computer system 10 is illustrated as an Apple Macintosh™ computer system, however, those of ordinary skill in the art will understand that alternative computer systems may be used.Computer system 10 includes amonitor 12 for visually displaying information to a computer user, a central processing unit (CPU), and internal memory, enclosed byhousing 14. Keyboard 16 allows a computer user to provide input to the CPU.Computer system 10 also includes a cursor control device, such asmouse 18, or a trackball, joystick, or special keys capable of signaling movement in a particular direction. By movingmouse 18 the computer user is able to control the movement of the two or three dimensional movement of a visible symbol, such ascursor 20, onmonitor 12.Computer system 10 also includes a signal generation device for signaling active and inactive control states. The signal generation device may be realized using a switch orbutton 19 on or near the cursor control device. The up position ofbutton 19 signals the inactive control state while the down position ofbutton 19 signals the active control state. -
Computer system 10 includes an operating system capable of tracking the position ofmouse 18 andcursor 20. The operating system also detects the state of the signal generation device and reports the position ofcursor 20 in each state of the signal generation device. These features of an operating system are well known in the art. Therefore, they will not be described in detail here. - The processing logic of the present invention is stored within the internal memory of
computer system 10, or on other machine readable media, and executed by the CPU. The processing logic can be separately compiled or loaded entity or incorporated as part of a larger word processing system. In either case, the processing logic of the present invention may be activated using techniques well known to those of ordinary skill in the art. - Briefly described, the processing logic of the present invention provides visual feedback to users of
computer system 10 during direct manipulation of text within a word processing application. That visual feedback includes de-emphasizing selected text at a source location, creating a text object resembling the selected text and “snapping” the text object to the pointer such that the text object travels with the pointer. Once a destination location is chosen, the selected text is visually “zoomed” from the source location to the destination location. - FIG. 2 illustrates in flow diagram form the operation of the processing logic of the present invention. FIG. 3 illustrates the effect of the processing logic upon selected
text 30 in relation to displayedtext 32. - Referring now to FIG. 2, the processing logic performs two major tasks: determining whether visual feedback is required and providing that visual feedback. In steps50-58 the processing logic determines whether visual feedback is required. The computer user indicates the desire for visual feedback by pressing
mouse button 19 down whilecursor 20 is over selectedtext 30, continuing to hold downmouse button 19 and movingcursor 20 from its position at the time of the mouse down event. The processing logic generates visual feedback during the manipulation of selectedtext 30 in steps 60-90. The processing logic generates visual feedback for the computer user until aftermouse button 19 is released; i.e., returns to the up position and the inactive state. Visual feedback whilemouse button 19 is down includes generation of a text object, snapping the text object to the cursor, flashing the insertion caret at location under cursor and moving the text object with the cursor. Once the computer user releasesmouse button 19 it returns to its up position, selectedtext 30 is visually zoomed from the source location to the destination location. - The processing logic is not activated until after text has been selected from the text displayed on
monitor 12. In FIG. 3A, selectedtext 30, “Once upon a time,” is indicated by reverse video, which distinguishes it fromtext 32 displayed onmonitor 12. - After
text 30 has been selected, the processing logic waits instep 50 for a mouse down event. When the mouse down event occurs, the processing logic is also informed of the cursor position at the time of the mouse down event. Let us refer to this cursor position as the “mouse down position.” Afterward, the processing logic advances to step 52. - In
step 52 the processing logic determines whether the computer user may require visual feedback to accompany the manipulation of selectedtext 30. This determination is made by comparing the mouse down position to the coordinates of selectedtext 30. The user does not require visual feedback for text manipulation if the computer user pressedmouse button 19 down whilecursor 20 was not over selectedtext 30. In that case, the processing logic branches to step 54 to handle the mouse down event in a conventional and appropriate manner. On the other hand, visual feedback may be required if the mouse down position was over selectedtext 30. The processing logic advances to step 56 to determine if so. - In
step 56 the processing logic waits to see if the computer user requires visual feedback to accompany manipulation of selected text. The computer user does not require visual feedback if a mouse up event occurs beforecursor 20 is moved from the mouse down position. Accordingly, the processing logic branches to step 54 to handle the mouse up as if a mouse down event occurred at the original mouse down location. However, in the absence of a mouse up event, the processing logic branches to step 58. - In
step 58 the processing logic determines the current position ofcursor 20. Let us refer to this position as the “current cursor position.” If the current cursor position is some arbitrary distance away from the mouse down position the computer user has indicated a desire to move, or pick-up, selectedtext 30. In one embodiment, the current cursor position is chosen to be three pixels away from the mouse down position to indicate a desire to manipulate selectedtext 30. The distance chosen may vary according to needs of the computer user. The processing logic responds to movement ofcursor 20 whilemouse button 19 is still down by branching to step 60. On the other hand, if the cursor current position is still very close to the mouse down position, then the processing logic continues to await indication of whether or not visual feedback is desired by branching back up tostep 56. - With
step 60 the processing logic begins providing visual feedback to the computer user by de-emphasizing selectedtext 30. Selectedtext 30 can be de-emphasized in a number of ways that will be apparent to those skilled in the art. In one embodiment, de-emphasis of selectedtext 30 is performed using a QuickDraw™ brand graphics processing routine, which transforms the white areas surrounding the text to light gray and black letters of the text to medium gray. FIG. 3C illustrates de-emphasized selected text 30 a according to this scheme. De-emphasis of selectedtext 30 achieved, the processing logic advances to step 62. - In
step 62 the processing logic createstext object 34 from selectedtext 30. As used herein “text object” refers to a visual representation of selectedtext 30, which may or may not include all of selectedtext 30.Text object 34 need only include sufficient text to remind the computer user selectedtext 30 without unnecessarily obscuring much of thetext 32 displayed onmonitor 12.Text object 34 may be created by generating a bit map of selectedtext 30 and limiting its size.Text object 34 may also be represented by a dotted outline of selectedtext 30, similar to what computer users see when they drag icons within the desktop window. Such a representation oftext object 34 requires less CPU time but does not inform the computer user of selectedtext 30 which is being moved. -
Text object 34 may also be emphasized as compared to displayedtext 32. In one embodiment,text object 34 is emphasized using reverse video. Other methods of emphasizingtext object 34 will be obvious to those skilled in the art. Therefore, the manner of creating and emphasizingtext object 34 will not be described in detail herein. The processing logic branches to step 64 after its creation oftext object 34. - The processing logic
displays text object 34 at the source location duringstep 64. As used herein, “source location” refers to the initial location of selectedtext 30. Stated another way, “source location” refers to the location that is the source of selectedtext 30. - Because it is initially displayed at the source location,
text object 34 obscures de-emphasized selectedtext 30 if they are both the same size. Oncetext object 34 is moved it no longer obscures selectedtext 30. The processing logic also displaysinsertion caret 36 nearcursor 20 instep 64. As using herein “insertion caret” refers to any visual representation that indicates to the computer user the precise location in a body of text where selectedtext 30 may be inserted. According to common word processing conventions,insertion caret 36 is visually represented by a thin vertical line. - To provide further visual emphasis that the computer user is able to manipulate selected
text 30,cursor 20 may be displayed as an arrow. Processing logic for doing so will be described in detail herein below. When cursor 20 is displayed as an arrow,insertion caret 36 is preferably displayed neartip 20 a ofcursor 20, as shown in FIGS. 3C and 3E. Initial display ofinsertion caret 36 and text object 34 begun, the processing logic proceeds to step 66. - The processing logic begins the process of snapping
text object 34 tocursor 20 instep 66. The processing logic does this by initializing an interpolation factor, which is used to snaptext object 34 tocursor 20 in a discrete and finite number of steps.Text object 34 is gradually moved tocursor 20 to prevent computer users from becoming disoriented bytext object 34 appearing in an unfamiliar location. This done, the processing logic enters the snapping and tracking logic by branching to step 68. - The loop of steps68-90
snaps text object 34 tocursor 20 and, once snapped, ensures that text object follows the movement ofcursor 20 onmonitor 12. The first step within the loop is determining whether the interpolation factor has reached its maximum value of one. If it has not,text object 34 has not yet snapped tocursor 20. To continue snapping, the processing logic branches to step 70. On the other hand, if text object 34 has snapped tocursor 20, as indicated by the maximum value of the interpolation factor, the processing logic advances to step 72. - During
step 70 the interpolation factor is incremented. The amount by which the interpolation factor is incremented depends upon the choice of the number of steps fortext object 34 to reachcursor 20. For example, if the number of steps chosen fortext object 34 to reachcursor 20 is four, then the interpolation factor is incremented by 0.25 each pass throughstep 70. The number of steps chosen will depend upon the visual effect and response time desired. After incrementing the interpolation factor, the processing logic branches to step 72. - The processing logic moves
text object 34 instep 72 to a location determined by the interpolation factor and the current distance between the source location andcursor 20. Thus, if the interpolation factor has not yet reached its maximum value,text object 34 will be moved towardcursor 20 along an imaginary straight line between the source location andcursor 20 to a locating determined by the distance multiplied by the interpolation factor. Stated another way, duringstep 72text object 34 is moved to a location determined by a weighted average of the source location and the destination location using the interpolation factor. When the interpolation factor reaches its maximum value, text object 34 snaps tocursor 20. Thereafter,text object 34 follows the movement ofcursor 20 providing a visual impression thattext object 34 is attached tocursor 20. This, in turn, gives the computer user a feeling of physical control. - In one embodiment,
text object 34 is snapped to the lower left hand ofcursor 20, as shown in FIGS. 3C and 3D. This location preventstext object 34 from obscuringinsertion caret 36 and focuses the computer user's attention oninsertion caret 36 andcursor 20 while maintaining the strong visual impression thattext object 34 is attached tocursor 20. Those skilled in the art will appreciate that other locations near or oncursor 20 may also convey the same visual impression. - After moving text object in
step 72, the processing logic advances to step 74. There the processing logic movesinsertion caret 36 ifcursor 20 has moved.Insertion caret 36 is moved to a location between the two characters nearest tocursor 20. This done the processing logic advances to step 76. - During
step 76 the processing logic compares the current location ofinsertion caret 36 to its previous location. Ifinsertion caret 36 has not moved from its previous location, the processing logic advances to step 78. Thereinsertion caret 36 is “flashed,” or blinked on and off, indicating an idle state. On the other hand, ifinsertion caret 36 changed its location duringstep 74 then the processing logic advances to step 80. - In
step 80 the processing logic determines whether the computer user has selected an insertion point, or destination location, for selectedtext 30. (As used herein “destination location” refers to the position ofcursor 20 at the time of a mouse up event.) The processing logic makes this determination by discovering whether a mouse up event has occurred. A mouse up event necessarily accompanies selection of a destination location or insertion point. The location ofcursor 20 at the time of the mouse up event determines the insertion point for selectedtext 30. The processing logic responds to a mouse up event by branching to step 82. Conversely, the absence of a mouse up event indicates that the computer user has not yet selected the destination location. The processing logic responds to this state of affairs by branching back to the start of the tracking and snapping loop,step 68. - After the mouse up event, with
step 82 the processing logic begins the process of indicating visually to the computer user the movement of selectedtext 30 from the source location to the destination location. This involves three steps. First, the processing logic calculates bounding rectangles for selectedtext 30 at both the source location and the destination location. Second, the processing logic zooms from the first bounding rectangle at the source location to the second bounding rectangle at the destination location. Finally, selectedtext 30 is displayed at the destination location in the third and final step. - In
step 82 the processing logic computes a bounding rectangle for selectedtext 30 at the source location. This step, and the second, are illustrated in FIG. 4 using selectedtext 31; i.e., “Jack and Jill.” Boundingrectangle 100 is defined by two points of selected text 31: the point furthest to the top left and the point furthest to the bottom right. The dimensions of boundingrectangle 100 computed, the processing logic proceeds to step 84. These points are chosen because the selected text may span vertically more than one line. - In
step 84, offscreen the processing logic moves selectedtext 31 to the destination location. As understood by those skilled in the art,computer 10 maintains in internal memory a copy of the text file that text displayedmonitor 12 is part of. This copy is said to be “offscreen.” Thus, it will be understood that the activities ofstep 84 do not alter the text displayed onmonitor 12. - In moving selected
text 31 to the destination location instep 84, the processing logic performs an intelligent paste, which is also commonly referred to as intelligent cut and paste. Intelligent cut and paste is a set of editing features that account for the need for spaces between words. FIGS. 4A and 4B illustrate how intelligent cut and paste performs. In FIG. 4A,insertion caret 102 is the space immediately following the word “the.” Without intelligent cut and paste, after inserting selectedtext 31 displayedtext 33 would not include a space between “Jack” and “the” unless selectedtext 32 includes a space before “Jack.” Stated another way, without intelligent cut and paste, after inserting selectedtext 31 thesixth line 104 of displayedtext 33 would read as “assumed theJack.” However, using intelligent cut and paste, thesixth line 104 includes a space between “the” and “Jack,” regardless of whether selectedtext 31 includes a space before “Jack.” Thus, using intelligent cut and pastesixth line 104 reads as “assumed the Jack,” as illustrated in FIG. 4B. - After moving selected
text 31 to the destination location, the processing logic emphasizes selectedtext 31. As a result when the offscreen copy of the file is finally displayed, the computer user's attention will be called to selectedtext 31 at the destination location. Selectedtext 31 may be emphasized visually in a great many ways, including displaying selectedtext 31 in reverse video. - Whether selected
text 31 is also removed from the source location duringstep 78 depends upon whether the computer user has requested a cut and paste or a copy and paste. If a cut and paste operation was selectedtext 31 is removed from the source location. Otherwise, selectedtext 31 remains at the source location, but it will no longer be de-emphasized. - The processing logic proceeds to step86 from
step 84. Having moved selectedtext 31 to the destination location, the processing logic now computes bounding rectangle for selectedtext 31 at the destination location. Like boundingrectangle 100, boundingrectangle 104 is defined by two points of selected text 31: the point of text furthest to the top left and the text point furthest to the bottom right. Giveninsertion point 102, the dimensions of boundingrectangle 104 may vary from those of boundingrectangle 100. This is the case in FIG. 4B. - Preparations for zooming complete, the processing logic branches to step88 from
step 86. Instep 88, the processing logic zooms from selectedtext 31 at the source location to selectedtext 31 at the destination location. Zooming is a well known visual effect that animates the movement of an object between two points onmonitor 12. Methods for animating zooms are well known and will not be described in detail herein. The dimensions of boundingrectangles rectangle 100 displayed near the source location. Two or more rectangles are displayed onmonitor 12 at a time, each successive rectangle more closely approaching the dimensions and final location of boundingrectangle 104. FIG. 4C attempts to illustrate this idea. The dimensions of interveningrectangle 106 differ slightly from those boundingrectangle 100, just starting to approach those of boundingrectangle 104. Similarly, the dimensions of interveningrectangle 108 differ from those of interveningrectangle 106, approaching more closely the dimensions of boundingrectangle 104. The zooms ends with the display of boundingrectangle 104 near the destination location. - The processing logic advances to step90 after the zoom is complete. There the processing logic displays onscreen selected
text 31 at the destination location. Following as it does the zoom, the effect ofstep 78 is that the zoom drops selectedtext 31 at the destination location. - The processing logic branches to step92 from
step 90 to await activation again. - Additional visual feedback may be provided to the computer user to emphasize that selected
text 30 may be dragged within a window, or between windows. Displayingcursor 20 as an arrow, rather than an I-beam, whilecursor 20 is over selectedtext 30 emphasizes this fact. FIG. 5 illustrates in flow diagram form a method for doing so as a background process. - The processing logic begins in
step 110 by determining whethercursor 20 is over selectedtext 30. Ifcursor 20 is not, the processing logic branches to step 112. There cursor 20 is displayed as an I-beam. On the other hand, ifcursor 20 is over selectedtext 30, the processing logic advances to step 114. During this step the processing logic displays cursor 20 as an arrow, indicating thatcursor 20 is over a draggable item. - Thus, a method and apparatus for providing visual feedback to a computer user while manipulating text has been described. The visual feedback includes creation of a text object from selected text, tracking the insertion point as user moves mouse and zooming the selected text from its source location to its destination location.
- In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
Claims (19)
1. A computer implemented method of providing visual feedback to a computer user during manipulation of selected text on a display device of a computer system, the computer system including a cursor control device for interactively positioning a cursor and an insertion caret on the display device, the computer also having a signal generation device for signaling an active state and an inactive state, the method comprising the computer implemented steps of:
a) in response to an active state of the signal generation device while the cursor is over the selected text at a source location on said display device:
1) creating and displaying a text object of the selected text;
2) de-emphasizing the selected text at the source location;
b) in a finite series of steps moving the text object on the display device along a line between the source location and the cursor until the text object reaches the cursor;
c) displaying the insertion caret near the cursor;
d) in response to an inactive state of the signal generation device while the cursor is over a destination location:
1) on the display device zooming from a first bounding rectangle for the selected text at the source location to a second bounding rectangle for the selected block of text at the destination location; and
2) displaying on screen the selected text at the destination location.
2. The method claim 1 further comprising the computer implemented steps of:
a) while the cursor is located over the selected text displaying the cursor on the display device as an arrow; and
b) while the cursor is not located over the selected block text displaying the cursor on the display device as an I-beam.
3. The method of claim 1 wherein the text object depicts only a portion of the selected text.
4. The method of claim 3 wherein the text object has a maximum size.
5. The method of claim 1 wherein the selected text is removed from the source location after the selected text is displayed at the destination location.
6. The method of claim 1 wherein the selected text remains displayed at the source location after the selected block of text has been displayed at the destination location.
7. A computer implemented method of providing visual feedback to a computer user while manipulating selected text displayed on a display device of a computer system, the computer system including a mouse for interactively positioning a cursor and an insertion caret on the display device, the mouse having a mouse button having an up position and a down position, the method comprising the computer implemented steps of:
a) in response to the mouse button being in the down position while the cursor is over a selected text at a source location:
1) creating a text object from the selected text;
2) displaying on the display device the text object over the selected text at the source location;
3) visually de-emphasizing on the display device the selected text at the source location;
b) initializing an interpolation factor;
c) incrementing the interpolation factor if it has not reached a maximum value;
d) computing a distance between the text object and a first point associated with the pointer;
e) moving the text object along an imaginary line between the source location and the first point to a point on the display device determined by the distance multiplied by the interpolation factor;
f) with the mouse button is in the down position repeating steps b) through e);
g) if the mouse button is in the up position over a destination location:
1) computing a first bounding rectangle for the selected text at the source location;
2) computing a second bounding rectangle for the selected text at the destination location; and
3) zooming from the first bounding rectangle at the source location to the second bounding rectangle at the destination location.
8. The method of claim 7 further comprising the computer implemented steps of:
a) selecting the selected text at the source location from available text, displayed on the display device; and
b) highlighting on the display device the selected text.
9. The method of claim 7 further comprising the computer implemented step of:
a) offscreen inserting the selected text at the destination location after computing the first bounding rectangle.
10. The method of claim 9 further comprising the computer implemented step of:
a) performing an intelligent paste when the selected text is inserted at the destination location.
11. The method of claim 9 further comprising the step of:
a) displaying on the display device the selected text at the destination location after zooming from the first bounding rectangle at the source location to the second bounding rectangle at the destination location.
12. The method claim 7 further comprising the computer implemented steps of:
a) while the cursor is located over the selected text displaying the cursor on the display device as an arrow; and
b) while the cursor is not located over the selected text displaying the cursor on the display device as an I-beam.
13. The method of claim 12 wherein the text object depicts only a portion of the selected text.
14. The method of claim 13 wherein the text object has a maximum size.
15. The method of claim 7 further comprising the computer implemented step of:
a) removing the selected text from the source location after the selected text is displayed at the destination location.
16. The method of claim 15 further comprising the step of:
a) performing an intelligent cut at the source location when the selected text is removed from the source location.
17. The method of claim 11 wherein the selected block of text remains displayed at the source location after the selected block of text has been displayed at the destination location.
18. A device for providing visual feedback to a computer user while manipulating selected text displayed on a display device of a computer system, the computer system including a mouse for interactively positioning a cursor and an insertion caret on the display device, the mouse having a mouse button having an up position and a down position, the device comprising:
a) a first means for creating a text object from the selected text, the first means responding to the mouse button being in the down position while the cursor is over the selected text at the source location;
b) a second means for visually de-emphasizing on the display device the selected text at the source location, the second means responding to the mouse button being in the down position;
c) a third means for initializing an interpolation factor and incrementing the interpolation factor if the interpolation factor has not reached a maximum value, the third means responding to the mouse button being in the down position;
d) a fourth means for computing a distance between the text object and a first point associated with the pointer, the fourth means responsive to the mouse button being in the down position;
e) a fifth means for moving the text object along an imaginary line between the source location and the first point to a point on the display device determined by the distance multiplied by the interpolation factor, the fifth means responding to the mouse button being in the down position; and
f) a zooming means for zooming from a first bounding rectangle for the selected block of text at the source location to a second bounding rectangle for the selected block of text at the destination location.
19. An apparatus providing visual feedback while manipulating highlighted text, the apparatus comprising:
a) a computer system including:
1) a central processing unit;
2) a display device for displaying images, text and the highlighted text, the display device being coupled to the central processing unit;
3) a mouse for interactively positioning a cursor and an insertion caret on the display device, the mouse having a mouse button having an up position and a down position, the mouse being coupled to the central processing unit;
b) a first means coupled to the central processing unit, the first means creating a text object from the highlighted text, the first means responding to the mouse button being in the down position while the cursor is over the highlighted text at a source location;
c) a second means coupled to the central processing unit, the second means visually de-emphasizing on the display device the highlighted text at the source location, the second means responding to the mouse button being in the down position;
d) a third means coupled to the central processing unit, the third means initializing an interpolation factor and incrementing the interpolation factor if it has not reached a maximum value, the third means responding to the mouse button being in the down position;
e) a fourth means coupled to the central processing unit, the fifth mean computing a distance between the text object and a first point associated with the pointer, the fourth means responsive to the mouse button being in the down position;
f) a fifth means coupled to the central processing unit, the fifth means moving the text object along an imaginary line between the source location and the first point to a point on the display device determined by the distance multiplied by the interpolation factor, the fifth means responding to the mouse button being in the down position; and
g) a zooming means coupled to the central processing unit, the zooming means zooming from a first bounding rectangle for the selected block of text at the source location to a second bounding rectangle for the selected block of text at the destination location.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/697,845 US7499024B2 (en) | 1992-12-21 | 2003-10-29 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US99378492A | 1992-12-21 | 1992-12-21 | |
US08/294,195 US5561444A (en) | 1992-12-21 | 1994-08-22 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US08/626,295 US5694151A (en) | 1992-12-21 | 1996-04-01 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US08/979,069 US6396474B1 (en) | 1992-12-21 | 1997-11-26 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US10/135,262 US6690356B2 (en) | 1992-12-21 | 2002-04-29 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US10/697,845 US7499024B2 (en) | 1992-12-21 | 2003-10-29 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/135,262 Continuation US6690356B2 (en) | 1992-12-21 | 2002-04-29 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
Publications (2)
Publication Number | Publication Date |
---|---|
US20040085368A1 true US20040085368A1 (en) | 2004-05-06 |
US7499024B2 US7499024B2 (en) | 2009-03-03 |
Family
ID=25539927
Family Applications (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/294,195 Expired - Lifetime US5561444A (en) | 1992-12-21 | 1994-08-22 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US08/626,295 Expired - Lifetime US5694151A (en) | 1992-12-21 | 1996-04-01 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US08/979,069 Expired - Fee Related US6396474B1 (en) | 1992-12-21 | 1997-11-26 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US10/135,262 Expired - Fee Related US6690356B2 (en) | 1992-12-21 | 2002-04-29 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US10/697,845 Expired - Fee Related US7499024B2 (en) | 1992-12-21 | 2003-10-29 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
Family Applications Before (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/294,195 Expired - Lifetime US5561444A (en) | 1992-12-21 | 1994-08-22 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US08/626,295 Expired - Lifetime US5694151A (en) | 1992-12-21 | 1996-04-01 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US08/979,069 Expired - Fee Related US6396474B1 (en) | 1992-12-21 | 1997-11-26 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
US10/135,262 Expired - Fee Related US6690356B2 (en) | 1992-12-21 | 2002-04-29 | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
Country Status (1)
Country | Link |
---|---|
US (5) | US5561444A (en) |
Cited By (100)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050216866A1 (en) * | 1997-06-25 | 2005-09-29 | Comet Systems, Inc. | System for replacing a cursor image in connection with displaying the contents of a web page |
US20060005151A1 (en) * | 2004-07-02 | 2006-01-05 | Adobe Systems | Graphical interface for adjustment of text selections |
US20090249239A1 (en) * | 2008-03-31 | 2009-10-01 | Vistaprint Technologies Limited | Target-Alignment-And-Drop Control For Editing Electronic Documents |
US7739604B1 (en) | 2002-09-25 | 2010-06-15 | Apple Inc. | Method and apparatus for managing windows |
US20110167350A1 (en) * | 2010-01-06 | 2011-07-07 | Apple Inc. | Assist Features For Content Display Device |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
US9535906B2 (en) | 2008-07-31 | 2017-01-03 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
US9620104B2 (en) | 2013-06-07 | 2017-04-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9626955B2 (en) | 2008-04-05 | 2017-04-18 | Apple Inc. | Intelligent text-to-speech conversion |
US9633674B2 (en) | 2013-06-07 | 2017-04-25 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
US9633660B2 (en) | 2010-02-25 | 2017-04-25 | Apple Inc. | User profiling for voice input processing |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US9646614B2 (en) | 2000-03-16 | 2017-05-09 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
US9798393B2 (en) | 2011-08-29 | 2017-10-24 | Apple Inc. | Text correction processing |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9953088B2 (en) | 2012-05-14 | 2018-04-24 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US9966065B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
US9966068B2 (en) | 2013-06-08 | 2018-05-08 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US9971774B2 (en) | 2012-09-19 | 2018-05-15 | Apple Inc. | Voice-based media searching |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US10079014B2 (en) | 2012-06-08 | 2018-09-18 | Apple Inc. | Name recognition system |
US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10089072B2 (en) | 2016-06-11 | 2018-10-02 | Apple Inc. | Intelligent device arbitration and control |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US10169329B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Exemplar-based natural language processing |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10185542B2 (en) | 2013-06-09 | 2019-01-22 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US10269345B2 (en) | 2016-06-11 | 2019-04-23 | Apple Inc. | Intelligent task discovery |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US10283110B2 (en) | 2009-07-02 | 2019-05-07 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US10297253B2 (en) | 2016-06-11 | 2019-05-21 | Apple Inc. | Application integration with a digital assistant |
US10318871B2 (en) | 2005-09-08 | 2019-06-11 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US10354011B2 (en) | 2016-06-09 | 2019-07-16 | Apple Inc. | Intelligent automated assistant in a home environment |
US10356243B2 (en) | 2015-06-05 | 2019-07-16 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US10410637B2 (en) | 2017-05-12 | 2019-09-10 | Apple Inc. | User-specific acoustic models |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
US10482874B2 (en) | 2017-05-15 | 2019-11-19 | Apple Inc. | Hierarchical belief states for digital assistants |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10521466B2 (en) | 2016-06-11 | 2019-12-31 | Apple Inc. | Data driven natural language event detection and classification |
US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US10568032B2 (en) | 2007-04-03 | 2020-02-18 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10706373B2 (en) | 2011-06-03 | 2020-07-07 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10733993B2 (en) | 2016-06-10 | 2020-08-04 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US10755703B2 (en) | 2017-05-11 | 2020-08-25 | Apple Inc. | Offline personal assistant |
US10791176B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Synchronization and task delegation of a digital assistant |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US10810274B2 (en) | 2017-05-15 | 2020-10-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US11217255B2 (en) | 2017-05-16 | 2022-01-04 | Apple Inc. | Far-field extension for digital assistant services |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
Families Citing this family (54)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5561444A (en) * | 1992-12-21 | 1996-10-01 | Apple Computer, Inc. | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
EP0626635B1 (en) * | 1993-05-24 | 2003-03-05 | Sun Microsystems, Inc. | Improved graphical user interface with method for interfacing to remote devices |
US5930813A (en) * | 1995-12-21 | 1999-07-27 | Adobe Systems Incorporated | Method and system for designating objects |
JP3422897B2 (en) * | 1996-05-17 | 2003-06-30 | 株式会社テクノクラフト | Character string extraction system and character string extraction method |
US6081818A (en) * | 1996-12-11 | 2000-06-27 | International Business Machines Corporation | Cut object dynamic display and method of displaying cut objects |
US6240430B1 (en) * | 1996-12-13 | 2001-05-29 | International Business Machines Corporation | Method of multiple text selection and manipulation |
JPH1173294A (en) * | 1997-08-25 | 1999-03-16 | Internatl Business Mach Corp <Ibm> | Pointing device and method therefor |
US20020057285A1 (en) * | 2000-08-04 | 2002-05-16 | Nicholas James J. | Non-intrusive interactive notification system and method |
WO2000073970A2 (en) * | 1999-05-19 | 2000-12-07 | Nicholas James J | Cursor movable interactive message |
US6792158B1 (en) * | 1999-10-28 | 2004-09-14 | Hewlett-Packard Development Company, L.P. | System and method for image enhancement |
JP2001236156A (en) * | 2000-02-24 | 2001-08-31 | Nec Corp | Responding method to user instruction of information processor, recording medium and information processor |
US20010030667A1 (en) * | 2000-04-10 | 2001-10-18 | Kelts Brett R. | Interactive display interface for information objects |
US6925497B1 (en) | 2000-09-26 | 2005-08-02 | Microsoft Corporation | Systems and methods for controlling the number of clients that access a server |
US7069507B1 (en) * | 2000-09-29 | 2006-06-27 | Microsoft Corporation | Event routing model for an extensible editor |
US7036078B1 (en) * | 2000-09-29 | 2006-04-25 | Microsoft Corporation | Highlight rendering services component for an extensible editor |
US7444587B1 (en) | 2000-09-29 | 2008-10-28 | Microsoft Corporation | Selection services component for an extensible editor |
US20080060020A1 (en) | 2000-12-22 | 2008-03-06 | Hillcrest Laboratories, Inc. | Methods and systems for semantic zooming |
TW554282B (en) * | 2001-03-15 | 2003-09-21 | Curtis Whitcomb | Ergonomic computer mouse |
JP2002290543A (en) * | 2001-03-26 | 2002-10-04 | Hitachi Kokusai Electric Inc | Portable telephone |
US7047503B1 (en) * | 2001-03-28 | 2006-05-16 | Palmsource, Inc. | Method and apparatus for the selection of records |
US7640098B2 (en) * | 2001-07-31 | 2009-12-29 | Stenbock & Everson, Inc. | Process for generating travel plans on the internet |
US7296244B2 (en) * | 2001-10-18 | 2007-11-13 | International Business Machines Corporation | Method of visually indicating transfer of data in response to a transfer data command |
US7607102B2 (en) * | 2002-03-14 | 2009-10-20 | Apple Inc. | Dynamically changing appearances for user interface elements during drag-and-drop operations |
US7062723B2 (en) * | 2002-05-20 | 2006-06-13 | Gateway Inc. | Systems, methods and apparatus for magnifying portions of a display |
US8555165B2 (en) * | 2003-05-08 | 2013-10-08 | Hillcrest Laboratories, Inc. | Methods and systems for generating a zoomable graphical user interface |
US7644367B2 (en) * | 2003-05-16 | 2010-01-05 | Microsoft Corporation | User interface automation framework classes and interfaces |
US8127252B2 (en) * | 2003-11-07 | 2012-02-28 | Microsoft Corporation | Method and system for presenting user interface (UI) information |
WO2006036887A2 (en) * | 2004-09-28 | 2006-04-06 | Yost David A | Improved system of gui text cursor, caret, and selection |
KR100677426B1 (en) * | 2005-01-14 | 2007-02-02 | 엘지전자 주식회사 | Short message display method for mobile communication device |
US7724239B2 (en) * | 2005-02-22 | 2010-05-25 | Research In Motion Limited | Handheld electronic device, cursor positioning sub-system and method employing cursor scaling control |
US7917860B2 (en) * | 2005-03-03 | 2011-03-29 | Microsoft Corporation | Simple styling |
US20090048821A1 (en) * | 2005-07-27 | 2009-02-19 | Yahoo! Inc. | Mobile language interpreter with text to speech |
US7827503B2 (en) * | 2005-07-27 | 2010-11-02 | Yahoo! Inc. | Automatically generating a search result in a separate window for a displayed symbol that is selected with a drag and drop control |
US7877685B2 (en) * | 2005-12-29 | 2011-01-25 | Sap Ag | Persistent adjustable text selector |
US20090209899A1 (en) | 2006-06-15 | 2009-08-20 | Harry Unger | Delivery system and process |
US8672682B2 (en) | 2006-09-28 | 2014-03-18 | Howard A. Engelsen | Conversion of alphabetic words into a plurality of independent spellings |
US8418079B2 (en) | 2009-09-01 | 2013-04-09 | James J. Nicholas, III | System and method for cursor-based application management |
US9292161B2 (en) * | 2010-03-24 | 2016-03-22 | Microsoft Technology Licensing, Llc | Pointer tool with touch-enabled precise placement |
US8982045B2 (en) | 2010-12-17 | 2015-03-17 | Microsoft Corporation | Using movement of a computing device to enhance interpretation of input events produced when interacting with the computing device |
US9244545B2 (en) | 2010-12-17 | 2016-01-26 | Microsoft Technology Licensing, Llc | Touch and stylus discrimination and rejection for contact sensitive computing devices |
US8988398B2 (en) | 2011-02-11 | 2015-03-24 | Microsoft Corporation | Multi-touch input device with orientation sensing |
US8994646B2 (en) | 2010-12-17 | 2015-03-31 | Microsoft Corporation | Detecting gestures involving intentional movement of a computing device |
US20120154295A1 (en) * | 2010-12-17 | 2012-06-21 | Microsoft Corporation | Cooperative use of plural input mechanisms to convey gestures |
US10048854B2 (en) * | 2011-01-31 | 2018-08-14 | Oracle International Corporation | Drag and drop interaction between components of a web application |
US9201520B2 (en) | 2011-02-11 | 2015-12-01 | Microsoft Technology Licensing, Llc | Motion and context sharing for pen-based computing inputs |
US9317196B2 (en) | 2011-08-10 | 2016-04-19 | Microsoft Technology Licensing, Llc | Automatic zooming for text selection/cursor placement |
US8902181B2 (en) | 2012-02-07 | 2014-12-02 | Microsoft Corporation | Multi-touch-movement gestures for tablet computing devices |
CN104111787B (en) * | 2013-04-18 | 2018-09-28 | 三星电子(中国)研发中心 | A kind of method and apparatus for realizing text editing on touch screen interface |
WO2015109507A1 (en) * | 2014-01-24 | 2015-07-30 | 华为终端有限公司 | Method and electronic device for inputting character |
GB201408258D0 (en) | 2014-05-09 | 2014-06-25 | British Sky Broadcasting Ltd | Television display and remote control |
US9727161B2 (en) | 2014-06-12 | 2017-08-08 | Microsoft Technology Licensing, Llc | Sensor correlation for pen and touch-sensitive computing device interaction |
US9870083B2 (en) | 2014-06-12 | 2018-01-16 | Microsoft Technology Licensing, Llc | Multi-device multi-user sensor correlation for pen and computing device interaction |
US10198246B2 (en) | 2016-08-19 | 2019-02-05 | Honeywell International Inc. | Methods and apparatus for voice-activated control of an interactive display |
WO2018212998A1 (en) * | 2017-05-16 | 2018-11-22 | Apple Inc. | Devices, methods, and graphical user interfaces for moving user interface objects |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4922448A (en) * | 1985-07-10 | 1990-05-01 | Brother Kogyo Kabushiki Kaisha | Word processing system having small-sized and large-sized displays for simultaneous display and automatic power shut-off circuit |
US5157763A (en) * | 1987-10-15 | 1992-10-20 | International Business Machines Corporation | Visually assisted method for transfer of data within an application or from a source application to a receiving application |
US5040131A (en) * | 1987-12-23 | 1991-08-13 | International Business Machines Corporation | Graphical processing |
US4937439A (en) * | 1988-05-13 | 1990-06-26 | National Computer Systems, Inc. | Method and system for creating and scanning a customized survey form |
US5442742A (en) * | 1990-12-21 | 1995-08-15 | Apple Computer, Inc. | Method and apparatus for the manipulation of text on a computer display screen |
US5561444A (en) * | 1992-12-21 | 1996-10-01 | Apple Computer, Inc. | Method and apparatus for providing visual feedback during manipulation of text on a computer screen |
-
1994
- 1994-08-22 US US08/294,195 patent/US5561444A/en not_active Expired - Lifetime
-
1996
- 1996-04-01 US US08/626,295 patent/US5694151A/en not_active Expired - Lifetime
-
1997
- 1997-11-26 US US08/979,069 patent/US6396474B1/en not_active Expired - Fee Related
-
2002
- 2002-04-29 US US10/135,262 patent/US6690356B2/en not_active Expired - Fee Related
-
2003
- 2003-10-29 US US10/697,845 patent/US7499024B2/en not_active Expired - Fee Related
Cited By (137)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7975241B2 (en) * | 1997-06-25 | 2011-07-05 | Lexos Media, Inc. | System for replacing a cursor image in connection with displaying the contents of a web page |
US20050216866A1 (en) * | 1997-06-25 | 2005-09-29 | Comet Systems, Inc. | System for replacing a cursor image in connection with displaying the contents of a web page |
US9646614B2 (en) | 2000-03-16 | 2017-05-09 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US8701030B2 (en) | 2002-09-25 | 2014-04-15 | Apple Inc. | Method and apparatus for managing windows |
US7739604B1 (en) | 2002-09-25 | 2010-06-15 | Apple Inc. | Method and apparatus for managing windows |
US20100257482A1 (en) * | 2002-09-25 | 2010-10-07 | David Anthony Lyons | Method and apparatus for managing windows |
US20060005151A1 (en) * | 2004-07-02 | 2006-01-05 | Adobe Systems | Graphical interface for adjustment of text selections |
US10318871B2 (en) | 2005-09-08 | 2019-06-11 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US10568032B2 (en) | 2007-04-03 | 2020-02-18 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US10381016B2 (en) | 2008-01-03 | 2019-08-13 | Apple Inc. | Methods and apparatus for altering audio output signals |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US8117556B2 (en) | 2008-03-31 | 2012-02-14 | Vistaprint Technologies Limited | Target-alignment-and-drop control for editing electronic documents |
AU2009249503B2 (en) * | 2008-03-31 | 2015-05-07 | Cimpress Schweiz Gmbh | Target-alignment-and-drop control for editing electronic documents |
WO2009142813A3 (en) * | 2008-03-31 | 2010-08-12 | Vistaprint Technologies Limited | Target-alignment-and-drop control for editing electronic documents |
WO2009142813A2 (en) * | 2008-03-31 | 2009-11-26 | Vistaprint Technologies Limited | Target-alignment-and-drop control for editing electronic documents |
US20090249239A1 (en) * | 2008-03-31 | 2009-10-01 | Vistaprint Technologies Limited | Target-Alignment-And-Drop Control For Editing Electronic Documents |
US9626955B2 (en) | 2008-04-05 | 2017-04-18 | Apple Inc. | Intelligent text-to-speech conversion |
US9865248B2 (en) | 2008-04-05 | 2018-01-09 | Apple Inc. | Intelligent text-to-speech conversion |
US10108612B2 (en) | 2008-07-31 | 2018-10-23 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
US9535906B2 (en) | 2008-07-31 | 2017-01-03 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
US10795541B2 (en) | 2009-06-05 | 2020-10-06 | Apple Inc. | Intelligent organization of tasks items |
US11080012B2 (en) | 2009-06-05 | 2021-08-03 | Apple Inc. | Interface for a virtual digital assistant |
US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US10475446B2 (en) | 2009-06-05 | 2019-11-12 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US10283110B2 (en) | 2009-07-02 | 2019-05-07 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US20110167350A1 (en) * | 2010-01-06 | 2011-07-07 | Apple Inc. | Assist Features For Content Display Device |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US9548050B2 (en) | 2010-01-18 | 2017-01-17 | Apple Inc. | Intelligent automated assistant |
US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10706841B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Task flow identification based on user intent |
US11423886B2 (en) | 2010-01-18 | 2022-08-23 | Apple Inc. | Task flow identification based on user intent |
US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
US9633660B2 (en) | 2010-02-25 | 2017-04-25 | Apple Inc. | User profiling for voice input processing |
US10049675B2 (en) | 2010-02-25 | 2018-08-14 | Apple Inc. | User profiling for voice input processing |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US10102359B2 (en) | 2011-03-21 | 2018-10-16 | Apple Inc. | Device access using voice authentication |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US10706373B2 (en) | 2011-06-03 | 2020-07-07 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
US11120372B2 (en) | 2011-06-03 | 2021-09-14 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US9798393B2 (en) | 2011-08-29 | 2017-10-24 | Apple Inc. | Text correction processing |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
US9953088B2 (en) | 2012-05-14 | 2018-04-24 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US10079014B2 (en) | 2012-06-08 | 2018-09-18 | Apple Inc. | Name recognition system |
US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
US9971774B2 (en) | 2012-09-19 | 2018-05-15 | Apple Inc. | Voice-based media searching |
US9966060B2 (en) | 2013-06-07 | 2018-05-08 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9620104B2 (en) | 2013-06-07 | 2017-04-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9633674B2 (en) | 2013-06-07 | 2017-04-25 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
US9966068B2 (en) | 2013-06-08 | 2018-05-08 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
US10657961B2 (en) | 2013-06-08 | 2020-05-19 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
US10185542B2 (en) | 2013-06-09 | 2019-01-22 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
US10169329B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Exemplar-based natural language processing |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
US11133008B2 (en) | 2014-05-30 | 2021-09-28 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US10497365B2 (en) | 2014-05-30 | 2019-12-03 | Apple Inc. | Multi-command single utterance input method |
US9966065B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
US9668024B2 (en) | 2014-06-30 | 2017-05-30 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10904611B2 (en) | 2014-06-30 | 2021-01-26 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10431204B2 (en) | 2014-09-11 | 2019-10-01 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US9986419B2 (en) | 2014-09-30 | 2018-05-29 | Apple Inc. | Social reminders |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
US11556230B2 (en) | 2014-12-02 | 2023-01-17 | Apple Inc. | Data detection |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US10311871B2 (en) | 2015-03-08 | 2019-06-04 | Apple Inc. | Competing devices responding to voice triggers |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US11087759B2 (en) | 2015-03-08 | 2021-08-10 | Apple Inc. | Virtual assistant activation |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US10356243B2 (en) | 2015-06-05 | 2019-07-16 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US11500672B2 (en) | 2015-09-08 | 2022-11-15 | Apple Inc. | Distributed personal assistant |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US11526368B2 (en) | 2015-11-06 | 2022-12-13 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
US11069347B2 (en) | 2016-06-08 | 2021-07-20 | Apple Inc. | Intelligent automated assistant for media exploration |
US10354011B2 (en) | 2016-06-09 | 2019-07-16 | Apple Inc. | Intelligent automated assistant in a home environment |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
US10733993B2 (en) | 2016-06-10 | 2020-08-04 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
US11037565B2 (en) | 2016-06-10 | 2021-06-15 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10297253B2 (en) | 2016-06-11 | 2019-05-21 | Apple Inc. | Application integration with a digital assistant |
US10089072B2 (en) | 2016-06-11 | 2018-10-02 | Apple Inc. | Intelligent device arbitration and control |
US10269345B2 (en) | 2016-06-11 | 2019-04-23 | Apple Inc. | Intelligent task discovery |
US10521466B2 (en) | 2016-06-11 | 2019-12-31 | Apple Inc. | Data driven natural language event detection and classification |
US11152002B2 (en) | 2016-06-11 | 2021-10-19 | Apple Inc. | Application integration with a digital assistant |
US10553215B2 (en) | 2016-09-23 | 2020-02-04 | Apple Inc. | Intelligent automated assistant |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US10755703B2 (en) | 2017-05-11 | 2020-08-25 | Apple Inc. | Offline personal assistant |
US10410637B2 (en) | 2017-05-12 | 2019-09-10 | Apple Inc. | User-specific acoustic models |
US11405466B2 (en) | 2017-05-12 | 2022-08-02 | Apple Inc. | Synchronization and task delegation of a digital assistant |
US10791176B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Synchronization and task delegation of a digital assistant |
US10482874B2 (en) | 2017-05-15 | 2019-11-19 | Apple Inc. | Hierarchical belief states for digital assistants |
US10810274B2 (en) | 2017-05-15 | 2020-10-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
US11217255B2 (en) | 2017-05-16 | 2022-01-04 | Apple Inc. | Far-field extension for digital assistant services |
Also Published As
Publication number | Publication date |
---|---|
US5694151A (en) | 1997-12-02 |
US6396474B1 (en) | 2002-05-28 |
US5561444A (en) | 1996-10-01 |
US7499024B2 (en) | 2009-03-03 |
US20020130837A1 (en) | 2002-09-19 |
US6690356B2 (en) | 2004-02-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6690356B2 (en) | Method and apparatus for providing visual feedback during manipulation of text on a computer screen | |
JP7097991B2 (en) | Devices and methods for measuring using augmented reality | |
US6100889A (en) | Display of menu items on a computer screen | |
US9684436B2 (en) | User interface for providing consolidation and access | |
US20020015064A1 (en) | Gesture-based user interface to multi-level and multi-modal sets of bit-maps | |
US6476831B1 (en) | Visual scrolling feedback and method of achieving the same | |
US6642936B1 (en) | Touch zoom in/out for a graphics display | |
EP0677803B1 (en) | A method and system for facilitating the selection of icons | |
US5754809A (en) | Perspective windowing technique for computer graphical user interface | |
Appert et al. | OrthoZoom scroller: 1D multi-scale navigation | |
US8578292B2 (en) | Simultaneous document zoom and centering adjustment | |
US5396590A (en) | Non-modal method and apparatus for manipulating graphical objects | |
JP4637455B2 (en) | User interface utilization method and product including computer usable media | |
US8522165B2 (en) | User interface and method for object management | |
EP0757309A2 (en) | Transient link indicators in image maps | |
US20110099511A1 (en) | Method and system for displaying, locating, and browsing data files | |
US20120256960A1 (en) | Defining motion in a computer system with a graphical user interface | |
WO2007011578A2 (en) | Indicating file type on a thumbnail preview icon | |
US6718518B1 (en) | Non-disruptive search facility | |
JP2011060301A (en) | Improved mobile communications terminal and method | |
US20030231218A1 (en) | System and method for indicating the focused control on a graphical user interface | |
Masui | LensBar-visualization for browsing and filtering large lists of data | |
JP2004192294A (en) | Method for displaying object and execution device therefor | |
EP0550838A1 (en) | Method and computer-aided design system for defining geometric relations | |
JP3256994B2 (en) | Display target movement method by touch input |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: APPLE INC., CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:APPLE COMPUTER, INC., A CALIFORNIA CORPORATION;REEL/FRAME:019234/0400 Effective date: 20070109 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20170303 |