US20140361971A1 - Visual enhancements based on eye tracking - Google Patents

Visual enhancements based on eye tracking Download PDF

Info

Publication number
US20140361971A1
US20140361971A1 US13/911,867 US201313911867A US2014361971A1 US 20140361971 A1 US20140361971 A1 US 20140361971A1 US 201313911867 A US201313911867 A US 201313911867A US 2014361971 A1 US2014361971 A1 US 2014361971A1
Authority
US
United States
Prior art keywords
gaze
item
user interface
location
method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US13/911,867
Other versions
US9965062B2 (en
Inventor
Pablo Luis Sala
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Priority to US13/911,867 priority Critical patent/US9965062B2/en
Publication of US20140361971A1 publication Critical patent/US20140361971A1/en
Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC reassignment MICROSOFT TECHNOLOGY LICENSING, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MICROSOFT CORPORATION
Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC reassignment MICROSOFT TECHNOLOGY LICENSING, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MICROSOFT CORPORATION
Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SALA, Pablo Luis
Publication of US9965062B2 publication Critical patent/US9965062B2/en
Application granted granted Critical
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/04806Zoom, i.e. interaction techniques or interactors for controlling the zooming operation

Abstract

Embodiments are disclosed that relate to selectively enhancing content displayed on a display device. For example, one disclosed embodiment provides a method comprising tracking a gaze location at which a gaze of a user intersects the graphical user interface, and upon detecting a gaze pattern that meets a defined gaze pattern condition indicative of difficulty in perceiving an item at the gaze location, displaying the item in at the gaze location a visually enhanced form via a user interface object displayed separately from the item at the gaze location.

Description

    BACKGROUND
  • Items displayed on a graphical user interface may at times be difficult for users to perceive. For example, text may be displayed in a color having a low contrast relative to a background color, or in a font that is small for a particular display screen size. Other types of displayed information may suffer similar problems. As such, computing devices may offer various ways for user to modify the appearance of displayed items. For example, users may be able to increase the size of text, zoom in on displayed images, change colors of displayed items, and take other such actions to enhance a visibility of a user interface item.
  • SUMMARY
  • Embodiments are disclosed that relate to automatically visually enhancing items displayed on a display device. For example, one disclosed embodiment provides a method comprising tracking a gaze location at which a gaze of a user intersects an item, and upon detecting a gaze pattern that meets a defined gaze pattern condition indicative of difficulty in perceiving an item at the gaze location, displaying the item at the gaze location in a visually enhanced form via a user interface object displayed separately from the item at the gaze location.
  • This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 schematically shows an example display device and graphical user interface in accordance with an embodiment of the present disclosure.
  • FIGS. 2-5 schematically show examples of visually enhancing items of the graphical user interface of FIG. 1 in accordance with embodiments of the present disclosure.
  • FIG. 6 is a flow diagram depicting an embodiment of a method of selectively enhancing a user interface item displayed on a display device.
  • FIG. 7 shows a block diagram of an example display device in accordance with an embodiment of the present disclosure.
  • FIG. 8 schematically shows an example display device and graphical user interface in accordance with another embodiment of the present disclosure.
  • FIG. 9 schematically shows an example of visually enhancing user interface items of the graphical user interface of FIG. 8 in accordance with an embodiment of the present disclosure.
  • FIG. 10 schematically shows an example of visually enhancing a view of a physical object in accordance with an embodiment of the present disclosure.
  • FIG. 11 is shows an example embodiment of a computing system.
  • DETAILED DESCRIPTION
  • As mentioned above, computing devices may offer various ways for users to modify the appearance of items displayed on a graphical user interface. The tools for making such modifications are generally invoked intentionally by users. For example, if a user of a laptop computer is struggling to read small text, the user may select a font size adjustment tool from a menu. Likewise, if a user of a smart phone is struggling to view a user interface item, the user may use a stretch gesture or the like to view the item at a higher magnification.
  • When a user struggles to view a user interface item, the visual struggle may have associated gaze movement characteristics that are detectable via gaze tracking. Thus, embodiments are disclosed that relate to automatically enhancing user interface items displayed on a display device based upon eye gaze motion characteristics detected via gaze tracking.
  • FIG. 1 shows an example use environment 100 comprising a computing device 102 displaying a graphical user interface 104. While shown in the context of a head-mounted display device in FIG. 1, it is understood that any other suitable computing device may be used, including but not limited to a smart phone, a personal computer, a game console, a mobile computing device, a server computing device, a tablet, etc. Likewise, while graphical user interface 104 is shown in the form of a holographic user interface, a head-mounted display also may be used to augment an appearance of a graphical user interface displayed on a separate display device (e.g. a computer monitor, television, smart phone, tablet, laptop computer screen, etc.). FIG. 1 also shows a user 108 interacting with the graphical user interface 104 via gaze. Items within a field of view 110 of the head-mounted display device are illustrated as seen through the head-mounted display device.
  • As described in more detail below, the computing device 102 may comprise a gaze detection subsystem configured to detect a gaze 112 of the user 108 and determine a location at which the gaze 112 of the user intersects the graphical user interface 104. The gaze detection subsystem may perform these operations periodically to output a periodically updated position signal. The graphical user interface 104 is shown as displaying a video item 114, an image item 116, and a text item 118 for the purpose of example, but it will be understood that the graphical user interface may display any other suitable item or items. It will be understood that the term “user interface item” and the like refer to any item displayed on a display device that may be visually enhanced as described herein.
  • Changes in the periodically updated position signal from the gaze tracking subsystem over time may allow recognized gaze patterns to be detected. For example, as mentioned above, gaze patterns may be used to detect whether a user is struggling to view a user interface item. As a more specific example, a user may read over the same sentence multiple times, dwell at a portion of an image for a relatively longer duration, move at a lower speed than usual when reading, and/or perform other such eye gaze behaviors that are detectable as motion conditions indicative of the user struggling to view a user interface item. Accordingly, the user's gaze path may be analyzed to identify gaze patterns associated with difficulty in perception. Any suitable methods may be used to identify such gaze patterns. Examples include, but not limited to, comparing gaze path data to pre-defined gaze patterns, comparing gaze path data to prior patterns of the current user (as learned via calibration and/or observation during ordinary use), and/or evaluating gaze path data via the use of classification functions trained with training data comprising gaze patterns corresponding to known conditions.
  • The computing device 102 further may be configured to determine contextual features associated with item(s) at the gaze location, and use contextual information as a filter or filters in determining whether a user is having difficulty viewing a user interface item. Any suitable contextual information may be utilized. For example, the computing device may determine a type of user interface item in the gaze location. Examples of item types include, but are not limited to, text, video items, image items, executable items, icons, action items (e.g. user interface control items), and/or any suitable type of item.
  • As an example of how contextual information may be used to distinguish eye patterns, a back-and-forth eye pattern that may indicate difficulty in reading an item may not indicate similar difficulties for other types of content. As a more specific example, a user viewing a video item may be expected to exhibit a gaze pattern that follows one or more objects within the video. In the illustrated example, video item 114 comprises a tennis match. As such, the gaze of the user may traverse the same location multiple times as the user changes view from player to player. In this example, the repetitive gaze pattern is not likely to be indicative of a struggle in perceiving the item. Contextual information may be utilized in other manners as well. For example, different visual enhancements may be applied depending upon a type of item at the gaze location.
  • As another example, as the user gazes at text item 118, the text item may be evaluated to determine characteristics such as a font, font size, font color, background color, font style (e.g., bold, italic, underline, subscript/superscript, strikethrough, embossed, outline, gradient, etc.), and other such formatting. The text item also may be evaluated to determine characteristics of the text content, such as a number of words in the gaze location, a complexity of words in the gaze location, etc.
  • Other types of items than text may be evaluated for a different set of characteristics than text. For example, a video item may be evaluated for factors such as an overall size of the displayed video content, a resolution of the content, and/or a level of complexity associated with moving images depicted therein. As a more specific example, a video with much high spatial frequency image data, and/or a large variation in image data between frames, may be determined to have a higher level of complexity than a video lacking such features.
  • Such features and characteristics may be used in any suitable manner to help determine and/or weigh the likelihood that a detected gaze pattern is indicative of a user having difficulty perceiving a user interface item. For example, in some embodiments, such information may be used to determine a confidence score that indicates a likelihood that the detected gaze pattern is indicative of such difficulties. In the context of text items, a contextual filter may specify a first confidence score for a smaller font size and a second confidence score for a larger font size, wherein the first confidence score represents a higher likelihood that the user is having difficulty perceiving an item than the second confidence score. The confidence score determined for a detected gaze pattern may be compared to a threshold score condition (e.g. below or above a threshold score) in order to determine whether a context-based gaze pattern condition is met. It will be understood that a confidence score may be based on any suitable characteristic or characteristics associated with the item.
  • In some instances, the user's gaze 112 may be determined to be directed to multiple items or item types. In such instances, the multiple items or item types may be prioritized, weighted to have different influences on a confidence score, or otherwise separately considered to assist in select a type for evaluation and potential modification.
  • In response to determining that a defined gaze pattern condition is met, the computing device 102 may visually enhance an appearance of the item at the gaze location. Any suitable visual enhancement may be applied, including but not limited to changes to color, size, font type, font style, gradient, sharpness, contrast, etc. For example, referring to FIG. 2, if it is determined that the “brown dog” text to which the user's gaze 112 is directed has a font size that is smaller than a threshold and/or has a confidence score that is below a threshold, then a font size of the text may be increased, or the text may otherwise be displayed as having a magnified appearance in the user interface object.
  • Visual enhancements to a user interface item may be displayed in any suitable manner. For example, in some embodiments, a visually enhanced version of the item may be displayed in a user interface object located over (e.g. at a higher z-order position than) the item, or otherwise located separately from the item. FIG. 2 shows an example of such a user interface object 202 as a text oval in which the text is displayed in the larger font, but it will be understood that such a user interface object may have any other suitable appearance.
  • FIGS. 3-5 show additional examples of visual enhancements made to a user interface object. First, in FIG. 3, the user's gaze 112 is directed to the portion of the text item 118 including the words “the quick.” As shown in FIG. 1, this text is presented on the graphical user interface a stylized font that utilizes stylized script-like lettering. If a visual pattern indicative of perception difficulty is detected, then the text may be displayed via a user interface object 302 in a different font and/or with other visual enhancements. For example, as illustrated in FIG. 3, the text including the words “the quick” has a larger font size and a different type of font than the original text in order to increase perceptibility with respect to the original form of the words.
  • FIG. 4 illustrates a visual enhancement configured to increase a contrast of text against a background color. The original version of the word “jumped” is light and difficult to read against a white background. As such, a user may have difficulty distinguishing the text from the background. Thus, the appearance (e.g. color, brightness, saturation, etc.) of the text and/or the background may be modified to increase a contrast of the text and background, and the resulting visually enhanced text may be displayed as user interface object 402. As another example, FIG. 5 shows a user interface object 502 that displays a portion of a text item having a different style of font (e.g., outline) than an original version of the text.
  • The examples illustrated herein are described for the purpose of example, and any other suitable visual enhancement or enhancements may be made to a user interface item, including but not limited to magnification. Furthermore, while the above-described figures illustrate visual enhancements made to text, it will be understood that similar alterations may be applied to action items, image items, video items, and/or any other suitable type of item.
  • FIG. 6 is a flow diagram that illustrates an embodiment of a method 600 of selectively enhancing content displayed on a display device. Method 600 comprises, at 602, displaying a graphical user interface comprising one or more items, and at 604, tracking a gaze location. As mentioned above, a computing device may comprise, or otherwise receive input from, a gaze tracking subsystem that provides a periodically updated location signal representing a determined location at which the user's gaze intersects a display. More information on an example gaze tracking subsystem is given below.
  • Method 600 further comprises, at 606, determining a gaze pattern from gaze path data that represents changes in the location of the position signal over a period of time. Gaze path data may include movements, dwells, combinations thereof, and/or other gaze path features. The gaze pattern may be determined from the gaze path data in any suitable manner. For example, the gaze pattern may be detected based upon a comparison of gaze path data to pre-defined patterns, to pre-defined conditions (e.g. gaze does not move beyond a selected number of pixels for a selected time), to prior detected patterns of the current user, to calibration data provided by the user during a calibration process as indicated at 608, and/or by use of classification functions trained with known gaze data. As a more specific example, the computing device may be calibrated to a current user via an automatic or semi-automatic calibration routine in which the user is instructed to perform one or more tasks relating to the perception of various items. The user may indicate a level of difficulty in perceiving the various items, and the tracked gaze patterns (and potentially contextual features of the items) may be compared to the indicated difficulty level to determine a gaze pattern condition. Statistical data relating gaze patterns of the user to levels of difficulty in perceiving items may thereby be collected for use in determining the confidence score.
  • In some embodiments, gaze patterns may be used to determine whether a user is having perception difficulties without considering any contextual data. In other embodiments, contextual data may be used along with gaze patterns to provide additional information for this determination. As such, method 600 may comprise applying a contextual filter to the gaze pattern determination, as indicated at 610. The contextual filter may be used with the determined gaze pattern, for example, to provide a confidence score based on one or more characteristics of an item at the gaze location as well as upon the determined gaze path.
  • Continuing, the gaze pattern may be evaluated in order to determine if it meets a gaze pattern condition, as indicated at 612. For example, the gaze pattern may be compared to predetermined paths and/or conditions, compared to prior user data, evaluated via a classification function, etc. Likewise, a confidence score may be compared to a threshold confidence score condition, and/or any other suitable evaluation may be utilized.
  • If the gaze pattern does not meet the gaze pattern condition, then method 600 returns to 604 to continue tracking a gaze location. On the other hand, if the gaze pattern condition is met, then method comprises, at 614, displaying an item at the gaze location in a visually enhanced form via a user interface object displayed separately from the item at the gaze location. For example, the user interface object may be displayed as an overlay of the original item (e.g. having a higher z-order position than the original item), and the item may be displayed therein with a magnified or otherwise altered appearance. In other embodiments, the user interface object may be displayed at a location that is spaced from the original item, and/or displayed in any other suitable manner.
  • The user interface object may persist for any suitable duration, and display of the user interface object may cease based upon any suitable events and/or conditions. For example, the computing device may cease to display the user interface object responsive to determining that the gaze is no longer directed to the gaze location and/or item. In other embodiments, the user may be able to move the user interface object to different locations, as if the user interface object were a lens that visually enhances any item located below the user interface object.
  • Method 600 may be performed via any suitable computing device that utilizes gaze tracking functionalities. FIG. 7 shows a block diagram of a computing system 700 comprising eye tracking functionality. The computing system 700 may represent the head-mounted display of FIG. 1, and/or any other suitable computing system configured to utilize a periodically updated position signal as an input. The computing system 700 includes a display 702 on which a graphical user interface may be displayed. The display 702 may take any suitable form. For example, in some embodiments, the computing system may include a near-eye display 704. Examples of near-eye display systems may include, but are not limited to head-mounted displays and handheld devices (e.g. smart phones, tablets, and/or other handheld devices having a near-eye display mode). In such devices, the near eye display may comprise a see-through display 706, or an opaque display. In other embodiments, the display may be external 708 to a computing device, such as in the case of a conventional monitor, television, or the like.
  • The computing system 700 further comprises an input subsystem 710 configured to receive an input of a periodically updated position signal from an eye tracking subsystem 712. In some embodiments, the eye tracking subsystem 712 may be head-mounted 714 (e.g. incorporated into a head-mounted near-eye display system) or otherwise incorporated into a computing device. In other embodiments, the eye tracking system may be external 716 to a computing device, such as in the case of an eye tracking system utilizing one or more cameras external to a conventional monitor or television used to display a graphical user interface.
  • The eye tracking system may utilize any suitable components. For example, in some embodiments, the eye tracking system may utilize one or more light sources configured to create glint reflections from the cornea of an eye, and one or more image sensors configured to acquire an image of the eye. A gaze direction may be determined from relative locations of the pupil of the eye and glints from the light sources. The gaze direction may be used, along with information regarding a distance from a user's eye to the display, to determine a location at which the gaze intersects the display. Such distance information may be based upon a predetermined estimate or calibration, or upon real-time depth information (e.g. from a stereo camera configuration).
  • Further, the input subsystem 710 may include other input mechanisms. For example, the input subsystem 710 may include one or more motion sensors 718 (e.g. incorporated into a head-mounted display or other portable device) and/or one or more image sensors 720 (e.g. one or more outward-facing image sensors, as well as inward-facing image sensors for eye tracking). It will be understood that these embodiments of user input devices that provide periodically updated positions signals are presented for the purpose of example, and are not intended to be limiting in any manner.
  • The computing system 700 also includes a logic subsystem 722, and a storage subsystem 724. The storage subsystem 724 may include stored instructions executable by the logic subsystem 722 to perform various tasks related to presenting a graphical user interface and receiving and processing gaze tracking information, for example, to identify potential instances in which a user struggles to view user interface items. Example computing systems are described in more detail below.
  • The examples of FIGS. 1-5 are shown in the context of a head-mounted display. However, gaze patterns may be used to detect user interface perceptibility issues in any suitable hardware context. FIGS. 8 and 9 illustrate another embodiment in which a gaze tracking subsystem includes cameras positioned in a fixed environmental location relative to a user, rather than being mounted to a head of a user. FIG. 8 shows a computing device 802 communicatively connected to a display device 804, such as a computer monitor or television. The computing device 802 is schematically depicted as being connected to one or more cameras 808 a and 808 b and a glint source 810. The cameras 808 a and 808 b may be arranged to provide images of a user 812 from known positions, and therefore allow a distance from the user's eye to the display to be determined. Further, the glint source 810 may be configured to emit a light toward the pupil of the user 812 for the detection of light reflected from the user's eye. This information may be used in combination with information regarding a location of a user's pupil to determine an eye gaze direction.
  • In FIG. 8, cameras 808 a and 808 b may detect that a gaze 814 of the user 812 is directed to a text item 816. Upon determining that a gaze pattern condition is met, the computing device may control the display device 804 to display a user interface object, as shown in FIG. 9 at 902, to visually enhance the appearance of the text to which the gaze 814 is directed.
  • Similar methods also may be used to enhance the appearance of physical objects being viewed via a see-through augmented reality display device. FIG. 10 illustrates an embodiment in which a physical object 1002 in the form of a book having text is viewed by a user 1004 through a head-mounted display 1006. A gaze tracking subsystem of the head-mounted display device 1006 may track the gaze 1008 of the user 1004. If the gaze tracking subsystem detects a gaze pattern that represents difficulty in perceiving an object, an image of the object at the gaze location (determined, e.g. by projecting determined gaze lines beyond the head mounted display into the physical environment) may be acquired via an imaging device integrated within the head-mounted display 1006 (and/or an imaging device positioned remotely from the head-mounted display 1006). The head mounted display device 1006 may then display a user interface object 1010 that visually enhances the appearance of the portion of the object 1002 to which the gaze 1008 is directed, for example, by displaying a magnified and/or otherwise altered view of the object or object portion. In the depicted embodiment, physical text at the location of user focus is displayed in magnified form via the user interface object 1010, but it will be understood that any suitable visual enhancement may be provided, including but not limited to those described above. As more specific examples, enhancements may include changing the brightness, contrast, and/or white balance of an image of the physical object in the user interface object 1010.
  • In some embodiments, the methods and processes described herein may be tied to a computing system of one or more computing devices. In particular, such methods and processes may be implemented as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product.
  • FIG. 11 schematically shows a non-limiting embodiment of a computing system 1100 that can enact one or more of the methods and processes described above. Computing system 1100 is shown in simplified form. Computing system 1100 may take the form of one or more personal computers, server computers, tablet computers, wearable computers (e.g. head-mounted display devices), home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices.
  • Computing system 1100 includes a logic machine 1102 and a storage machine 1104. Computing system 1100 may optionally include a display subsystem 1106, input subsystem 1108, communication subsystem 1110, and/or other components not shown in FIG. 11.
  • Logic machine 1102 includes one or more physical devices configured to execute instructions. For example, the logic machine may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.
  • The logic machine may include one or more processors configured to execute software and/or machine readable instructions. Additionally or alternatively, the logic machine may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. Processors of the logic machine may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and/or distributed processing. Individual components of the logic machine optionally may be distributed among two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the logic machine may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.
  • Storage machine 1104 includes one or more physical devices configured to hold instructions executable by the logic machine to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage machine 1104 may be transformed—e.g., to hold different data.
  • Storage machine 1104 may include removable and/or built-in devices. Storage machine 1104 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage machine 1104 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.
  • It will be appreciated that storage machine 1104 includes one or more physical devices. However, aspects of the instructions described herein alternatively may be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device.
  • Aspects of logic machine 1102 and storage machine 1104 may be integrated together into one or more hardware-logic components. Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP/ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.
  • The terms “module” and “engine” may be used to describe an aspect of computing system 1100 implemented to perform a particular function. In some cases, a module or engine may be instantiated via logic machine 1102 executing instructions held by storage machine 1104. It will be understood that different modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms “module” and “engine” may encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.
  • When included, display subsystem 1106 may be used to present a visual representation of data held by storage machine 1104. This visual representation may take the form of a graphical user interface (GUI). For example, the display subsystem 1106 may be used to present the graphical user interface 104 of FIG. 1. As the herein described methods and processes change the data held by the storage machine, and thus transform the state of the storage machine, the state of display subsystem 1106 may likewise be transformed to visually represent changes in the underlying data. Display subsystem 1106 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with logic machine 1102 and/or storage machine 1104 in a shared enclosure, or such display devices may be peripheral display devices.
  • When included, input subsystem 1108 may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may comprise or interface with selected natural user input (NUI) componentry. Such componentry may be integrated or peripheral, and the transduction and/or processing of input actions may be handled on- or off-board. Example NUI componentry may include a microphone for speech and/or voice recognition; an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity.
  • When included, communication subsystem 1110 may be configured to communicatively couple computing system 1100 with one or more other computing devices. Communication subsystem 1110 may include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network. In some embodiments, the communication subsystem may allow computing system 1100 to send and/or receive messages to and/or from other devices via a network such as the Internet.
  • It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.
  • It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.
  • The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.

Claims (20)

1. On a computing device, a method of selectively enhancing content displayed on a display device, the method comprising:
displaying a graphical user interface comprising one or more items;
tracking a gaze location at which a gaze of a user intersects the graphical user interface; and
upon detecting a gaze pattern that meets a defined gaze pattern condition indicative of difficulty in perceiving an item at the gaze location, displaying the item at the gaze location in a visually enhanced form in a user interface object displayed separately from the item at the gaze location.
2. The method of claim 1, wherein displaying the item at the gaze location in the visually enhanced form comprises displaying the item at the gaze location in a magnified form.
3. The method of claim 1, wherein the item at the gaze location comprises a text item, and wherein displaying the item at the gaze location in the visually enhanced form comprises displaying the text item in one or more of a different font, a different font size, a different font style, a different font color, and/or against a different background color.
4. The method of claim 1, the method further comprising applying a contextual filter to determine a likelihood of the user having difficulty in perceiving the particular item.
5. The method of claim 4, wherein applying a contextual filter comprises determining a confidence score via the contextual filter, and wherein the defined gaze pattern condition comprises a condition in which the confidence score meets a threshold condition.
6. The method of claim 4, wherein applying a contextual filter further comprises applying a contextual filter based upon one or more of a font size, a font type, a font style, a font color, and a background color.
7. The method of claim 1, wherein detecting the gaze pattern comprises one or more of comparing gaze path data to pre-defined gaze patterns, comparing gaze path data to prior patterns of the current user, and evaluating gaze path data via a classification function.
8. The method of claim 1, further comprising ceasing display of the user interface object in response to detecting that the gaze of the user has moved from the item.
9. The method of claim 1, wherein the item at the gaze location comprises one or more of image data and video data.
10. A computing device, comprising:
a display device;
a gaze tracking subsystem;
a logic machine; and
a storage machine storing instructions executable by the logic machine to
display on the display device a graphical user interface comprising one or more items;
track via the gaze tracking system a gaze location at which a gaze of the user intersects the graphical user interface; and
upon detecting a gaze pattern that meets a defined gaze pattern condition indicative of difficulty in perceiving an item at the gaze location, display the item at the gaze location with a visually enhanced appearance via a user interface object displayed separately from the item at the gaze location.
11. The computing device of claim 10, wherein the computing device comprises a head-mounted display device.
12. The computing device of claim 10, wherein the gaze tracking subsystem is at a fixed environmental location relative to a user.
13. The computing device of claim 10, wherein the instructions are executable to detect a gaze pattern by one or more of comparing gaze path data to pre-defined gaze patterns, comparing gaze path data to prior patterns of the current user, and evaluating gaze path data via a classification function.
14. The computing device of claim 10, wherein the instructions are executable to display the item at the gaze location with the visually enhanced appearance by displaying the item with a magnified appearance.
15. The computing device of claim 10, wherein the item at the gaze location is a text item, and wherein the instructions are executable to display the item at the gaze location with the visually enhanced appearance by displaying the text item in one or more of a different font, a different font size, a different font style, a different font color, and/or against a different background color.
16. The computing device of claim 10, wherein the item at the gaze location comprises one or more of a video item and an image item.
17. On a computing device, a method of selectively enhancing an appearance of an object, the method comprising:
tracking a gaze location at which a gaze of the user intersects the object;
detecting a gaze pattern that meets a defined gaze pattern condition; and
displaying on a graphical user interface an image of the object in a visually enhanced form via a user interface object displayed separately from the object at the gaze location.
18. The method of claim 17, wherein the object comprises a virtual object displayed on a graphical user interface.
19. The method of claim 17, wherein the object comprises a physical object viewable through a see-through display of the computing device.
20. The method of claim 17, wherein the computing device comprises a head-mounted display device.
US13/911,867 2013-06-06 2013-06-06 Visual enhancements based on eye tracking Active 2033-09-01 US9965062B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/911,867 US9965062B2 (en) 2013-06-06 2013-06-06 Visual enhancements based on eye tracking

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US13/911,867 US9965062B2 (en) 2013-06-06 2013-06-06 Visual enhancements based on eye tracking
CN201480032122.7A CN105339868B (en) 2013-06-06 2014-06-03 Vision enhancement based on eyes tracking
PCT/US2014/040584 WO2014197409A1 (en) 2013-06-06 2014-06-03 Visual enhancements based on eye tracking
EP14737366.6A EP3005035A1 (en) 2013-06-06 2014-06-03 Visual enhancements based on eye tracking

Publications (2)

Publication Number Publication Date
US20140361971A1 true US20140361971A1 (en) 2014-12-11
US9965062B2 US9965062B2 (en) 2018-05-08

Family

ID=51168349

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/911,867 Active 2033-09-01 US9965062B2 (en) 2013-06-06 2013-06-06 Visual enhancements based on eye tracking

Country Status (4)

Country Link
US (1) US9965062B2 (en)
EP (1) EP3005035A1 (en)
CN (1) CN105339868B (en)
WO (1) WO2014197409A1 (en)

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150042552A1 (en) * 2013-08-06 2015-02-12 Inuitive Ltd. Device having gaze detection capabilities and a method for using same
US20150169048A1 (en) * 2013-12-18 2015-06-18 Lenovo (Singapore) Pte. Ltd. Systems and methods to present information on device based on eye tracking
US20150234459A1 (en) * 2014-01-24 2015-08-20 Tobii Technology Ab Gaze driven interaction for a vehicle
US20150264299A1 (en) * 2014-03-14 2015-09-17 Comcast Cable Communications, Llc Adaptive resolution in software applications based on dynamic eye tracking
US20160048223A1 (en) * 2013-05-08 2016-02-18 Fujitsu Limited Input device and non-transitory computer-readable recording medium
US20160134863A1 (en) * 2014-11-06 2016-05-12 Intel Corporation Calibration for eye tracking systems
US20160239081A1 (en) * 2013-11-01 2016-08-18 Sony Corporation Information processing device, information processing method, and program
US9535497B2 (en) 2014-11-20 2017-01-03 Lenovo (Singapore) Pte. Ltd. Presentation of data on an at least partially transparent display based on user focus
US20170038835A1 (en) * 2015-08-07 2017-02-09 Tobii Ab Gaze direction mapping
WO2017040214A1 (en) * 2015-09-04 2017-03-09 Sony Interactive Entertainment Inc. Apparatus and method for dynamic graphics rendering based on saccade detection
US9633252B2 (en) 2013-12-20 2017-04-25 Lenovo (Singapore) Pte. Ltd. Real-time detection of user intention based on kinematics analysis of movement-oriented biometric data
US20170169649A1 (en) * 2015-12-11 2017-06-15 Igt Canada Solutions Ulc Enhanced electronic gaming machine with gaze-based dynamic messaging
US20170177076A1 (en) * 2015-12-22 2017-06-22 Delphi Technologies, Inc. Automated vehicle human-machine interface system based on glance-direction
US20170249861A1 (en) * 2016-02-29 2017-08-31 Osterhout Group, Inc. Providing enhanced images for navigation
WO2017127457A3 (en) * 2016-01-18 2017-09-14 Waveshift Llc Evaluating and reducing myopiagenic effects of electronic displays
US20170280024A1 (en) * 2016-03-23 2017-09-28 GM Global Technology Operations LLC Dynamically colour adjusted visual overlays for augmented reality systems
US20170364150A1 (en) * 2015-08-07 2017-12-21 Tobii Ab Gaze direction mapping
US9940900B2 (en) 2013-09-22 2018-04-10 Inuitive Ltd. Peripheral electronic device and method for using same
US10169846B2 (en) 2016-03-31 2019-01-01 Sony Interactive Entertainment Inc. Selective peripheral vision filtering in a foveated rendering system
US10180716B2 (en) 2013-12-20 2019-01-15 Lenovo (Singapore) Pte Ltd Providing last known browsing location cue using movement-oriented biometric data
US10192528B2 (en) 2016-03-31 2019-01-29 Sony Interactive Entertainment Inc. Real-time user adaptive foveated rendering
US10372205B2 (en) 2016-03-31 2019-08-06 Sony Interactive Entertainment Inc. Reducing rendering computation and power consumption by detecting saccades and blinks
US10401952B2 (en) 2016-03-31 2019-09-03 Sony Interactive Entertainment Inc. Reducing rendering computation and power consumption by detecting saccades and blinks
US10438410B2 (en) 2015-12-22 2019-10-08 Intel Corporation Text enhancements for head-mounted displays
US10497096B2 (en) 2013-09-17 2019-12-03 Nokia Technologies Oy Determination of a display angle of a display
EP3403130A4 (en) * 2016-01-12 2020-01-01 Esight Corp Language element vision augmentation methods and devices

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105425971B (en) * 2016-01-15 2018-10-26 中意工业设计(湖南)有限责任公司 A kind of exchange method, device and the near-eye display at eye movement interface
US10303341B2 (en) 2016-05-25 2019-05-28 International Business Machines Corporation Modifying screen content based on gaze tracking and user distance from the screen

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070164990A1 (en) * 2004-06-18 2007-07-19 Christoffer Bjorklund Arrangement, method and computer program for controlling a computer apparatus based on eye-tracking
US20110029918A1 (en) * 2009-07-29 2011-02-03 Samsung Electronics Co., Ltd. Apparatus and method for navigation in digital object using gaze information of user
US20110254865A1 (en) * 2010-04-16 2011-10-20 Yee Jadine N Apparatus and methods for dynamically correlating virtual keyboard dimensions to user finger size
US20140181106A1 (en) * 2012-12-20 2014-06-26 Bradley Horowitz Sharing photos
US20140247273A1 (en) * 2011-10-21 2014-09-04 New York University Reducing visual crowding, increasing attention and improving visual span

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6351273B1 (en) 1997-04-30 2002-02-26 Jerome H. Lemelson System and methods for controlling automatic scrolling of information on a display or screen
US7429108B2 (en) 2005-11-05 2008-09-30 Outland Research, Llc Gaze-responsive interface to enhance on-screen user reading tasks
US8494215B2 (en) 2009-03-05 2013-07-23 Microsoft Corporation Augmenting a field of view in connection with vision-tracking
WO2010118292A1 (en) 2009-04-09 2010-10-14 Dynavox Systems, Llc Calibration free, motion tolerant eye-gaze direction detector with contextually aware computer interaction and communication methods
US20100295774A1 (en) 2009-05-19 2010-11-25 Mirametrix Research Incorporated Method for Automatic Mapping of Eye Tracker Data to Hypermedia Content
CN101943982B (en) 2009-07-10 2012-12-12 北京大学 Method for manipulating image based on tracked eye movements
GB2490864A (en) 2011-05-09 2012-11-21 Nds Ltd A device with gaze tracking and zoom
WO2013033842A1 (en) 2011-09-07 2013-03-14 Tandemlaunch Technologies Inc. System and method for using eye gaze information to enhance interactions

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070164990A1 (en) * 2004-06-18 2007-07-19 Christoffer Bjorklund Arrangement, method and computer program for controlling a computer apparatus based on eye-tracking
US20110029918A1 (en) * 2009-07-29 2011-02-03 Samsung Electronics Co., Ltd. Apparatus and method for navigation in digital object using gaze information of user
US20110254865A1 (en) * 2010-04-16 2011-10-20 Yee Jadine N Apparatus and methods for dynamically correlating virtual keyboard dimensions to user finger size
US20140247273A1 (en) * 2011-10-21 2014-09-04 New York University Reducing visual crowding, increasing attention and improving visual span
US20140181106A1 (en) * 2012-12-20 2014-06-26 Bradley Horowitz Sharing photos

Cited By (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9804671B2 (en) * 2013-05-08 2017-10-31 Fujitsu Limited Input device and non-transitory computer-readable recording medium
US20160048223A1 (en) * 2013-05-08 2016-02-18 Fujitsu Limited Input device and non-transitory computer-readable recording medium
US9563283B2 (en) * 2013-08-06 2017-02-07 Inuitive Ltd. Device having gaze detection capabilities and a method for using same
US20150042552A1 (en) * 2013-08-06 2015-02-12 Inuitive Ltd. Device having gaze detection capabilities and a method for using same
US10497096B2 (en) 2013-09-17 2019-12-03 Nokia Technologies Oy Determination of a display angle of a display
US9940900B2 (en) 2013-09-22 2018-04-10 Inuitive Ltd. Peripheral electronic device and method for using same
US20160239081A1 (en) * 2013-11-01 2016-08-18 Sony Corporation Information processing device, information processing method, and program
US9891707B2 (en) * 2013-11-01 2018-02-13 Sony Corporation Information processing device, information processing method, and program for controlling a state of an application by gaze position
US20150169048A1 (en) * 2013-12-18 2015-06-18 Lenovo (Singapore) Pte. Ltd. Systems and methods to present information on device based on eye tracking
US10180716B2 (en) 2013-12-20 2019-01-15 Lenovo (Singapore) Pte Ltd Providing last known browsing location cue using movement-oriented biometric data
US9633252B2 (en) 2013-12-20 2017-04-25 Lenovo (Singapore) Pte. Ltd. Real-time detection of user intention based on kinematics analysis of movement-oriented biometric data
US20150234459A1 (en) * 2014-01-24 2015-08-20 Tobii Technology Ab Gaze driven interaction for a vehicle
US9817474B2 (en) * 2014-01-24 2017-11-14 Tobii Ab Gaze driven interaction for a vehicle
US9580081B2 (en) 2014-01-24 2017-02-28 Tobii Ab Gaze driven interaction for a vehicle
US10035518B2 (en) 2014-01-24 2018-07-31 Tobii Ab Gaze driven interaction for a vehicle
US10324527B2 (en) 2014-01-24 2019-06-18 Tobii Ab Gaze driven interaction for a vehicle
US20150264299A1 (en) * 2014-03-14 2015-09-17 Comcast Cable Communications, Llc Adaptive resolution in software applications based on dynamic eye tracking
US10264211B2 (en) * 2014-03-14 2019-04-16 Comcast Cable Communications, Llc Adaptive resolution in software applications based on dynamic eye tracking
US20160134863A1 (en) * 2014-11-06 2016-05-12 Intel Corporation Calibration for eye tracking systems
US9936195B2 (en) * 2014-11-06 2018-04-03 Intel Corporation Calibration for eye tracking systems
US9535497B2 (en) 2014-11-20 2017-01-03 Lenovo (Singapore) Pte. Ltd. Presentation of data on an at least partially transparent display based on user focus
US10228763B2 (en) * 2015-08-07 2019-03-12 Tobii Ab Gaze direction mapping
US9829976B2 (en) * 2015-08-07 2017-11-28 Tobii Ab Gaze direction mapping
US10185394B2 (en) 2015-08-07 2019-01-22 Tobii Ab Gaze direction mapping
US20170364150A1 (en) * 2015-08-07 2017-12-21 Tobii Ab Gaze direction mapping
US10114459B2 (en) 2015-08-07 2018-10-30 Tobii Ab Gaze direction mapping
US9870051B2 (en) * 2015-08-07 2018-01-16 Tobii Ab Gaze direction mapping
US20170038835A1 (en) * 2015-08-07 2017-02-09 Tobii Ab Gaze direction mapping
US10331209B2 (en) * 2015-08-07 2019-06-25 Tobii Ab Gaze direction mapping
US9857871B2 (en) * 2015-09-04 2018-01-02 Sony Interactive Entertainment Inc. Apparatus and method for dynamic graphics rendering based on saccade detection
WO2017040214A1 (en) * 2015-09-04 2017-03-09 Sony Interactive Entertainment Inc. Apparatus and method for dynamic graphics rendering based on saccade detection
US20170068312A1 (en) * 2015-09-04 2017-03-09 Sony Computer Entertainment Inc. Apparatus and method for dynamic graphics rendering based on saccade detection
CN107949819A (en) * 2015-09-04 2018-04-20 索尼互动娱乐股份有限公司 Apparatus and method for being rendered based on the motion graphics that saccade detects
US20170169649A1 (en) * 2015-12-11 2017-06-15 Igt Canada Solutions Ulc Enhanced electronic gaming machine with gaze-based dynamic messaging
US10339758B2 (en) * 2015-12-11 2019-07-02 Igt Canada Solutions Ulc Enhanced electronic gaming machine with gaze-based dynamic messaging
US10438410B2 (en) 2015-12-22 2019-10-08 Intel Corporation Text enhancements for head-mounted displays
US9841813B2 (en) * 2015-12-22 2017-12-12 Delphi Technologies, Inc. Automated vehicle human-machine interface system based on glance-direction
US20170177076A1 (en) * 2015-12-22 2017-06-22 Delphi Technologies, Inc. Automated vehicle human-machine interface system based on glance-direction
EP3403130A4 (en) * 2016-01-12 2020-01-01 Esight Corp Language element vision augmentation methods and devices
WO2017127457A3 (en) * 2016-01-18 2017-09-14 Waveshift Llc Evaluating and reducing myopiagenic effects of electronic displays
US20170249861A1 (en) * 2016-02-29 2017-08-31 Osterhout Group, Inc. Providing enhanced images for navigation
CN107229328A (en) * 2016-03-23 2017-10-03 通用汽车环球科技运作有限责任公司 Dynamic color adjustment type vision for augmented reality system is covered
US20170280024A1 (en) * 2016-03-23 2017-09-28 GM Global Technology Operations LLC Dynamically colour adjusted visual overlays for augmented reality systems
US10129439B2 (en) * 2016-03-23 2018-11-13 GM Global Technology Operations LLC Dynamically colour adjusted visual overlays for augmented reality systems
US10192528B2 (en) 2016-03-31 2019-01-29 Sony Interactive Entertainment Inc. Real-time user adaptive foveated rendering
US10372205B2 (en) 2016-03-31 2019-08-06 Sony Interactive Entertainment Inc. Reducing rendering computation and power consumption by detecting saccades and blinks
US10401952B2 (en) 2016-03-31 2019-09-03 Sony Interactive Entertainment Inc. Reducing rendering computation and power consumption by detecting saccades and blinks
US10169846B2 (en) 2016-03-31 2019-01-01 Sony Interactive Entertainment Inc. Selective peripheral vision filtering in a foveated rendering system

Also Published As

Publication number Publication date
CN105339868B (en) 2019-02-05
CN105339868A (en) 2016-02-17
EP3005035A1 (en) 2016-04-13
WO2014197409A1 (en) 2014-12-11
US9965062B2 (en) 2018-05-08

Similar Documents

Publication Publication Date Title
EP2862042B1 (en) User interface interaction for transparent head-mounted displays
CN105393191B (en) Adaptive event identification
US10354449B2 (en) Augmented reality lighting effects
US20160320855A1 (en) Touch fee interface for augmented reality systems
EP2912659B1 (en) Augmenting speech recognition with depth imaging
US9092600B2 (en) User authentication on augmented reality display device
EP3037924A1 (en) Augmented display and glove with markers as us user input device
TWI534654B (en) Method and computer-readable media for selecting an augmented reality (ar) object on a head mounted device (hmd) and head mounted device (hmd)for selecting an augmented reality (ar) object
US9041623B2 (en) Total field of view classification for head-mounted display
US9904055B2 (en) Smart placement of virtual objects to stay in the field of view of a head mounted display
US9024844B2 (en) Recognition of image on external display
US9530252B2 (en) Interactions of virtual objects with surfaces
US9754420B2 (en) Mixed reality interactions
JP2016511863A (en) Mixed reality display adjustment
EP3008567B1 (en) User focus controlled graphical user interface using an head mounted device
US9244539B2 (en) Target positioning with gaze tracking
EP3005030B1 (en) Calibrating eye tracking system by touch input
US9165381B2 (en) Augmented books in a mixed reality environment
KR20160024986A (en) Eye tracking via depth camera
CN105009031A (en) Context-aware augmented reality object commands
US9727136B2 (en) Gaze detection calibration
US9202313B2 (en) Virtual interaction with image projection
EP3172644B1 (en) Multi-user gaze projection using head mounted display devices
US9280972B2 (en) Speech to text conversion
US9761057B2 (en) Indicating out-of-view augmented reality images

Legal Events

Date Code Title Description
AS Assignment

Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034747/0417

Effective date: 20141014

Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:039025/0454

Effective date: 20141014

AS Assignment

Owner name: MICROSOFT CORPORATION, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SALA, PABLO LUIS;REEL/FRAME:036731/0263

Effective date: 20130604

STCF Information on status: patent grant

Free format text: PATENTED CASE