US20150062158A1

US20150062158A1 - Integration of head mounted displays with public display devices

Info

Publication number: US20150062158A1
Application number: US14/012,335
Authority: US
Inventors: Evan Robert HILDRETH; Babak Forutanpour; Phuong Lam Ton
Original assignee: Qualcomm Inc
Current assignee: Qualcomm Inc
Priority date: 2013-08-28
Filing date: 2013-08-28
Publication date: 2015-03-05
Also published as: US20150062159A1; TWI592696B; WO2015031167A1; US9466266B2; TW201514847A; TW201514539A

Abstract

Various arrangements for presenting private information are presented. Private information to be displayed via a head mounted display to a user may be identified. A marker displayed by a public display device may also be identified. This public display device may be visible in a vicinity of the user. The private information and an indication of the marker may be output to the head-mounted display of the user, such that the private information is displayed by the head-mounted display in relation to the marker displayed by the public display device.

Description

CROSS REFERENCES TO RELATED APPLICATIONS

This application is related to U.S. application Ser. No. ______, filed Aug. 28, 2013, entitled “Dynamic Display Markers”, Atty. Dkt. No. 131532U2, which is hereby incorporated in its entirety by reference for all purposes.

BACKGROUND

Privacy, when using a public display device, such as a laptop computer screen or tablet computer, in various environments, can be a concern. For example, while a user may desire to interact with confidential information on a public display device in a public setting, such as at a coffee shop, the user may not want the confidential information to be visible to passers-by, curious onlookers, intentional snoops, or other persons in the user's vicinity.
While a head-mounted display (HMD) presents information such that only the user of the HMD can view the information, use of an HMD may have various drawbacks. For example, the brightness, colors, and resolution of an HMD may be limited as compared to other types of display devices. Further, the more information that is presented by an HMD, the more power that may be consumed by the HMD, thus resulting in the HMD requiring more power (e.g., a higher-capacity battery) or functioning for a shorter period of time between battery charges.

SUMMARY

Various embodiments of presenting private information is presented. In some embodiments, a method for presenting private information is described. The method may include identifying, by a display control system, private information to be displayed. The private information may be viewed via a head mounted display by a user. The method may include identifying, by the display control system, a marker present in public information displayed by a public display device. The public display device may be visible in a vicinity of the user. The method may include outputting, by the display control system, the private information and an indication of the marker to the head mounted display of the user, such that the private information is displayed by the head mounted display in relation to the marker displayed by the public display device.
Embodiments of such a method may include one or more of the following features: The method may include identifying, by the display control system, public information to be displayed by the public display device. The method may include outputting, by the display control system, the public information to the public display device, wherein the public information is displayed by the public display device. At least a portion of the public information may be used as the marker. Information associated with the marker may indicate that the private information is to be displayed by the head mounted display overlaying the public display device. The method may include presenting, by the head mounted display, the private information in relation to the marker, such that the private information overlays a portion of the public display device. The method may include identifying, by the display control system, a second piece of private information to be displayed to a second user, wherein the second piece of private information is to be viewed by the second user to the exclusion of the user. The method may include identifying, by the display control system, a second marker displayed by the public display device, wherein the public display device is visible in the vicinity of the user and a second vicinity of the second user. The method may include outputting, by the display control system, the second piece of private information and a second indication of the second marker to a second head-mounted display, such that the second piece of private information is displayed by the second head-mounted display in relation to the second marker displayed by the public display device. The marker and the second marker may be a same marker. Identifying the private information to be displayed may include receiving, by the display control system, a definition of a rule for identifying the private information from the user. Identifying, by the display control system, the private information to be displayed may include using the rule. The method may include outputting, by the display control system, to the public display device, false information, wherein the private information displayed by the head mounted display is configured to overlay the false information displayed by the public display device.
In some embodiments, a system for presenting private information is described. The system may include one or more processors. The system may include a memory communicatively coupled with and readable by the one or more processors.
The memory may have stored therein processor-readable instructions which, when executed by the one or more processors, cause the one or more processors to identify private information to be displayed, wherein the private information is to be viewed via a head mounted display by a user. The processor-readable instructions may further cause the one or more processors to identify a marker present in public information displayed by a public display device, wherein the public display device is visible in a vicinity of the user. The processor-readable instructions may further cause the one or more processors to output the private information and an indication of the marker to the head mounted display of the user, such that the private information is displayed by the head mounted display in relation to the marker displayed by the public display device.
Embodiments of such a system may include one or more of the following features: The processor-readable instructions may further cause the one or more processors to identify public information to be displayed by the public display device. The processor-readable instructions may further cause the one or more processors to output the public information to the public display device, wherein the public information is displayed by the public display device. At least a portion of the public information may be used as the marker. Information associated with the marker may indicate that the private information is to be displayed by the head mounted display overlaying the public display device. The system may further include the head mounted display, configured to present the private information in relation to the marker, such that the private information overlays a portion of the public display device. The processor-readable instructions, when executed by the one or more processors, may further cause the one or more processors to identify a second piece of private information to be displayed to a second user, wherein the second piece of private information is to be viewed by the second user to the exclusion of the user.
Additionally or alternatively, embodiments of such a system may include one or more of the following features: The processor-readable instructions may further cause the one or more processors to identify a second marker displayed by the public display device, wherein the public display device is visible in the vicinity of the user and a second vicinity of the second user. The processor-readable instructions may further cause the one or more processors to output the second piece of private information and a second indication of the second marker to a second head-mounted display, such that the second piece of private information is displayed by the second head-mounted display in relation to the second marker displayed by the public display device. The marker and the second marker may be a same marker. The instructions that cause the one or more processors to identify the private information to be displayed may include processor-readable instructions which, when executed, cause the one or more processors to receive a definition of a rule for identifying the private information from the user. The processor-readable instructions configured to cause the one or more processors to identify the private information to be displayed may include processor-readable instructions configured to cause the one or more processors to identify the private information to be displayed using the rule. The processor-readable instructions, when executed by the one or more processors, may further cause the one or more processors to output, to the public display device, false information, wherein the private information displayed by the head mounted display is configured to overlay the false information displayed by the public display device.
In some embodiments, a non-transitory computer-readable medium for presenting private information is described. The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to identify private information to be displayed, wherein the private information is to be viewed via a head mounted display by a user. The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to identify a marker present in public information displayed by a public display device, wherein the public display device is visible in a vicinity of the user. The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to output the private information and an indication of the marker to the head mounted display of the user, such that the private information is displayed by the head mounted display in relation to the marker displayed by the public display device.
Embodiments of such a non-transitory computer-readable medium may include one or more of the following features: The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to identify public information to be displayed by the public display device. The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to output the public information to the public display device, wherein the public information is displayed by the public display device. At least a portion of the public information may be used as the marker. Information associated with the marker may indicate that the private information is to be displayed by the head mounted display overlaying the public display device. The processor-readable instructions may be further configured to cause the one or more processors to identify a second piece of private information to be displayed to a second user, wherein the second piece of private information is to be viewed by the second user to the exclusion of the user. The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to identify a second marker displayed by the public display device, wherein the public display device is visible in the vicinity of the user and a second vicinity of the second user.
Additionally or alternatively, embodiments of such a non-transitory computer-readable medium may include one or more of the following features: The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to output the second piece of private information and a second indication of the second marker to a second head-mounted display, such that the second piece of private information is displayed by the second head-mounted display in relation to the second marker displayed by the public display device. The marker and the second marker may be a same marker. The instructions configured cause the one or more processors to identify the private information to be displayed may include processor-readable instructions configured to cause the one or more processors to receive a definition of a rule for identifying the private information from the user, wherein the processor-readable instructions configured to cause the one or more processors to identify the private information to be displayed comprises processor-readable instructions configured to cause the one or more processors to identify the private information to be displayed using the rule. The non-transitory computer-readable medium may include processor-readable instructions configured to cause one or more processors to output, to the public display device, false information, wherein the private information displayed by the head mounted display is configured to overlay the false information displayed by the public display device.
In some embodiments, an apparatus for presenting private information is described. The apparatus may include means for identifying private information to be displayed, wherein the private information is to be viewed via a means for presenting private information by a user. The apparatus may include means for identifying a marker present in public information displayed by a means for presenting public information, wherein the means for presenting public information is visible in a vicinity of the user. The apparatus may include means for outputting the private information and an indication of the marker to the means for presenting private information of the user, such that the private information is displayed by the means for presenting private information in relation to the marker displayed by the means for presenting public information.
Embodiments of such an apparatus may include one or more of the following features: The apparatus may include means for identifying public information to be displayed by the means for presenting public information. The apparatus may include means for outputting the public information to the means for presenting public information, wherein the public information is displayed by the means for presenting public information. At least a portion of the public information may be used as the marker. Information associated with the marker may indicate that the private information is to be displayed by the means for presenting private information overlaying the means for presenting public information. The apparatus may include means for presenting the private information in relation to the marker, such that the private information overlays a portion of the means for presenting public information. The apparatus may include means for identifying a second piece of private information to be displayed to a second user, wherein the second piece of private information is to be viewed by the second user to the exclusion of the user. The apparatus may include means for identifying a second marker displayed by the means for presenting public information, wherein the means for presenting public information is visible in the vicinity of the user and a second vicinity of the second user. The apparatus may include means for outputting the second piece of private information and a second indication of the second marker to a second means for presenting private information, such that the second piece of private information is displayed by the second means for presenting private information in relation to the second marker displayed by the means for presenting public information. The marker and the second marker may be a same marker. The means for identifying the private information to be displayed may include means for receiving a definition of a rule for identifying the private information from the user, wherein the means identifying the private information to be displayed comprises means for using the rule. The apparatus may include means for outputting to the public display device, false information, wherein the private information displayed by the means for presenting private information is configured to overlay the false information displayed by the means for presenting public information.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of various embodiments may be realized by reference to the following figures. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

FIG. 1A illustrates an embodiment of a system for presenting private information using a private display in conjunction with a public display to a user.

FIG. 1B illustrates an embodiment of a system for presenting different sets of private information to multiple users using multiple private displays and a single public display.

FIG. 2 illustrates an embodiment of a computer system for identifying, marking, and presenting private information.

FIG. 3A illustrates an embodiment of a scene in which a user is viewing private information in conjunction with a public display.

FIG. 3B illustrates an embodiment of private information overlaid by a private display device onto public information presented by a public display device.

FIG. 3C illustrates an embodiment of how a public display device may appear when viewed by a person not wearing private display device.

FIG. 3D illustrates another embodiment of how a public display device may appear when viewed by a person not wearing a private display device.

FIG. 4 illustrates an embodiment of a scene in which multiple users are viewing different sets of private information and viewing a common public display.

FIG. 5 illustrates an embodiment of a method for presenting private information using a private display device in conjunction with a public display device.

FIG. 6 illustrates another embodiment of a method for presenting private information using a private display device in conjunction with a public display device.

FIG. 7 illustrates an embodiment of a computer system.

DETAILED DESCRIPTION

In a conventional arrangement, a user may use a single display device to display information of which some is private, such as the content of a document (e.g., an email), a markup of a document, credit card information, or cards for a card game, and some is public (e.g., graphics which the user does not care if other persons view). In contrast, embodiments detailed herein are directed to integrating the use of a private display device, such as a head-mounted display (HMD), with use of a public display device. To prevent unauthorized viewing of private information, an HMD may be used to supplement information displayed by a public display device. A public display device may be a laptop computer screen, a computer monitor, a table computer's display screen, an overhead projector, a television, or some other type of display device. A public display device can be defined as a display device that permits multiple users to simultaneously view content presented by the public display device. For instance, a laptop computer screen could be a public display device because two persons can view content presented by the laptop computer screen at a same time. However, an HMD is not a public display device because an HMD is configured to be worn and used by a single user at a time.
In embodiments presented herein, information to be presented to a user may be designated as public or private. Public information, that is, information which is not considered of a private nature, may be presented by the public display device to the user. It may be of little consequence to the user as to whether another person views this public information. For example, icons, web pages, web graphics, videos, operating system information, and/or background graphics are basic examples of information which may be considered public information. Such public information may be presented in a higher resolution, higher brightness level, and/or higher contrast display format than is available via an HMD of the user. Private information may be information that is considered of a private nature to the user. For example, correspondence (e.g., text of an email), biographical data, passwords, comments, financial account information, and the content of documents (e.g., stored text) may be considered private information. Private information may be presented by an HMD directly to the user such that other persons who can view the public display device cannot see the private information. As an example, if the user is writing an email, the address, subject, and text of the email may be treated as private information while all other displayed information may be treated as public information. A person (other than the user wearing the HMD) viewing the public display device may see the public information but not the private information.
Information that is tagged (or otherwise designated) as private may be transmitted to a private display device of the user for display and viewing, but not to the public display device. The private information may be linked with a marker that is also transmitted to the private display device. A marker, which also may be referred to as a fiduciary or anchor, may be a visual indicator used by the private display device as a reference for determining where to display the private information in relation to the displayed public information. The computer system that identifies information as public or private may designate a marker among the public information displayed by the public display device. This marker, which may be a portion of the public information displayed, may be used to properly position the private information for display by the private display device. For instance, in many embodiments, the private information presented by the private display device may be overlaid on the user's view of the public display device. Coordinates, a vector, or other location information may be used in combination with the marker to determine where in the user's field-of-vision the private information should be displayed by the private display device. As the public information presented to the user by the public display device changes, the marker may be updated such that the marker is present in the public information currently being presented by the public display device. If the location of the marker is changed, the vector, coordinates, or other location information may also be changed.
The private information and markers may be transmitted to the private display device such that the private display device presents the private information to the user overlaying the user's view of the public display device. By private information overlaying a portion of the user's view, objects directly behind the private information may be partially visible. In some embodiments, the private information may be less than fully opaque, allowing the private information and objects over which the private information is overlaid to be visible. Therefore, from the user's point-of-view, it may appear as if the private information is being presented on or by the public display device. However, to a person other than the user, the private information is not visible (because the information is being presented to the user via the private display device). Returning to the example of an email message, the user may be able to view the address, subject and message text overlaid on the corresponding graphical fields of the public display device, regardless of the private display device's orientation to the public display device (e.g., as the user's head moves, the private display device keeps the private information overlaying the public display device).
The presentation of private information may be on a user-by-user basis. For example, multiple users each using a private display device may view a public display device. Depending on each user's permissions, the private information presented to each user may vary. For instance, a first user may be able to view a first set of private information in conjunction with public information displayed by the public display device while a second user is presented a second set of private information displayed in conjunction with the public information displayed by the public display device. The second set of private information may represent different private information and/or a subset of the private information than the private information presented to the first user. The same or different markers presented by the public display device may be used for each user. For instance, the markers used as references for each HMD may be determined as detailed in relation to the application incorporated by reference.
In some embodiments, the public display device may leave blank space where private information is to be presented to the user by the private display device. For example, a dark colored region (e.g., black region) may be displayed by the public display device to use as a backdrop where private information is overlaid by the private display device. . In some embodiments, the public display device may display public information where private information is to be presented to the user by the private display device. For example, if displaying an electronic version of cards, such as poker, the backs of cards (so that the cards appear to be face-down) may be presented by the public display, onto which the private display device may overlay the fronts of cards (so that the cards appear to be face-up). In other embodiments, false information may be presented by the public display device in a location to be overlaid with private information for the user. This false information may be overlaid by private information presented by the private display device, thus obscuring the false information to the user. A person other than the user may see the false information, while the user will see the correct, private information. The presence of the false information may help conceal from persons other than the user that the user is not displaying all information via the public display device or the false information may serve to purposefully mislead the persons (e.g., during an electronic version of a card game or when financial account information is presented).
FIG. 1A illustrates an embodiment of a system 100A for presenting private information to a user using a private display device in conjunction with a public display device. System 100A may include a display control system 110, a public display device 120, and a private display device 130-1. Display control system 110 may be configured to execute an operating system, one or more applications, and/or receive information to be presented from an external source. The operating system and/or one or more applications may output information to be displayed visually to the user. More detail regarding various embodiments of display control system 110 is provided in relation to FIG. 2. Display control system 110 may be in communication with at least two display devices: public display device 120 and private display device 130-1. In the illustrated embodiment of system 100A, display control system 110 is in wireless communication with private display device 130-1 and is in wired communication with public display device 120. Such an arrangement may vary by embodiment. Wireless or wired communication may be possible between public display device 120, private display device 130-1, and display control system 110.
Information determined to be public may be transmitted by display control system 110 to public display device 120. Public display device 120 may be a display device capable of being simultaneously viewed by multiple persons in a vicinity of public display device 120. For example, common public display devices can include laptop computer screens, tablet computer screens, mobile phone (smartphone) screens, televisions, projectors, and computer monitors. Each of these types of public display devices presents information such that multiple persons can look at the screen simultaneously. Display control system 110 may assess whether information should be displayed as public or private information. Information, which may include graphics and text, that is to be publicly displayed, may be transmitted by display control system 110 to public display device 120.
In many scenarios involving system 100A, the vast majority of information to be presented to a user may be public and thus may be presented to the user via public display device 120. For example, if a user is viewing a public web page and composing an email, the entire web page, the user's desktop background, and the user's start menu (to name only a few examples), may all be presented via public display device 120. Information presented by public display device 120 is represented as being visible to the user by the dotted arrow to user 101. Such information presented by public display device 120 may also be visible to persons 102, who may or may not be present and/or attempting to view public display device 120. User 101 may or may not be aware that one or more of persons 102 are viewing public display device 120 or are in the vicinity. Persons 102 may not be using any private display device or may be using one or more private display devices that are not authorized to receive the private information presented to user 101 via private display device 130-1.
Information determined to be private may be transmitted by display control system 110 to private display device 130-1 for display. Private display device 130-1 may be a display device capable of being viewed by only a single user of private display device 130-1 at a given time. For example, a private display device can be an HMD. An HMD may present information (e.g., graphics and/or text) such that only user 101 who is wearing (or otherwise using) private display device 130-1 can view the information presented by private display device 130-1. While persons 102 may be aware that user 101 is wearing private display device 130-1, it may be difficult or impossible for persons 102 to effectively view the information presented by private display device 130-1 to user 101. Display control system 110 may assess whether information should be presented as public or private information to a user; alternatively, in some embodiments, information for display received by display control system 110 may be tagged as public or private information. In such embodiments, display control system 110 may serve to route the information for display appropriately. Information, which may include graphics and text, that is to be privately displayed, may be transmitted by display control system 110 to private display device 130-1; in some cases such transmission may be wireless.
Since, in many scenarios involving system 100A, the vast majority of information to be presented to a user may be public and thus may be presented to the user via public display device 120, only small snippets of private information may be present. Returning to the previous example, if a user is viewing a public web page and composing an email, the entire public web page, the user's desktop background, and the user's start menu (to name only a few examples), may all be presented via public display device 120. The private information presented by private display device 130-1 may include text of the email (including email addresses and/or recipients indicated in the “to,” “from,” and “cc” fields, the subject line, and the body of the email), and/or some pieces of information presented about the user in the web page (e.g., a user name, account number, credit card number). Information presented by private display device 130-1 is represented as being visible to only the user by the dotted arrow to user 101 from private display device 130-1. Such private information is not visible to persons 102.
To enhance the user experience of user 101, as much information may be presented to user 101 via public display device 120 as possible without compromising the privacy of user 101. Public display device 120 may have a higher resolution, a higher contrast ratio, a higher refresh rate, more accurate color reproduction, and/or have access to power more readily than private display device 130-1. As such, it may be desirable to minimize the amount of information presented to user 101 via private display device 130-1. In many instances, information that is identified as private and is to be presented to user 101 by private display device 130-1 may typically be in the form of text, while video and graphics may tend to be presented to user 101 via public display device 120. Generally, HMDs require one or more batteries for power (such that the HMDs can be wireless). The more information presented by an HMD to user 101, the greater the amount of power that is consumed by the HMD. Therefore, to reduce the capacity of the one or more batteries of the HMD and/or increase the battery life of the HMD, the amount of information presented to user 101 via an HMD may be desired to be minimized.
User 101 may view public display device 120 and private display device 130-1 simultaneously. In some embodiments, information presented to user 101 via private display device 130-1 is overlaid (e.g., superimposed) on the user's view of public display device 120. Again, returning to the previous example, if the user is composing an e-mail, the graphics for the e-mail program may be presented by public display device 120. However, the text of the e-mail may be presented to user 101 by private display device 130-1. To user 101, the text of the e-mail presented by private display device 130-1 may appear as part of the e-mail program presented by public display device 120.
FIG. 1B illustrates an embodiment of a system 100B for presenting different sets of private information to multiple users. In system 100B, multiple users 103 may be viewing a same public display device 120, but may each be using a private display device that allows users 103 to view the same and/or different private information. Further, one or more persons 102 may be present who can view only the public information presented by public display device 120. System 100B may include a display control system 110, a public display device 120, and private display devices 130. Display control system 110 may be configured to execute an operating system and one or more applications. The operating system and/or one or more applications may output information to be displayed to the multiple users. More detail regarding various embodiments of display control system 110 is provided in relation to FIG. 2. Display control system 110 may be in communication with at least three display devices: public display device 120 and two or more private display devices (e.g., private display device 130-1, private display device 130-2, and private display device 130-3). In the illustrated embodiment of system 100B, display control system 110 is in wireless communication with private display devices 130 and wired communication with public display device 120. Such an arrangement may vary by embodiment. Wireless or wired communication may be possible between public display device 120, private display devices 130, and display control system 110.
Information determined to be acceptable for public display may be transmitted by display control system 110 to public display device 120. Such information may be viewed by users 103 and, if present, one or more persons 102 who may or may not be looking at public display device 120. As previously noted, public display device 120 may be a display device capable of being viewed by multiple persons in a vicinity of public display device 120 such as laptop computer screens, tablet computer screens, mobile phone (smartphone) screens, televisions, projectors, and computer monitors. Each of these public display devices presents information such that multiple persons may view displayed information simultaneously. Display control system 110 may assess whether information should be presented as public or private information to a user. Information, which may include graphics and text, that is to be publicly displayed, may be transmitted by display control system 110 to public display device 120.
In many scenarios involving system 100B, the vast majority of information to be presented to users 103 may be public and thus may be presented to the users via public display device 120. For example, if users 103 are playing an electronic version of cards, such as poker, the graphics for the card game may be presented via public display device 120. Information which may not be presented by public display device 120 includes the specific cards each user of users 103 holds. Information presented by public display device 120 is represented as being visible to the users by the dotted arrows to users 103 from public display device 120. Such information presented by public display device 120 may also be visible to persons 102, which is also represented by a dotted arrow from public display device 120 to persons 102. One or more users of users 103 may or may not be aware that one or more of persons 102 are viewing public display device 120. Persons 102 may not be using any private display device or may be using one or more private display devices that are not authorized to receive the private information presented to users 103 via private display devices 130.
Information determined to be private may be transmitted by display control system 110 to one or more of private display devices 130. Private display devices 130 may be display devices capable of being viewed by only a single user at a time. For example, a private display device can be a head-mounted display. An HMD may present information (e.g., graphics and/or text) such that only the user who is wearing (or otherwise using) a private display device can view information presented by the private display device. While persons 102 and one or more other users of users 103 may be aware that a user is wearing a private display device, it may be difficult or impossible for persons 102 or other users of users 103 to effectively view information presented by the private display device to the user. Display control system 110 may assess whether information should be presented as public or private information to a user. Information, which may include graphics and text, that is to be privately displayed, may be transmitted by display control system 110 to the appropriate one or more private display devices of private display devices 130. In some embodiments, such transmission is wireless.
Each user of users 103 may view the same or different private information. Display control system 110 may have identification information to determine which user of users 103 is using which private display device of private display devices 130. Therefore, private information intended for user 103-1 may be presented only by private display device 130-1; private information intended for user 103-2 may be presented only by private display device 130-2; and finally private information intended for user 103-3 may be presented only by private display device 130-3. Returning to the example of a card game, each private display device of private display devices 130 may present the card numbers (and graphics) specific to its user. Therefore, private display device 130-2 may only present the card numbers for cards held by a user 103-2. Each of users 103 and, possibly, persons 102 may view on public display device 124 other text and/or graphics associated with the card game (e.g., such as a green felt background and an animated representation of a dealer). In some embodiments, multiple private display devices of private display device 130 may present the same private information. For example, during a presentation, user 103-1 and user 103-3 may view the same private information while a user 103-2 is presented either a subset of the private information presented to users 103-1 and 103-3 or different private information and possibly persons 102 are viewers of the presentation not permitted to view any of the private information.
To enhance the user experience of users 103, as much information may be presented to users 103 via public display device 120 as possible without compromising the privacy of each user of users 103. Public display device 120 may have a higher resolution, a higher contrast ratio, a higher refresh rate, and/or more accurate color reproduction then private display device 130-1. As such, it may be desirable to decrease (e.g., minimize) the amount of information presented to users 103 via private display devices 130. Each user of users 103 may view public display device 120 and the user's private display device of private display devices 130 simultaneously. In some embodiments, information presented to users 103 via private display devices 130 may be overlaid on each user's view of public display device 120, thereby giving each user the impression of viewing a single display device.
FIG. 2 illustrates an embodiment of a display control system 110 for identifying, marking, and presenting private information. Display control system 110 of FIG. 2 represents display control system 110 of FIGS. 1A and 1B. However, it should be understood, that alternate embodiments of display control system 110 may be used in systems 100A and 100B of FIGS. 1A and 1B, respectively. Further, display control system 110 of FIG. 2 may be implemented in conjunction with systems other than systems 100A and 100B of FIGS. 1A and 1B, respectively. Display control system 110 of FIG. 2 contains various components. Such components may be implemented using hardware, software, and/or firmware. The functions of various components may be performed based on instructions executed by general purpose processors. For instance, the functions of some components of display control system 110 may be performed by the components of computer system 700 of FIG. 7. Display control system 110 may include: data classification analysis engine 210, user interface environment 220, display analysis rules database 230, user credential database 235, marker definition engine 240, public data display modification engine 250, private data display modification engine 260, public display interface 270, and private display interface 280.
Data classification analysis engine 210 may receive information (e.g., text, graphics) to be displayed from user interface environment 220. User interface environment may comprise software being executed by display control system 110. User interface environment may include an operating system 222 and/or one or more applications being executed, such as application 224. User interface environment may output visual information that is to be presented to one or more users. The visual information output by user interface environment 220 may or may not include tagged information. If tagged, pieces of visual information may be labeled as public or private. For instance, application 224 may assess information it outputs for privacy. Privacy as determined by application 224 may be based on how a developer of application 224 has labeled various categories of information that may be output for display by application 224 or user preferences, such as in display analysis rules database 230. For instance, whenever an account number is output for display by application 224, application 224 may label the account number as private. In other embodiments, an application may simply label all text output for display as private. Similarly, just as an application such as application 224 may label information to be displayed as public or private, operating system 222 may label information output for display by operating system 222 and/or may label information output by applications being executed through operating system 222 as either private or public.
In some embodiments, display control system 110 may receive information to be displayed from an external source. Rather than information to be displayed being generated by the user interface environment of display control system 110, a separate computer system or other data source may be in communication with display control system 110. Information received from this external source may be processed via user interface environment 220 and/or data classification analysis engine 210.
Data classification analysis engine 210 may identify whether information to be displayed should be categorized as public or private and/or may route data based on the privacy of the data. Data classification analysis engine 210 (and other components of display control system 110) are illustrated as being operated outside of user interface environment 220. In some embodiments, it should be understood that data classification analysis engine 210 and/or other components of display control system 110 may be in the form of software executed via operating system 222. For instance, data classification analysis engine 210 may be a type of application. In other embodiments, such components may be implemented using firmware and/or specialized hardware.
If operating system 222 or application 224 has labeled data to be displayed as public or private, data classification analysis engine 210 may serve to properly route the public data and private data for display. Public data may be routed to public data display modification engine 250. In some embodiments, data for display received by data classification analysis engine 210 may not be labeled as either public or private. Data classification analysis engine 210 may analyze the data for display and determine whether the data should be classified as public or private. In some embodiments, text will treated as private information while graphics will be treated as public information. Such a rule may be useful for a user desiring a high level of privacy. If a user is only desirable of a lower level of privacy, only numbers, email addresses, addresses, and passwords (for example) may be treated as private information. Data classification analysis engine 210 may use a set of rules to determine information that should be treated as public or private. Display analysis rules database 230 may contain rules that control whether information is presented to the user as public or private. Such rules may be at least partially defined by a user. In some embodiments, a user may select a level of privacy (e.g., high, medium, or low). In some embodiments, the user may define specific rules (e.g., all text output for display by an email program should be treated as private). The rules stored by display analysis rules database 230 may be application-specific or may be used for information output for display by multiple applications.
In some embodiments, the rules followed regarding which information is treated as public or private is based on location. For instance, if the user is determined to be viewing information in his office or home, it may be assumed that the user does not need to protect displayed information from casual onlookers. However, if a person is located at a public location, like a coffee shop or mall, a higher level of privacy may be enforced. In some embodiments, gaze detection may be used to determine if a person other than the user is attempting to look at the user's public display device. For instance, a camera that has a field-of-view of a scene where the public display device is visible may be used to determine if a person other than the user is looking at the user's public display device. If so, private information may be transitioned to being presented via a private display device to the user. In some embodiments, false information may be substituted for the private information to fool the person other than the user looking at the public display device.
Marker definition engine 240 may define a marker (e.g., reference, anchor, fiduciary) for displaying the private information by the private display device in relation to the public information displayed by the public display device. For example, if a user is using an HMD as the private display device, the private information displayed to the user may be desired to be presented in the user's field of vision such that the private data appears to the user as displayed seamlessly with the public data displayed by the public display device. Therefore, as the user's head moves, the private data may need to be repositioned by the private display device such that the user has the impression of the private data remaining in a same location relative to the public information displayed by the public display device.
In order for the private display device to properly position the private information for display in relation to the public information displayed by the public display device, a marker from among the public information being displayed may be defined. The marker may be a visual pattern that can be recognized by the private display device from among the public information presented by the public display device. Techniques for locating a defined marker are known to those with skill in the art and for brevity are not detailed herein. For example, U.S. patent application Ser. No. 12/756,373, entitled “Enhanced Handheld Screen-Sensing Pointer”, which is hereby incorporated by reference, described various techniques for location a defined marker which may be used. Marker definition engine 240 may use public information being displayed by the public display device to be used as the marker. Therefore, marker definition engine 240 may access public information being output for display by public display interface 270 to determine what public information is to be used as a marker. The marker definition engine 240 may analyze the public information for display be public display interface 270, to detect visually detectable features (such as edges, corners, colors, etc) within the public information, and select what public information is to be used as a marker based on the presence of uniquely identifiable features within that public data. Marker data may be in the form of features (such as edges, corners, colors, etc). As the public information output for display by public display interface 270 changes, the public information used as the marker may need to be modified. Further, marker definition engine 240 may include vector data that indicates where private data should be displayed in relation to the defined marker. As such, the vector data may indicate a direction and the distance for the private information to be presented relative to the marker. Such vector data may also be in the form of coordinates or some other form of location information. Marker and vector data defined by marker definition engine 240 may be routed to the private display device of one or more users via private display interface 280. This marker and vector data may be used to position private data received by the private display device for display.
Data that is to be displayed privately to the user may be routed to private data display modification engine 260 by data classification analysis engine 210. Data that is to be displayed publicly to the user may be routed by data classification analysis engine 210 to public data display modification engine 250. Information related to the location of private data to be displayed via private display devices and the public data to be displayed via the public display device may also be routed to marker definition engine 240.
Both public information and private information to be presented by their respective display devices may need to be modified for display by the respective display device. For instance, the resolution of a private display device may be different from the resolution of the public display device. Private data display modification engine 260 may receive private information to be presented by the private display device. Private data display modification engine 260 may modify the private information to be more effectively presented by the private display device. This may include adjusting the contrast, brightness, size, and/or resolution of the private information. Similarly, public data display modification engine 250 may receive public information to be presented by the public display device. Public data display modification engine 250 may modify the public information to be more effectively presented by the public display device and to facilitate display of the private information. This may include adjusting the contrast, brightness, size, and/or resolution of the private information. Public information to be presented behind private information may be modified to facilitate display of the private information. For example, a graphic to be displayed behind private text may be decreased in brightness to allow the text to be more visible against the graphic background. As an example, consider the background graphic of a body section of an email in relation to the text of the email, which may be treated as private information. By decreasing the brightness of at least portions of the public information to be presented, the brightness output by the private display device required for visibility of private information may be decreased, thus possibly resulting in power savings. If public information is to be displayed by the public display device in a location onto which private information is to be overlaid by the private display device, public data display modification engine 250 may create and/or access public information to be presented in that location. As such, this public information may be visible to persons other than the user, but the display of private information may obscure this public information from the user viewing the private information via the private display device. For example, in an electronic version of cards, such as poker, the public information may be the decorative backs of cards, and the private information may be the fronts of cards displayed only to the user “holding” the cards. Further, if false information is to be displayed by the public display device, public data display modification engine 250 may create and/or access false information to be presented in a location which will be overlaid with the (real) private information for the user. As such, this false information may be visible to persons other than the user, but the display of the private information may obscure the false information presented by the public display device. For example, in the example of a financial account number, the false information may be a random number of the same number of digits.
User credential database 235 may be used when multiple users are each using a private display device receiving private data from display control system 110. User credential database 235 may manage which users are permitted to view one or more sets of private data. For instance, different users may be able to view different sets or subsets of private data. Based on the user's permissions, which private data is presented via the user's private display device may vary. In some embodiments, multiple users may be able to view the same private information. Persons without a private display device (or, possibly, without the proper permissions for their private display device) may be unable to see any of the data indicated as private by data classification analysis engine 210.
Private display interface 280 may serve to interface with the private display device of one or more users. For instance, private display interface 280 may serve to interface with the private display device. Private display interface 280 may cause wireless (or wired) transmission of the private data to be displayed, the marker data, and associated vector data to the private display device of the one or more users. Such data may be encrypted. Similarly, public display interface 270 may serve to interface with the public display device being viewed by the user (and, possibly, one or more other persons). Public display interface 270 may output the public information to be displayed by the public display device including the public information that is to be used as the marker for positioning the private information. The public display device may be in wired or wireless communication with public display interface 270. When viewed by the user, private information may be overlaid in the user's field-of-view over at least some of the private information. As such, to the user, it may appear that the public display device is presenting all of the information. However, to another person in the area, the private information being presented to the user via the private display device is not visible, thereby protecting the private information being viewed by the user.
FIG. 3A illustrates an embodiment of a scene 300A in which a user is viewing private information via a private display device in conjunction with a public display device. Scene 300A may be an implementation of system 100A of FIG. 1A. In scene 300A, user 101 is sitting in a coffee shop using his laptop. The user's laptop has a display screen functioning as public display device 120 and the user is using private display device 130-1, which displays information visible to only user 101. In the illustrated embodiment, private display device 130-1 is an HMD. In scene 300A, person 102-1 is attempting to see what is on public display device 120 of the user's laptop. User 101 may or may not be aware that person 102-1 is attempting to see the information being presented by the public display device 120 of the user's laptop.
In the illustrated scene 300A, user 101 is composing an e-mail. This e-mail may contain sensitive materials that user 101 desires to keep confidential. While the user is located in a public location which may have persons, such as person 102-1, attempting to view the user's laptop, the user may be able to maintain privacy by having only public information be presented via public display device 120. Private information, such as the text of the e-mail that user 101 is composing, may be presented to user 101 via private display device 130-1. Therefore, person 102-1 and any other person attempting to view public display device 120 may not see any information that may be private to user 101. However, user 101 may view the private information via private display device 130-1. Further, the private information may appear to be displayed by public display device 120 to user 101 on public display device 120 because private display device 130-1 overlays (or superimposes) the private information on public display device 120. For instance, FIG. 3B represents what user 101 may see when looking at public display device 120 through private display device 130-1. To be clear, a private display device, such as an HMD, can display text and/or graphics on a transparent surface, thus allowing a user to directly view real-world objects through the HMD.
FIG. 3B illustrates an embodiment 300B of private information overlaid by a private display device onto public information presented by a public display device. Embodiment 300B may represent the view of user 101 of FIG. 3A while looking at public display device 120. In embodiment 300B, the majority of the information presented may be considered public information. For instance, the sending and delete graphics in the upper right, the background box for the e-mails body text, and application bar 340 may each be pieces of public information presented by the public display device that are viewed directly from public display device 120 by user 101.
However, text 310-1, which represents e-mail recipients and the subject of the e-mail, and text 310-2, which is the body text of the email, may be treated as private information. Such private information may be presented to user 101 via a private display device, such as private display device 130-1 of FIG. 3A. Private information, such as texts 310 may be superimposed on the public information presented by public display device 120. As such, to user 101, the presentation of the public and private information may have the appearance of being presented by a single display device, while protecting private information from viewing by other persons such as person 102-1.
FIG. 3C illustrates an embodiment 300C of how embodiment 300B may appear when viewed by a person not wearing private display device 130-1. Embodiment 300C may represent the view of person 102-1 of FIG. 3A while looking at public display device 120. In embodiment 300C, person 102-1 may be able to view all of the public information, but not the private information. Therefore, person 102-1 may not see the private information of user 101. FIG. 3D illustrates an alternate embodiment 300D of how embodiment 300B may appear when viewed by a person not wearing private display device 130-1. Rather than leaving blank space where private information is to be overlaid on public display device 120 for user 101, false information may be presented by the public display device to deceive persons, such as person 102-1. For example, text 310-1 may be presented to user 101, while person 102-1 may see false text 350-1 of FIG. 3D. Further, while user 101 may see text 310-2 via private display device 130-1, person 102-1 may see false text 350-2 presented by public display device 120. False texts 350 may be obscured from viewing by user 101 by the private information presented by private display device 130-1 (e.g., the private information may be displayed brighter and superimposed directly over false texts 350). Such false text may be random (e.g., false text 350-1) or used from a public source (e.g., false text 350-2), which may be retrieved from the Internet.
Returning to FIG. 3B, in order for private display device 130-1 to properly position text 310 over public display device 120, private display device 130-1 may require to be notified of a marker that is displayed by public display device 120 and vector data for where the private information should be presented in relation to the marker. A marker definition engine, such as marker definition engine 240 of FIG. 2, may select some or all of a graphic (or other presented information) as marker 320. Marker 320 may be located by a camera of private display device 130-1 which is aimed in the direction of the field of vision of user 101. From marker 320, vector data may be transmitted to private display device 130-1 and may indicate where private information should be positioned for display by private display device 130-1. For example vector data may indicate a direction and distance, indicated by line 330-1, to present text 310-1. A different vector which indicates a different distance and direction, indicated by line 330-2, may indicate where text 310-2 is to be displayed in relation to marker 320.
If the public information presented by public display device 120 is altered such that marker 320 is no longer displayed, a different portion of the information presented by public display device 120 may be assigned to be the marker; private display device 130-1 may be notified of the new marker and new vector data for use in positioning private information to be displayed by private display device 130-1.
FIG. 4 illustrates an embodiment 400 of a scene in which multiple users are viewing different sets of private information. Scene 400 may be an implementation of system 100B of FIG. 1B. Scene 400 is from the point of view of a user, such as user 103-2 of FIG. 1B. As such, user 103-2 may be sitting (or standing) in the back of a conference room where a presentation is being given. For instance, while persons, such as person 102-2, are viewing the presentation slides presented by public display device 120, user 103-2 may be addressing the crowd about the presentation.
While user 103-2 is viewing public display device 120, private information may be (partially) overlaid public display device 120. In some embodiments, private information may be presented in relation to, but may not be overlaying (either partially or fully), public display device 120, such as to the side of public display device 120. Person 102-2, who is either not wearing a private display device or is not wearing a private display device having the proper permissions, cannot view private information 410. Private information 410 is presented to user 103-2 via a private display device (such as private display device 130-2 of FIG. 1B) that user 103-2 is wearing. In this example, private information 410 serves to alert user 103-2 of various topics to discuss while the picture of the new CEO is displayed and also topics to avoid.
User 103-3 may also be present and may be wearing a private display device, such as private display device 130-3, as shown in FIG. 1B. Depending on the permissions associated with user 103-3, user 103-3 may be presented the same private information 410 as user 103-2. Alternatively, based on the user permissions of user 103-3, user 103-3 may be presented only private information 410-1. In yet another embodiment, user 103-3 may be presented entirely different private information based on the permissions of user 103-3.
As detailed in relation to FIG. 3B, in order for the private display devices used by users 103-2 and 103-3 to properly position private information 410-1 (and private information 410-2 for user 103-2) over public display device 120, each private display device may require to be notified of a marker that is displayed by public display device 120 and vector data for where the private information should be presented in relation to the marker. A marker definition engine, such as marker definition engine 240 of FIG. 2, may select a portion of information as marker 430. A portion of information may be selected as a marker based on the presence of a (uniquely) identifiable combination of features (such as edges, corners, colors, etc). Marker 430 presented by public display device 120 may be located by cameras of the private display devices which are aimed in the direction of the field of vision user's 103-3 and 103-2. From marker 430, vector data may be provided to private display device 130-1 and may indicate where private information should be positioned for display by private display device 130-1. For example vector data may indicate a direction and distance, indicated by line 440-1, to present private information 410-1. A different vector which indicates a different distance and direction, indicated by line 440-2, may indicate where private information 410-2 is to be displayed in relation to marker 430. If the public information presented by public display device 120 is altered such that marker 430 is no longer displayed, a different portion of the information presented by public display device 120 may be assigned to be the marker; private display devices may be notified of the new marker data and new vector data for use in positioning private information to be displayed by the private display devices.
Various methods may be performed using the systems presented and described in relation to FIGS. 1A-4. FIG. 5 illustrates an embodiment of a method 500 for presenting private information. Method 500 may be performed using the system 100A of FIG. 1A and display control system 110 of FIG. 2. For example, method 500 may be performed in a situation similar to scene 300A of FIG. 3A. It should be understood that method 500 may be performed using various embodiments of systems differing from system 100A of FIG. 1A and display control system 110 of FIG. 2. The steps of method 500 may be implemented using software, firmware, and/or hardware. For instance, instructions to perform steps of method 500 may be stored and executed by general-purpose computer components, such as components of computer system 700 of FIG. 7. Accordingly, means for performing the steps of method 500 include one or more instances of: components of system 100A of FIG. 1A, components of display control system 110, and/or components of computer system 700 of FIG. 7.
At step 510, from information to be presented to a user, private information that is to be displayed may be identified. At step 520, from the information to be presented to the user, public information that is to be displayed may be identified. Information that is not classified as private may be, by default, classified as public; the reverse may also be true. Referring to FIG. 2, a data classification analysis engine 210 may receive information that is to be presented to a user. In some embodiments, information received by the data classification analysis engine may be tagged as either public or private. Information that is tagged as public may be routed for display by a public display device while information that is tagged as private may be routed for display to a private display device. In some embodiments, the data classification analysis engine may analyze the information to be presented to determine which information should be classified as public and which information should be classified as private. Such classification may be based on one or more rules, such as rules stored by display analysis rules database 230 of display control system 110. Such rules may be defined or modified by a user. As such, only information the user desires to be treated as private may be presented to the user via a private display device. Means for performing step 510 and step 520 may include a data classification analysis engine. For instance, referring to FIG. 3A, the laptop of user 101 may have hardware, firmware, and/or software that functions as the data classification analysis engine. Generally, a data classification analysis engine may be implemented using software, firmware, and/or hardware. For example, one or more specialized or general purpose processors may perform the functions of a data classification analysis engine.
At step 530, a portion (or possibly all) of the public information to be displayed by the public display device may be selected to serve as a marker for presentation of the private information to be presented by the private display device. In some embodiments multiple markers may be assigned. The marker assigned at step 530 may be large enough for a camera of a private display device located a distance away to detect the marker. Means for performing step 530 may include a marker definition engine, such as marker definition engine 240 of FIG. 2. Different markers may be assigned for different private display devices. In some embodiments, markers may be assigned based on feedback received from a private display device or another form of analysis in which operating conditions of a private display device are analyzed in determining the marker to be used. For example, the application, incorporated by reference, details such assignment of markers based on feedback from a private display device and/or the analysis of operation conditions of the private display device. Such a marker definition engine may be implemented using software, firmware, and/or hardware. For example, one or more specialized or general purpose processors may perform the functions of a marker definition engine.
At step 540, one or more vectors may be determined The vectors determined at step 540 may indicate the spatial relationship between the marker and the private information to be presented by the private display device. Therefore, the vector data determined at step 540 may include a distance and direction for use by the private display device in order to determine where the private information identified at step 510 should be positioned relative to the marker assigned at step 530. It should be understood that at step 540 multiple vectors for multiple pieces of private information may be determined For instance, different vectors for different markers may be determined Such vector determination may be implemented using software, firmware, and/or hardware, such as in the form of marker definition engine 240 of FIG. 2. For example, one or more specialized or general purpose processors may perform such vector determination. In other embodiments, rather than vector data being determined, coordinates or other location information may be determined at step 540 to position private information relative to a marker displayed by a public display device.
At step 550, an indication of the marker (marker data), the vector data, and the private information may be transmitted to the private display device of the user. At step 560, the public information, including the public information that is being used as the marker, may be output to a public display device. At step 570, the public information may be caused to be presented by a public display device, such as a computer monitor. The information transmitted to the private display device at step 550 may cause presentation of the private data to the user via the private display device, which may be an HMD, at step 580. As the public and private information is presented, the private information may appear superimposed over the public display device to the user. Means for performing steps 550, 560, 570, and 580 include computerized software, firmware, and/or hardware. More specifically, means for performing step 550 include a private display interface 280 configured to transmit information to a private display device; means for performing step 560 include a public display interface; means for performing step 570 include a public display device, such as a monitor, television, screen, projector, etc.; and means for performing step 580 include a private display device such as an HMD (or, possibly, a mobile device which could be positioned such that it is only visible to the user).
FIG. 6 illustrates an embodiment of a method 600 for presenting private information using a private display device in conjunction with a public display device. Method 600 may be performed using the system 100B of FIG. 1B and display control system 110 of FIG. 2. For example, method 600 may be performed in a situation similar to scene 400 of FIG. 4. It should be understood that method 600 may be performed using various embodiments of systems differing from system 100B of FIG. 1B and display control system 110 of FIG. 2. The steps of method 600 may be implemented using software, firmware, and/or hardware. For instance, instructions to perform steps of method 600 may be stored and executed by general-purpose computer components, such as components of computer system 700 of FIG. 7. Accordingly, means for performing the steps of method 600 include one or more instances of: components of system 100B of FIG. 1B, components of display control system 110, and/or components of computer system 700 of FIG. 7.
At step 610, an indication of a first user's credentials may be received. At step 620, a second user's credentials may be received. The private data that will be presented to each user may be dependent on each user's credentials. The credentials may be provided to a display control system from each user's private display device, such as an HMD. For example, an HMD may have user credentials stored. When a user enters the range of the display control system and/or provides input indicating to communicate with the display control system, the user credentials may be provided to the display control system in order to register the user. The display control system may compare received user credentials to stored credential information in a user credential database, such as user credential database 235, which may be stored locally by a display control system or may be remotely accessible. If a user's credentials do not indicate access to private data, the user associated with the user credentials may not be provided any private information to view via the user's private display device. If the first and second user each have a same level of access to private data, the same private data may be presented to each of the first and second users via each user's private display device. For the remainder of the described example of method 600, it is assumed that the first user and the second user each have access to different sets of private information. Performance of steps 610 and 620 may include private display devices transmitting (e.g., wirelessly) information to a private display interface, such as private display interface of display control system 110. A private data display modification engine 260 may analyze the received user credentials against stored credentials in a user credential database, such as user credential database 235. Accordingly, means for performing steps 610 and 620 include processors, communication interfaces, and non-transitory computer-readable storage mediums.
At step 610, from information to be presented to a user, private information that is to be displayed to each user, based on each user's credentials, may be identified. At step 620, from the information to be presented, public information that is to be displayed may be identified. Such public information, at minimum, will be presented to both users via the public display device. Additional persons, who may not view any private information, such as person 102-2 of FIG. 4, may view the public information presented via a public display device. Information that is not classified as private may be, by default, classified as public; the reverse may also be true. Referring to FIG. 2, a data classification analysis engine 210 may receive information that is to be presented to at least the two users. In some embodiments, information received by the data classification analysis engine may be tagged as either public, private, or private and available to only particular users. Information that is tagged as public may be routed for display by a public display device while information is tagged as private may be routed for display to a private display device. In some embodiments, the data classification analysis engine may analyze the information to be presented to determine which information should be classified as public and which information should be classified as private. Such classification may be based on one or more rules, such as rules stored by display analysis rules database 230 of display control system 110. Such rules may be defined or modified by a user. As such, only information the users desire to be treated as private may be presented to the users via a private display device.
A private data display modification engine may enforce user credentials such that private data is presented only to users who have the credentials specific for that private data. Referring to FIG. 4, for example, while user 103-2 may be able to view private information 410-1 and 410-2, user 103-3 may only have sufficient credentials to view private information 410-1. Means for performing step 630 and step 640 may include a data classification analysis engine, private data display modification engine, and user credential database. For instance, referring to FIG. 3B, a computerized device providing output to private display devices and public display device 120 may have hardware, firmware, and/or software that functions as the data classification analysis engine, private data display modification engine, and user credential database.
Generally, such functions may be implemented using software, firmware, and/or hardware. For example, one or more specialized or general purpose processors and non-transitory computer-readable mediums may perform the functions of a data classification analysis engine, private data display modification engine, and user credential database.
At step 650, a portion (or possibly all) of the public information to be displayed by the public display device may be selected to serve as a marker for presentation of the private information to be presented by the private display device of each user. In some embodiments multiple markers may be assigned (e.g., different markers may be used for different private display devices). The same marker may be used for multiple private display devices; for example, referring to FIG. 4, marker 430 may be used for the private display devices of user 103-2 and user 103-3. The marker assigned at step 650 may be large and distinct enough for a camera of a private display device located a distance away to detect the marker. Means for performing step 650 may include a marker definition engine, such as marker definition engine 240 of FIG. 2. In some embodiments, markers may be assigned based on feedback received from a private display device or another form of analysis in which operating conditions of a private display device are analyzed in determining the marker to be used. For example, the application, incorporated by reference, details such assignment of markers based on feedback from a private display device and/or the analysis of operation conditions of the private display device. Such a marker definition engine may be implemented using software, firmware, and/or hardware. For example, one or more specialized or general purpose processors may perform the functions of a marker definition engine.
At step 660, one or more vectors may be determined The vectors determined at step 660 may indicate the spatial relationship between the marker and the private information to be presented by each private display device. As such, if private information is being presented in different locations to different users, the vector data provided to each of the user's private display devices may vary, even if the same marker is used for each private display device. Therefore, the vector data determined at step 660 may include a distance and direction for use by each private display device in order to determine where private information should be positioned relative to the marker assigned at step 650. It should be understood that at step 660 multiple vectors for multiple pieces of private information may be determined For instance, different vectors for different markers may be determined Such vector determination may be implemented using software, firmware, and/or hardware, such as in the form of marker definition engine 240 of FIG. 2. For example, one or more specialized or general purpose processors may perform such vector determination. In other embodiments, rather than vector data being determined, coordinates or other location information may be determined at step 660 to position private information relative to a marker displayed by a public display device.
At step 670, an indication of the marker, the vector data, and the private information may be transmitted to each private display device. Only private data to be viewed using a particular private display device may be provided to that private display device. For example, referring to FIG. 4, the private display device of user 103-3 may not receive any indication of private information 410-2. At step 680, the public information, including the public information that is being used as the marker(s) for each private device, may be output to a public display device (which is viewable by at least the two users). At step 690, the public information may be caused to be presented by a public display device, such as a projector in FIG. 4. The information transmitted to the private display device at step 670 may cause presentation of the private data to each user via each user's private display device, which may be an HMD, at step 695. As the public and private information is presented, the private information may appear superimposed over at least a portion of the public display device to the user or may be presented in a position relative to the marker presented by the public display device (e.g., a distance to the side of the public display device).
Means for performing steps 670, 680, 690, and 695 include computerized software, firmware, and/or hardware. More specifically, means for performing step 670, include a private display interface 280 configured to transmit information to a private display device; means for performing step 680 include a public display interface; means for performing step 690 include a public display device, such as a monitor, television, screen, projector, etc.; and means for performing step 695 include at least two private display devices such as HMDs (or, possibly, mobile devices which could be positioned such that they are only visible to the corresponding user).
FIG. 7 illustrates an embodiment of a computer system. A computer system as illustrated in FIG. 7 may be incorporated as part of the previously described computerized devices, such as the display control systems, public display devices, and private display devices. FIG. 7 provides a schematic illustration of one embodiment of a computer system 700 that can perform various steps of the methods provided by various embodiments. It should be noted that FIG. 7 is meant only to provide a generalized illustration of various components, any or all of which may be utilized as appropriate. FIG. 7, therefore, broadly illustrates how individual system elements may be implemented in a relatively separated or relatively more integrated manner.
The computer system 700 is shown comprising hardware elements that can be electrically coupled via a bus 705 (or may otherwise be in communication, as appropriate). The hardware elements may include one or more processors 710, including without limitation one or more general-purpose processors and/or one or more special-purpose processors (such as digital signal processing chips, graphics acceleration processors, video decoders, and/or the like); one or more input devices 715, which can include without limitation a mouse, a keyboard, remote control, and/or the like; and one or more output devices 720, which can include without limitation a display device, a printer, and/or the like. Output devices 720 may include a public display device.
The computer system 700 may further include (and/or be in communication with) one or more non-transitory storage devices 725, which can comprise, without limitation, local and/or network accessible storage, and/or can include, without limitation, a disk drive, a drive array, an optical storage device, a solid-state storage device, such as a random access memory (“RAM”), and/or a read-only memory (“ROM”), which can be programmable, flash-updateable and/or the like. Such storage devices may be configured to implement any appropriate data stores, including without limitation, various file systems, database structures, and/or the like.
The computer system 700 might also include a communications subsystem 730, which can include without limitation a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and/or a chipset (such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, cellular communication device, etc.), and/or the like. The communications subsystem 730 may permit data to be exchanged with a network (such as the network described below, to name one example), other computer systems, and/or any other devices described herein. In many embodiments, the computer system 700 will further comprise a working memory 735, which can include a RAM or ROM device, as described above.
The computer system 700 also can comprise software elements, shown as being currently located within the working memory 735, including an operating system 740 (which may correspond to operating system 222 of FIG. 2), device drivers, executable libraries, and/or other code, such as one or more application programs 745 (which may correspond to application 224 of FIG. 2), which may comprise computer programs provided by various embodiments, and/or may be designed to implement methods, and/or configure systems, provided by other embodiments, as described herein. Merely by way of example, one or more procedures described with respect to the method(s) discussed above might be implemented as code and/or instructions executable by a computer (and/or a processor within a computer); in an aspect, then, such code and/or instructions can be used to configure and/or adapt a general purpose computer (or other device) to perform one or more operations in accordance with the described methods.
A set of these instructions and/or code might be stored on a non-transitory computer-readable storage medium, such as the non-transitory storage device(s) 725 described above. In some cases, the storage medium might be incorporated within a computer system, such as computer system 700. In other embodiments, the storage medium might be separate from a computer system (e.g., a removable medium, such as a compact disc), and/or provided in an installation package, such that the storage medium can be used to program, configure, and/or adapt a general purpose computer with the instructions/code stored thereon. These instructions might take the form of executable code, which is executable by the computer system 700 and/or might take the form of source and/or installable code, which, upon compilation and/or installation on the computer system 700 (e.g., using any of a variety of generally available compilers, installation programs, compression/decompression utilities, etc.), then takes the form of executable code.
It will be apparent to those skilled in the art that substantial variations may be made in accordance with specific requirements. For example, customized hardware might also be used, and/or particular elements might be implemented in hardware, software (including portable software, such as applets, etc.), or both. Further, connection to other computing devices such as network input/output devices may be employed.
As mentioned above, in one aspect, some embodiments may employ a computer system (such as the computer system 700) to perform methods in accordance with various embodiments of the invention. According to a set of embodiments, some or all of the procedures of such methods are performed by the computer system 700 in response to processor 710 executing one or more sequences of one or more instructions (which might be incorporated into the operating system 740 and/or other code, such as an application program 745) contained in the working memory 735. Such instructions may be read into the working memory 735 from another computer-readable medium, such as one or more of the non-transitory storage device(s) 725. Merely by way of example, execution of the sequences of instructions contained in the working memory 735 might cause the processor(s) 710 to perform one or more procedures of the methods described herein. For instance, processor(s) 710 may perform various functions of the components of display control system 110 of FIG. 2, such as data classification analysis engine 210 and marker definition engine 240.
The terms “machine-readable medium,” “computer-readable storage medium” and “computer-readable medium,” as used herein, refer to any medium that participates in providing data that causes a machine to operate in a specific fashion. These mediums may be non-transitory. In an embodiment implemented using the computer system 700, various computer-readable media might be involved in providing instructions/code to processor(s) 710 for execution and/or might be used to store and/or carry such instructions/code. In many implementations, a computer-readable medium is a physical and/or tangible storage medium. Such a medium may take the form of a non-volatile media or volatile media. Non-volatile media include, for example, optical and/or magnetic disks, such as the non-transitory storage device(s) 725. For instance, non-transitory storage device(s) 725 may store display analysis rules database 230 and/or user credential database 235 of FIG. 2. Volatile media include, without limitation, dynamic memory, such as the working memory 735.
Common forms of physical and/or tangible computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, any other physical medium with patterns of marks, a RAM, a PROM, EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read instructions and/or code.
Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor(s) 710 for execution. Merely by way of example, the instructions may initially be carried on a magnetic disk and/or optical disc of a remote computer. A remote computer might load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and/or executed by the computer system 700.
The communications subsystem 730 (and/or components thereof) generally will receive signals, and the bus 705 then might carry the signals (and/or the data, instructions, etc. carried by the signals) to the working memory 735, from which the processor(s) 710 retrieves and executes the instructions. The instructions received by the working memory 735 may optionally be stored on a non-transitory storage device 725 either before or after execution by the processor(s) 710. Communications subsystem 730 may perform functions of private display interface 280 for communicating with a private display device.
It should further be understood that the components of computer system 700 can be distributed across a network. For example, some processing may be performed in one location using a first processor while other processing may be performed by another processor remote from the first processor. Other components of computer system 700 may be similarly distributed. As such, computer system 700 may be interpreted as a distributed computing system that performs processing in multiple locations. In some instances, computer system 700 may be interpreted as a single computing device, such as a distinct laptop, desktop computer, or the like, depending on the context.
The methods, systems, and devices discussed above are examples. Various configurations may omit, substitute, or add various procedures or components as appropriate. For instance, in alternative configurations, the methods may be performed in an order different from that described, and/or various stages may be added, omitted, and/or combined. Also, features described with respect to certain configurations may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Also, technology evolves and, thus, many of the elements are examples and do not limit the scope of the disclosure or claims.
Specific details are given in the description to provide a thorough understanding of example configurations (including implementations). However, configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail in order to avoid obscuring the configurations. This description provides example configurations only, and does not limit the scope, applicability, or configurations of the claims. Rather, the preceding description of the configurations will provide those skilled in the art with an enabling description for implementing described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the disclosure.
Also, configurations may be described as a process which is depicted as a flow diagram or block diagram. Although each may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may have additional steps not included in the figure. Furthermore, examples of the methods may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored in a non-transitory computer-readable medium such as a storage medium. Processors may perform the described tasks.
Having described several example configurations, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, wherein other rules may take precedence over or otherwise modify the application of the invention. Also, a number of steps may be undertaken before, during, or after the above elements are considered.

Claims

What is claimed is:

1. A method for presenting private information, the method comprising:

identifying, by a display control system, private information to be displayed, wherein the private information is to be viewed via a head mounted display by a user;

identifying, by the display control system, a marker present in public information displayed by a public display device, wherein the public display device is visible in a vicinity of the user; and

outputting, by the display control system, the private information and an indication of the marker to the head mounted display of the user, such that the private information is displayed by the head mounted display in relation to the marker displayed by the public display device.

2. The method for presenting private information of claim 1, the method further comprising:

identifying, by the display control system, public information to be displayed by the public display device; and

outputting, by the display control system, the public information to the public display device, wherein the public information is displayed by the public display device.

3. The method for presenting private information of claim 2, wherein at least a portion of the public information is used as the marker.

4. The method for presenting private information of claim 1, wherein information associated with the marker indicates that the private information is to be displayed by the head mounted display overlaying the public display device.

5. The method for presenting private information of claim 4, further comprising:

presenting, by the head mounted display, the private information in relation to the marker, such that the private information overlays a portion of the public display device.

6. The method for presenting private information of claim 1, further comprising:

identifying, by the display control system, a second piece of private information to be displayed to a second user, wherein the second piece of private information is to be viewed by the second user to the exclusion of the user;

identifying, by the display control system, a second marker displayed by the public display device, wherein the public display device is visible in the vicinity of the user and a second vicinity of the second user; and

outputting, by the display control system, the second piece of private information and a second indication of the second marker to a second head-mounted display, such that the second piece of private information is displayed by the second head-mounted display in relation to the second marker displayed by the public display device.

7. The method for presenting private information of claim 6, wherein the marker and the second marker are a same marker.

8. The method for presenting private information of claim 1, wherein identifying the private information to be displayed comprises receiving, by the display control system, a definition of a rule for identifying the private information from the user; and

identifying, by the display control system, the private information to be displayed comprises using the rule.

9. The method for presenting private information of claim 1, further comprising:

outputting, by the display control system, to the public display device, false information, wherein the private information displayed by the head mounted display is configured to overlay the false information displayed by the public display device.

10. A system for presenting private information, the system comprising:

one or more processors; and

a memory communicatively coupled with and readable by the one or more processors and having stored therein processor-readable instructions which, when executed by the one or more processors, cause the one or more processors to:

identify private information to be displayed, wherein the private information is to be viewed via a head mounted display by a user;

identify a marker present in public information displayed by a public display device, wherein the public display device is visible in a vicinity of the user; and

output the private information and an indication of the marker to the head mounted display of the user, such that the private information is displayed by the head mounted display in relation to the marker displayed by the public display device.

11. The system for presenting private information of claim 10, wherein the processor-readable instructions, when executed by the one or more processors, further cause the one or more processors to:

identify public information to be displayed by the public display device; and

output the public information to the public display device, wherein the public information is displayed by the public display device.

12. The system for presenting private information of claim 11, wherein at least a portion of the public information is used as the marker.

13. The system for presenting private information of claim 10, wherein information associated with the marker indicates that the private information is to be displayed by the head mounted display overlaying the public display device.

14. The system for presenting private information of claim 13, the system further comprising the head mounted display, configured to present the private information in relation to the marker, such that the private information overlays a portion of the public display device.

15. The system for presenting private information of claim 10, wherein the processor-readable instructions, when executed by the one or more processors, further cause the one or more processors to:

identify a second piece of private information to be displayed to a second user, wherein the second piece of private information is to be viewed by the second user to the exclusion of the user;

identify a second marker displayed by the public display device, wherein the public display device is visible in the vicinity of the user and a second vicinity of the second user; and

output the second piece of private information and a second indication of the second marker to a second head-mounted display, such that the second piece of private information is displayed by the second head-mounted display in relation to the second marker displayed by the public display device.

16. The system for presenting private information of claim 15, wherein the marker and the second marker are a same marker.

17. The system for presenting private information of claim 10, wherein the instructions that cause the one or more processors to identify the private information to be displayed comprises processor-readable instructions which, when executed, cause the one or more processors to:

receive a definition of a rule for identifying the private information from the user, wherein the processor-readable instructions configured to cause the one or more processors to identify the private information to be displayed comprises processor-readable instructions configured to cause the one or more processors to identify the private information to be displayed using the rule.

18. The system for presenting private information of claim 10, wherein the processor-readable instructions, when executed by the one or more processors, further cause the one or more processors to:

output, to the public display device, false information, wherein the private information displayed by the head mounted display is configured to overlay the false information displayed by the public display device.

19. A non-transitory computer-readable medium for presenting private information, comprising processor-readable instructions configured to cause one or more processors to:

20. The non-transitory computer-readable medium for presenting private information of claim 19, wherein the processor-readable instructions are further configured to cause the one or more processors to:

identify public information to be displayed by the public display device; and

21. The non-transitory computer-readable medium for presenting private information of claim 20, wherein at least a portion of the public information is used as the marker.

22. The non-transitory computer-readable medium for presenting private information of claim 19, wherein information associated with the marker indicates that the private information is to be displayed by the head mounted display overlaying the public display device.

23. The non-transitory computer-readable medium for presenting private information of claim 19, wherein the processor-readable instructions are further configured to cause the one or more processors to:

24. The non-transitory computer-readable medium for presenting private information of claim 23, wherein the marker and the second marker are a same marker.

25. The non-transitory computer-readable medium for presenting private information of claim 19, wherein the instructions configured cause the one or more processors to identify the private information to be displayed comprises processor-readable instructions configured to cause the one or more processors to:

26. The non-transitory computer-readable medium for presenting private information of claim 19, wherein the processor-readable instructions are further configured to cause the one or more processors to:

27. An apparatus for presenting private information, the apparatus comprising:

means for identifying private information to be displayed, wherein the private information is to be viewed via a means for presenting private information by a user;

means for identifying a marker present in public information displayed by a means for presenting public information, wherein the means for presenting public information is visible in a vicinity of the user; and

means for outputting the private information and an indication of the marker to the means for presenting private information of the user, such that the private information is displayed by the means for presenting private information in relation to the marker displayed by the means for presenting public information.

28. The apparatus for presenting private information of claim 27, the apparatus further comprising:

means for identifying public information to be displayed by the means for presenting public information; and

means for outputting the public information to the means for presenting public information, wherein the public information is displayed by the means for presenting public information.

29. The apparatus for presenting private information of claim 28, wherein at least a portion of the public information is used as the marker.

30. The apparatus for presenting private information of claim 27, wherein information associated with the marker indicates that the private information is to be displayed by the means for presenting private information overlaying the means for presenting public information.

31. The apparatus for presenting private information of claim 30, further comprising:

means for presenting the private information in relation to the marker, such that the private information overlays a portion of the means for presenting public information.

32. The apparatus for presenting private information of claim 27, further comprising:

means for identifying a second piece of private information to be displayed to a second user, wherein the second piece of private information is to be viewed by the second user to the exclusion of the user;

means for identifying a second marker displayed by the means for presenting public information, wherein the means for presenting public information is visible in the vicinity of the user and a second vicinity of the second user; and

means for outputting the second piece of private information and a second indication of the second marker to a second means for presenting private information, such that the second piece of private information is displayed by the second means for presenting private information in relation to the second marker displayed by the means for presenting public information.

33. The apparatus for presenting private information of claim 32, wherein the marker and the second marker are a same marker.

34. The apparatus for presenting private information of claim 27, wherein the means for identifying the private information to be displayed comprises:

means for receiving a definition of a rule for identifying the private information from the user, wherein the means identifying the private information to be displayed comprises means for using the rule.

35. The apparatus for presenting private information of claim 27, further comprising:

means for outputting to the public display device, false information, wherein the private information displayed by the means for presenting private information is configured to overlay the false information displayed by the means for presenting public information.