US20130063493A1

US20130063493A1 - Devices and Methods Involving Display Interaction Using Photovoltaic Arrays

Info

Publication number: US20130063493A1
Application number: US13/337,538
Authority: US
Inventors: Christopher David House
Original assignee: HTC Corp
Current assignee: HTC Corp
Priority date: 2011-09-14
Filing date: 2011-12-27
Publication date: 2013-03-14
Also published as: US20130063823A1; CN102999229A; TW201312218A; TW201317550A; CN103218036A; TWI461663B; CN103218036B; TWI491959B; CN102999229B; US20130065647A1; TW201312428A; CN103002692B; TWI479392B; CN103002692A; US9110538B2; US9026179B2

Abstract

Devices and methods involving display interaction using photovoltaic arrays are provided. In this regard, a representative device incorporates: a display operative to display images to a user; and a photovoltaic array positioned in an overlying relationship with at least a portion of the display and being operative to detect light incident upon the photovoltaic array; the display being further operative to respond to a sensed localized differential in intensity of the light incident upon the photovoltaic array such that, responsive to the sensed localized differential, operation of the display is altered.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a utility application that claims priority to co-pending U.S. Provisional Patent Application entitled, “Photovoltaic Film Application”, having Ser. No. 61/534,715, filed Sep. 15, 2011, which is entirely incorporated herein by reference.

TECHNICAL FIELD

The present disclosure generally relates to interactive displays.

BACKGROUND

Over the years, portable handheld devices such as smartphones have become prevalent. Many of these devices use various techniques for implementing touch sensing so that users can provide inputs to the devices. Typically, touch sensing is accomplished through the use of resistive or capacitive sensing. Using such a technique, a “touch event” is recognized by a device when the user's finger contacts the touch surface, which is often the outer surface of the device display. As such, the user inputs are two dimensional and require direct contact with the device.

SUMMARY

Devices and methods involving display interaction using photovoltaic arrays are provided. Briefly described, one embodiment, among others, is device comprising: a display operative to display images to a user; and a photovoltaic array positioned in an overlying relationship with at least a portion of the display and being operative to detect light incident upon the photovoltaic array; the display being further operative to respond to a sensed localized differential in intensity of the light incident upon the photovoltaic array such that, responsive to the sensed localized differential, operation of the display is altered.
Another embodiment is a method for interacting with a display comprising: sensing a localized differential in intensity of incident light, the incident light corresponding to light incident upon the display; and altering an operation of the display based, at least in part, on the localized differential in sensed incident light.
Other systems, methods, features, and advantages of the present disclosure will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.

FIG. 1 is a partially-exploded, schematic view of an example embodiment of a device.

FIG. 2 is a schematic diagram depicting an example embodiment of a device and a method for using the device.

FIG. 3 is a schematic diagram depicting the example of FIG. 2 in greater detail.

FIG. 4 is a flowchart depicting an example embodiment of a method.

FIG. 5 is a schematic diagram depicting an example embodiment of a device.

FIG. 6A is a schematic diagram depicting a representative displayed image.

FIG. 6B is a schematic diagram depicting an example embodiment of a method associated with the displayed image of FIG. 6A.

FIG. 7A is a schematic diagram depicting a representative displayed image.

FIG. 7B is a schematic diagram depicting an example embodiment of a method associated with the displayed image of FIG. 7A.

FIG. 8A is a schematic diagram depicting a representative displayed image.

FIG. 8B is a schematic diagram depicting an example embodiment of a method associated with the displayed image of FIG. 8A.

FIG. 9A is a schematic diagram depicting a representative displayed image.

FIG. 9B is a schematic diagram depicting an example embodiment of a method associated with the displayed image of FIG. 9A.

DETAILED DESCRIPTION

Having summarized various aspects of the present disclosure, reference will now be made in detail to that which is illustrated in the drawings. While the disclosure will be described in connection with these drawings, there is no intent to limit the scope of legal protection to the embodiment or embodiments disclosed herein. Rather, the intent is to cover all alternatives, modifications and equivalents included within the spirit and scope of the disclosure as defined by the appended claims.
Devices and methods involving display interaction using photovoltaic arrays are provided. In some embodiments, a transparent photovoltaic array is positioned in an overlying relationship with a display of a device, such as a mobile phone. The array detects incident light and is able to sense a localized differential in intensity of the light incident thereupon. Responsive to a sensed localized differential, which can be caused by a user of the device interrupting a path of light to the display (e.g., moving a pen over the display), operation of the display can be altered. By way of example, a graphical actuator can be actuated responsive to the sensed localized differential. As another example, the size, position and/or orientation of an image being displayed can be changed.
In some embodiments, a three dimensional sensing volume is created above the display so that a user can interact with the device without having to touch the display. Movement of the user away from and/or toward the display may also be sensed, in some embodiments, resulting in a vast number of user movements within the 3-D sensing volume that can be recognized as device commands, for example.
In this regard, FIG. 1 is a partially-exploded, schematic view of an example embodiment of a device. As shown in FIG. 1, device 100 is configured as a mobile phone that incorporates a housing 102, a cover, 104, a display 106 and a photovoltaic array 108. The housing and cover define an interior in which the various other components of the device (some of which are not depicted) are located. The display (e.g., a liquid crystal display, LED display, OLED display, etc.) includes a display module 110 with a display side 112.
In this embodiment, the photovoltaic array is provided as a transparent layer (e.g., a film) that is positioned between the display and the cover. The array is positioned in an overlying relationship with at least a portion of the display (in this case, the entire displayable area of the display) and is operative to sense light incident thereupon. Responsive to the incident light, the array generates electrical signals, the strength of which corresponds to the intensity of the incident light at various portions of the array. The electrical signals are used to provide corresponding input so that an onboard processing device (not shown in FIG. 1) can determine whether or not a localized differential in the incident light is present. By way of example, a localized differential can be determined to exist if one of a subset of adjacent zones of the array exhibits a difference in signal strength that corresponds to a predetermined threshold. It should also be noted that, in some embodiments, electricity generated by the array can be used to power the device.
The photovoltaic array of FIG. 1 exhibits zones (e.g. zone 114), each of which corresponds to a different physical region of the array. These zones of the photovoltaic array are associated with corresponding zones of the display (e.g., zone 116).
FIG. 2 is a schematic diagram depicting an example embodiment of device 100 of FIG. 1 and a method for using the device. As shown in FIG. 2, device 100 includes a display surface 118, which, in effect, corresponds to the outer surface of cover 104. As the user (e.g., a hand 120 of the user) approaches the display surface, a shadowed region 122 is formed on the display surface owing to the user obstructing the optical path between light source 124 and the device. Note that the shadowed region impinges upon a displayed actuator 126.
FIG. 3 is a schematic diagram depicting the example of FIG. 2 in greater detail. Specifically, FIG. 3 shows a top view of the photovoltaic array, which is depicted in zones and upon which the shadowed region 122 is shown to include portions with varying degrees of shadowing. For instance, portion 127 is the more shadowed of the portions exhibiting a lower intensity of light, and portion 129 is the less shadowed of the portions. Notably, the more shadowed portion generally corresponds to the portion of the shadowed region closest in distance to the light obstructing feature, which in this example is the tip 130 of the user's finger.
In FIG. 3, zones in a vicinity of the shadowed region include zones 132, 134, 136, 142, 144, 146, 152, 154 and 156, with the more shadowed portion being located mostly within zone 144, and with spillover into each of zones 142 and 146. The less shadowed portion is located mostly in zone 146, and with spillover into each of zones 142, 134, 144, 154, 136 and 156.
Using information corresponding to the intensity of incident light for each of the zones enables a determination to be made as to whether a localized differential in the sensed light exists. For instance, by comparing the intensities of light incident upon adjacent zones, it can be readily discerned that a localized differential exists between the zones adjacent to zones 144 and 146, with the most pronounced localized differential existing between each of zones 132, 134, 136, 152, 154, 156 and zone 144. As such, zone 144 may be determined to be a zone of interest. Notably, zone 144 of the array can then be correlated with a zone of the display (i.e., a zone that the user is attempting to interact with).
Additionally or alternatively, some embodiments may be able to determine user intent by measuring the rate of change of the localized photovoltaic effect within and/or across the array. As described above, a node with constant differential relative to the surrounding node may provide one user input method. A change and/or acceleration of change within and/or across the nodes may provide additional methods for user input, for example.
Responsive to determining that a localized differential exists in the light incident upon the array, that information can be used to perform various functions. For instance, if it is determined that the differential corresponds to a position of a graphical actuator, the actuator could be actuated. For instance, if zone 144 of the array is determined to be of interest and zone 144 corresponds to a zone of the display that includes actuator 126, then actuator 126 may be actuated.
FIG. 4 is a flowchart depicting an example embodiment of a method for interacting with a display. As shown in FIG. 4, the method includes sensing a localized differential in intensity of incident light (block 160). Notably, the incident light corresponds to light incident upon the device and to an associated photovoltaic array. In block 162, operation of a display of the device is altered based, at least in part, on the localized differential in sensed incident light.
FIG. 5 illustrates mobile device 100 of FIG. 1 that may be embodied as a smartphone but may also be embodied in any one of a wide variety of wired and/or wireless computing devices. As shown in FIG. 5, mobile device 100 includes a processing device (processor) 170, input/output interfaces 172, display 106, a network interface 176, a memory 178, an operating system 180, a mass storage 182 and a light sensing system 184, with each communicating across a local data bus 186. Notably, the light sensing system may, in conjunction with one or more other components, perform the method described above with respect to FIG. 4, for example.
The processing device may include a custom made or commercially available processor, a central processing unit (CPU) or an auxiliary processor among several processors associated with the mobile device, a semiconductor based microprocessor (in the form of a microchip), a macroprocessor, one or more application specific integrated circuits (ASICs), a plurality of suitably configured digital logic gates, and other electrical configurations comprising discrete elements both individually and in various combinations to coordinate the overall operation of the system.
The memory can include any one of a combination of volatile memory elements (e.g., random-access memory (RAM, such as DRAM, and SRAM, etc.)) and nonvolatile memory elements. The memory typically comprises native operating system, one or more native applications, emulation systems, or emulated applications for any of a variety of operating systems and/or emulated hardware platforms, emulated operating systems, etc. For example, the applications may include application specific software which may comprise some or all the components of the mobile device. In accordance with such embodiments, the components are stored in memory and executed by the processing device. Note that although depicted separately in FIG. 5, light sensing system 184 may be resident in memory such as memory 178.
In this embodiment, the light sensing system 184 further comprises photovoltaic array 108, which facilitates a touchless interaction with a user. However, in other embodiments, a touchscreen interface could be provided to detect contact within the display area of the display 106.
One of ordinary skill in the art will appreciate that the memory can, and typically will, comprise other components which have been omitted for purposes of brevity. Note that in the context of this disclosure, a non-transitory computer-readable medium stores one or more programs for use by or in connection with an instruction execution system, apparatus, or device.
With further reference to FIG. 5, the network interface device comprises various components used to transmit and/or receive data over a networked environment. When such components are embodied as an application, the one or more components may be stored on a non-transitory computer-readable medium and executed by the processing device.
Reference is now made to FIGS. 6A and 6B, in which FIG. 6B is a schematic diagram depicting an example embodiment of a method associated with the displayed image of FIG. 6A. As shown in FIG. 6A, device 200 displays an image 202 on a display 204. Note the size and position of the image, which is generally centered on the display and which takes up about 20% of the display area.
In FIG. 6B, a user of the device places hand 210 above the display and then moves the hand toward the display as indicated by the arrow. Responsive to the hand moving to the position above the display shown in FIG. 7B (and without touching the device), the image is altered to that shown in FIG. 7A. In particular, the image reduces in size (e.g., reduces by approximately 70%). This is accomplished responsive to a photovoltaic array of the device sensing a localized differential that changes in size and intensity due to the movement of the hand.
In some embodiments, this can be accomplished in two general steps; determining user intent and determining the extent of the alteration desired. With respect to recognizing user intent to change image size (e.g., zoom), user intent may be identified with a combination of the state that the device is in and user input. For example, the state the device is in may be one that typically allows zooming functionality (e.g., photo gallery mode, map application, internet browsing). A user input may then indicate that the user wishes to modify the existing state of the image being displayed (e.g., photo, map, web page). This could be accomplished with the user “selecting” the image by holding a finger or hand above the image being displayed such that the system recognizes a localized change within the array (similar to method shown in FIG. 2). A time period may be associated with this user action to reduce detection error or accidental “selection”.
With respect to the extent of alteration desired, user input may be used again. For instance, the image size may be reduced responsive to the user's movement toward the device (FIGS. 7A and 7B). In some embodiments, the photovoltaic array may use one or more of several metrics to determine the extent of alteration desired. These metrics may include, but are not limited to: by moving a finger/hand perpendicular to display only, the system could determine that the user desired to change the size of the image (e.g., zoom) not the positioning of the image with in the display; the change of the electrical output from the photovoltaic sensor in that particular region would provide input as the user's intent to zoom in or out (e.g., higher electrical output means more light, more light means the user is blocking less light and the user intents the image to become larger); and the system could measure the rate of change by sampling the electrical current produced by the photovoltaic array—a high rate a change signals the system to zoom more quickly and perhaps scale the image more over the same distance traveled versus a slow rate of change.
In FIGS. 8A and 8B, FIG. 8B is a schematic diagram depicting an example embodiment of a method associated with the displayed image of FIG. 8A. As shown in FIG. 8A, device 200 displays an image 202 once again, generally centered on the display and taking up about 20% of the display area.
In FIG. 8B, a user of the device places hand 210 to a first side of the display and then moves the hand across the display as indicated by the arrow. Responsive to the hand moving to the position above the display shown in FIG. 9B (and without touching the device), the image is altered to that shown in FIG. 9A. In particular, the image moves in a direction corresponding to the direction of motion of the hand while maintaining its original size. This is accomplished responsive to a photovoltaic array of the device sensing a localized differential that moves across the device. Specifically, the system determines that the user's intent is to move the image within the display region due to the change in the photovoltaic effect from one array node to the next (as the user's finger/hand moved across the display instead of perpendicular as in the case of zooming in/out).
If embodied in software, it should be noted that each block depicted in the flowcharts represents a module, segment, or portion of code that comprises program instructions stored on a non-transitory computer readable medium to implement the specified logical function(s). In this regard, the program instructions may be embodied in the form of source code that comprises statements written in a programming language or machine code that comprises numerical instructions recognizable by a suitable execution system such as the mobile device 100 shown in FIG. 5. The machine code may be converted from the source code, etc. If embodied in hardware, each block may represent a circuit or a number of interconnected circuits to implement the specified logical function(s). Additionally, although the flowcharts show specific orders of execution, it is to be understood that the orders of execution may differ.
It should be emphasized that the above-described embodiments are merely examples of possible implementations. Many variations and modifications may be made to the above-described embodiments without departing from the principles of the present disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims

1. A device comprising:

a display operative to display images to a user; and

a photovoltaic array positioned in an overlying relationship with at least a portion of the display and being operative to detect light incident upon the photovoltaic array;

the display being further operative to respond to a sensed localized differential in intensity of the light incident upon the photovoltaic array such that, responsive to the sensed localized differential, operation of the display is altered.

2. The device of claim 1, wherein the photovoltaic array is configured as a transparent layer.

3. The device of claim 1, wherein:

a first of the images displayed is a graphical actuator; and

the display is operative to actuate the graphical actuator responsive to the sensed localized differential.

4. The device of claim 1, wherein the images are viewable through the photovoltaic array.

5. The device of claim 1, wherein the device is a mobile phone.

6. The device of claim 1, wherein the display is further operative to respond to a change in the sensed localized differential such that, responsive to detecting that the localized differential is increasing in size, the display expands a size of a first of the images being displayed.

7. The device of claim 1, wherein the display is further operative to respond to a change in the sensed localized differential such that, responsive to detecting that the localized differential is decreasing in size, the display shrinks a size of a first of the images being displayed.

8. The device of claim 1, wherein the display is further operative to respond to a change in the sensed localized differential such that, responsive to detecting that the localized differential is increasing in magnitude, the display shrinks a size of a first of the images being displayed.

9. The device of claim 1, wherein the display is further operative to respond to a change in the sensed localized differential such that, responsive to detecting that the localized differential is decreasing in magnitude, the display expands a size of a first of the images being displayed.

10. The device of claim 1, wherein the display is further operative to respond to a change in the sensed localized differential such that, responsive to detecting movement of the localized differential is across the photovoltaic array, the display scrolls a first of the images being displayed across the display in a direction corresponding to the movement.

11. The device of claim 1, wherein the first feature of the device is actuated without the user physically contacting a display area of the display.

12. A method for interacting with a display comprising:

sensing a localized differential in intensity of incident light, the incident light corresponding to light incident upon the display; and

altering an operation of the display based, at least in part, on the localized differential in sensed incident light.

13. The method of claim 12, wherein altering an operation of the display further comprises altering a configuration of an image displayed on the display.

14. The method of claim 12, wherein the sensing further comprises sensing the localized differential with a photovoltaic array positioned proximate to the display.

15. The method of claim 12, wherein:

the method further comprises correlating the localized differential with a location on the display; and

altering a configuration is performed responsive to the correlating of the location on the display.

16. The method of claim 12, wherein the configuration further comprises enlarging the image.

17. The method of claim 12, wherein the configuration further comprises reducing the image.

18. The method of claim 12, wherein the configuration further comprises moving a position of the image.

19. The method of claim 12, wherein altering an operation of the display further comprises actuating a feature associated with an area of the display corresponding to the sensed localized differential.

20. The method of claim 19, wherein actuating the feature further comprises actuating a graphical actuator associated with the feature.