EP4602448A1 - Power saving device with variable touchscreen display performance - Google Patents

Power saving device with variable touchscreen display performance

Info

Publication number
EP4602448A1
EP4602448A1 EP23848749.0A EP23848749A EP4602448A1 EP 4602448 A1 EP4602448 A1 EP 4602448A1 EP 23848749 A EP23848749 A EP 23848749A EP 4602448 A1 EP4602448 A1 EP 4602448A1
Authority
EP
European Patent Office
Prior art keywords
display
touchscreen display
visual content
frame
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP23848749.0A
Other languages
German (de)
French (fr)
Inventor
Hsin Hsueh Li
Chien-Hui Wen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google LLC filed Critical Google LLC
Publication of EP4602448A1 publication Critical patent/EP4602448A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G06F1/325—Power saving in peripheral device
    • G06F1/3262—Power saving in digitizer or tablet
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
    • G06F1/3215—Monitoring of peripheral devices
    • G06F1/3218—Monitoring of peripheral devices of display devices
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G06F1/325—Power saving in peripheral device
    • G06F1/3265—Power saving in display device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/001—Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2330/00—Aspects of power supply; Aspects of display protection and defect management
    • G09G2330/02—Details of power systems and of start or stop of display operation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user

Definitions

  • Visual content presented on areas of the touchscreen display that are under a user’s hand or fingers can be presented with a lesser display performance compared to other areas of the touchscreen display.
  • the size and shape of the area in which the display performance is reduced can vary based on various factors such as a location of the user’s touch, a size and shape of the touch, an orientation or movement of the device, a type of user interface being presented by the touchscreen display, or any combination thereof.
  • the displayperformance of a region of the display can be reduced, for example, by decreasing display brightness, by decreasing color depth, by reducing refresh rate, and/or by reducing spatial resolution of the region of the display. Display performance of the region of the display can be reduced while presenting the same visual content that would be presented at higher levels of display performance.
  • Embodiment 13 is the computer-implemented method of any one of the embodiments 1 through 12, wherein: the different levels of display performance include different color depths for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance for the different portions of the frame of visual content includes: (i) presenting the first portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a first number of bits; and (ii) presenting the second portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a second number of bits that is lower than the first number of bits.
  • Embodiment 14 is the computer-implemented method of any one of the embodiments 1 through 13, comprising: determining, by the computing device, that the user input has ended its interaction with the touchscreen display; and presenting, by the computing device after having determined that the user input ended its interaction with the touchscreen display, another frame of visual content on the touchscreen display, using a same level of display performance for the first portion of the another frame and the second portion of the another frame, as a result of having determined that the user input ended its interaction with the touchscreen display.
  • Embodiment 21 is the computing system of any one of the embodiments 15 through 20, wherein: the operations comprise determining, by the computing system, an indication of an orientation of a user finger that provided the first user contact at the first location of the touchscreen display, based on analysis of a shape of an area of the first user contact at the first location; and identifying the selected region of the touchscreen display includes identifying the selected region based on the determined orientation of the user finger; and the selected region includes: (1) the area of the first user contact at the first location; and (2) a second area outside the first user contact at the first location, and that is between the area of the first user contact at the first location and an edge of the touchscreen display along the determined orientation of the user finger, based on the determined orientation of the user finger, such that the second area represents area under the user finger and not contacted by the user finger during the first user contact.
  • Display performance can be reduced in areas of the display that are concealed or likely concealed by an object such as a user’s hand.
  • Display performance can be reduced in a portion of the display, for example, by adjusting a spatial resolution of the portion of the display, by adjusting a refresh rate of the portion of the display, by adjusting a color depth of the portion of the display, by adjusting a brightness of the portion of the display, or any combination of these. Reducing display performance in a portion of the display can reduce the amount of power consumed by the display and improve battery life of the power saving device.
  • FIG. 1 shows an example electronic device 190 with a display 104.
  • the electronic device 190 can be a smart phone, a tablet computer, a laptop computer, a television, a smart watch, or a handheld game console.
  • the electronic device 190 is a mobile telephone and the display 104 is a display panel of the mobile telephone.
  • the display 104 is configured to sense contact by a user.
  • FIG. 1 illustrates an individual contacting the display 104 with finger 101.
  • the display 104 can be configured to receive a touch input and generate a touch input signal based on the location of the touch.
  • the display 104 can also generate the touch input signal based on other characteristics of the touch, e.g., a number of touches, a path of a touch swipe, a size or shape of the touch input, etc.
  • the touch input signal can indicate various properties of the touch, such as the location on the display 104, the area covered by the touch, the direction and speed of movement of the touch (e.g., the path or trajectory), and so on.
  • FIG. 2 is a diagram of an example display system 100 of computing device 190.
  • the device 190 includes a display 104 housed in a chassis 109.
  • a region of the device 190 between the edge of the display 104 and the edge of the chassis is a bezel region 103.
  • a frame time is an amount of time between a start of a frame and a start of a next frame.
  • the frame time can be the inverse of a frame rate of a display system. For example, a frame rate of 60 frames per second (fps) corresponds to a frame time of one- sixtieth of a second, or 0.0167 seconds.
  • the pixel array 112 extends in a plane and includes rows and columns. Each row extends horizontally across the pixel array 112. For example, the first row 120 of the pixel array 112 includes pixels Pl 1, P21, and P31. Each column extends vertically down the pixel array 112. For example, the first column 130 of the pixel array 112 includes pixels Pl 1, P12, P13, and P14. Only a few pixels are shown in FIG. 1 for simplicity. In practice, there may be thousands or millions of pixels in the pixel array 112. Increasing the numbers of pixels in a display that remains the same size results in a higher image resolution.
  • the display system 100 includes a display driver integration circuit (DDIC) 106 that receives display input data 102.
  • the display input data 102 can include color values for each pixel of the pixel array 112. The color value for a pixel corresponds with a color to be emitted by the pixel.
  • the display input data 102 can include brightness values for each pixel of the pixel array 112. The brightness value for a pixel corresponds with a brightness of the light to be emitted by the pixel.
  • the display input data 102 includes a pixel value that incorporates both color data and brightness data.
  • the RGB values of typical digital images do not directly correspond to the physical light intensities, but are rather compressed by a gamma correction function. This transformation better utilizes the limited number of bits in the encoded image by choosing a gamma value that matches the non-linear human perception of luminance.
  • the display input data 102 can include a gamma corrected pixel value for each subpixel of each pixel of the array 112. Addressing a pixel using the gamma corrected pixel value causes the pixel to emit light at the color and brightness specified by the gamma corrected pixel value.
  • the DDIC 106 receives the display input data 102 from a system-on-chip (SoC) 105.
  • SoC system-on-chip
  • the SoC 105 is a microchip with all the necessary electronic circuits and parts for a given system, such as a smartphone or wearable computer, on a single integrated circuit (IC).
  • the SoC 105 is an integrated circuit that includes multiple components on a single chip.
  • the SoC 105 can include, for example, a processor, a memory 306, and input/output (I/O) ports.
  • the SoC 105 can be implemented on a single substrate, such as silicon.
  • the SoC 105 can process digital signals, analog signals, and mixed signals.
  • the DDIC 106 can be, for example, a semiconductor integrated circuit or a state machine.
  • the DDIC 106 generates signals with suitable voltage, current, timing, and demultiplexing to cause a display 104 to show images according to display input data 102.
  • the DDIC 106 can be a microcontroller and may incorporate RAM, Flash memory, EEPROM, ROM, etc.
  • the DDIC 106 drives the pixel array 112 to emit light according to the display input data 102.
  • the data signal generator 138 of the DDIC 106 generates image data signals 144 from the display input data 102 and provides the image data signals 144 to the data drivers 110.
  • the image data signals 144 can include voltages for each subpixel of the pixel array 112 to drive the subpixels to emit light at a color and brightness specified by the display input data 102.
  • the DDIC 106 includes a timing controller 134, a clock signal generator 136, and a data signal generator 138.
  • the DDIC 106 generates control signals 142.
  • the control signals 142 can include, for example, signals that control a display frame start time and a display frame stop time of each frame presented by the display 104, where a frame represents a single image in a sequence of images that are presented by the display 104.
  • the control signals 142 or other signals not illustrated in FIG. 1 can control a display emission start time and a display emission stop time of each emission cycle of the display 104.
  • the SCAN/EM drivers 108, the data drivers 110, or both can be integrated with the DDIC 106.
  • the SCAN/EM drivers supply SCAN and EM signals to rows of the pixel array 112.
  • the SCAN/EM drivers 108 supply scan signals via scan lines SI to S4, and EM signals via EM lines El to E4, to the rows of pixels, with each row of pixels in the pixel array 112 being addressed by a scan line and a corresponding emission line.
  • the first row 120 of the pixel array 112 is addressed by scan line SCAN1 and emission line El.
  • the data drivers 110 supply data voltages via the data lines D 1 to D3.
  • each of the data lines DI to D3 represent multiple data lines.
  • the pixel P 11 can include three subpixels (e. g. , P 11 R for a red subpixel, P 11 G for a green subpixel, and Pl IB for a blue subpixel), and the data line DI can represent three corresponding data lines, each addressing a corresponding subpixel of pixel Pl 1.
  • the display system 100 includes a power supply 150.
  • the power supply 150 provides a first supply voltage ELVDD and a second supply voltage ELVSS, both of which are provided to each pixel in the pixel array 112.
  • the power supply 150 can be integrated with the DDIC 106.
  • Each pixel in the pixel array 112 is addressable by a horizontal scan line, a horizontal EM line, and a vertical data line.
  • the pixel Pl 1 is addressable by the data line DI, the scan line SI, and the EM line El.
  • the pixel P23 is addressable by the data line D2, the scan line S3, and the EM line E3.
  • FIG. 2 illustrates that each row is addressed by a single scan line, each row may be addressed by multiple scan lines (e g., nSCAN and pSCAN).
  • FIG. 1 illustrates example components of an OLED display
  • the described techniques may be applied to other flat panel display technologies that include an array of pixels.
  • the techniques can be applied to curved displays, flexible displays, foldable displays, and rollable displays.
  • the technology may be applied to LED displays, LCD displays, plasma display panels (PDP), and CRT displays.
  • the technology can also be applied to projectors (e.g., digital light processing projectors) to reduce power consumption and to reduce the amount of heat absorbed and dissipated by the projector.
  • projectors e.g., digital light processing projectors
  • FIG. 3 shows a diagram of a pixel circuit of a display device, which pixel circuit includes an LED and corresponding drive circuitry for the pixel circuit.
  • FIG. 3 may illustrate a more detailed view of a single pixel from the array of pixels shown in FIG. 1. While this disclosure sometimes refers to the components shown in FIG. 3 as a “pixel circuit”, this disclosure may also refer to such components as simply a “pixel.” Further, the pixel shown in FIG. 3 can represent a sub-pixel.
  • the pixel circuit may be an active matrix OLED (AMOLED) pixel circuit.
  • the pixel circuit receives an emission signal (EM) on an emission line, SCAN signals on scan signal lines, and a data voltage (VDATA) signal on a data line.
  • the pixel circuit 200 receives a first supply voltage ELVDD on a first voltage supply line, a second supply voltage ELVSS on a second voltage supply line, and an initial reference voltage VINIT on an initial voltage supply line.
  • the pixel circuit includes an organic light-emitting diode (OLED).
  • OLED organic light-emitting diode
  • the OLED includes a layer of an organic compound that emits light in response to an electric current, IOLED.
  • the organic layer is positioned between two electrodes: an anode and a cathode.
  • the OLED is driven by a driving transistor Tl, which receives the supply voltage ELVDD and acts as a current source that drives the OLED to emit light.
  • the pixel also includes a storage capacitor CST and transistors T2 through T7.
  • the operation of the pixel is defined by states of the control signals SCAN, EM, and VDATA.
  • An amount/level of the OLED current (IOLED) is set by a voltage present at a gate terminal of the driving transistor Tl, referred to herein as the “G” node.
  • FIG. 4 shows a timing diagram of the control signals provided to and received by the pixel shown in FIG. 3. These control signals repeatedly transition during operation of the display system 100 between an initialization stage, a programming stage, and an emission stage.
  • the EM signal transitions to an off state (e.g., by changing from a low state to a high state). This transition turns off transistors T5 and T6, which interrupts current being provided from ELVDD to the OLED, therefore stopping light emission by the OLED. Since the EM signal may be provided to an entire line of pixels, this transition can turn off all pixels in the line of pixels.
  • the SCAN[n-l] signal turns to an on state (e.g., by changing from a high state to a low state), which turns on transistor T4 for a period of time and initializes the G node to the initialization voltage VINIT. Since the SCAN[n-l] signal may be provided to an entire line of pixels, this initialization stage can erase the data values that were previously stored at each pixel in the line of pixels.
  • the SCAN[n-l] signal may be the SCAN[n] signal provided to a preceding row by a state machine of the SCAN/EM drivers 108.
  • the EM signal turns to an on state (e.g., by going low), which turns on transistors T5 and T6.
  • a current level provided to the OLED in each pixel is determined by the voltage present at the Gnode of the pixel (e.g., with the G node voltage level having been programmed by the voltage data VDATA line).
  • An intensity or brightness of light emitted by the OLED directly correlates to an amount of electrical current IOLED applied to the OLED, with higher current corresponding to a greater intensity of light than a lower current.
  • the storage capacitor CST maintains the voltage at the G node, so that the OLED continues to emit light at roughly the same level for a duration of the emission stage.
  • the voltage at the G node may decrease slightly during the emission stage.
  • the current IOLED applied to the OLED and the intensity of light emitted by the OLED may decrease or increase slightly during the emission stage, depending on a type of pixel circuit design (e.g., with p-channel transistors in the pixel circuit, lower voltage levels at the G node cause higher IOLED and higher intensity of OLED light).
  • FIG. 5 shows a block diagram of a system 500 for operating a power saving device with variable touchscreen display performance.
  • the system 500 includes the SoC 105 and the DDIC 106 of the display system 100.
  • the SoC 105 includes a memory 306, a processor 304, and a NPU 311.
  • the processor 304 can be, for example, a graphical processing unit (GPU), a data processing unit (DPU), or a central processing unit (CPU).
  • the processor 304 can include, for example, a bus interface, a power management unit, a video processing unit, a graphics memory controller, a display interface, or any combination of these.
  • the processor 304 can include a digital signal processor (DSP).
  • DSP digital signal processor
  • the DSP can perform signal processing operations such as data collection and data processing.
  • the processor 304 can generate visual content data, such as a frame of a video.
  • the visual content data may be for a video sequence that is pre-rendered, e.g., for a fdm.
  • the visual content data may be for a video sequence that is dynamically generated, e.g., for a video game or for user navigation through various operating system screens and menus.
  • the visual content data can be compressed, using any appropriate method.
  • the visual content data can be uncompressed.
  • the processor 304 can store the generated visual content data in the memory 306.
  • the memory 306 may be any appropriate type of memory.
  • the memory 306 can be a random access memory (RAM).
  • the SoC 105 can identify image data 310 (e.g., by receiving the image data 310 from another source or by generating the image data 310).
  • the image data 310 can be, for example, image data for an image frame.
  • the SoC 105 can receive, as input, ambient brightness data 330, accelerometer data 332, application data 336, or any combination of these.
  • the ambient brightness data 330 can indicate brightness of the environment in which the device 190 is located.
  • the device 190 includes a light sensor configured to detect ambient brightness.
  • the SoC 105 can receive an indication of an amount of light sensed by a light sensor of the device 190, and can determine the ambient brightness using the indication of the amount of light sensed by the light sensor.
  • the SoC 105 modifies a display brightness setting based on the amount of light sensed by the light sensor, for example, to increase an overall brightness of the display when in high ambient light environments.
  • the ambient brightness data 330 can affect the amount by which the image data 310 is modified by the SoC 105.
  • the SoC 105 can select a larger area of the display within which to present visual content using a reduced level of display performance.
  • the SoC 105 can modify the image data 310 by a greater amount in order to achieve a greater amount of power savings.
  • the SoC 105 only presents visual content using a reduced level of display performance when ambient brightness and/or a display brightness level exceeds a threshold (e.g., such that image data is modified only when the display is operating at a high brightness level when image modification has a greatest effect).
  • the accelerometer data 332 can include data indicating device orientation, device movement, or both.
  • the accelerometer data 332 can be generated by an accelerometer of the device 190 or any movement sensor or orientation sensor of the device 190.
  • An accelerometer is an electromechanical device that measures the force of acceleration caused by movement, by acceleration (such as acceleration due to gravity), and by vibration.
  • the accelerometer is a dynamic accelerometer that detects a change in orientation or tilt of the display. In response to detecting the change in orientation of the display, the display can change between portrait mode and landscape mode.
  • the accelerometer can also detect movement such as movement of a user walking while carrying the device, or movement of a vehicle in which the device is located.
  • the accelerometer data 332 can affect the portions of the display that are visible to the user.
  • different areas of the display may be visible to the user when the user holds the device 190 in a landscape orientation compared to a portrait orientation. Additionally, different areas of the display may be visible to the user when the device 190 is mounted to a dashboard of a moving vehicle compared to when the device 190 is placed on a stationary desk.
  • the application data 336 can include data indicating applications operated by the device 190.
  • the application data 336 includes data indicating a user interface presented on the display 104.
  • the application data 336 can indicate that the device 190 is operating an application that presents videos, and that the display 104 is presenting a display that shows a list of videos and a scrolling user interface element.
  • the application data 336 can indicate that the device 190 is operating an application that presents videos, and that the display 104 is presenting a display that shows a single video and user interface elements for controlling the video.
  • the application data 336 can affect the portions of the display that are likely to be touched by the user, and the orientation of the user’s finger when touching the display. Thus, the application data 336 can affect the portions of the display that are visible to the user.
  • the touch input signal 340 indicates that a touch input was received by the display 104.
  • the touch input signal can indicate characteristics of the touch such as a number of touches, a path of a touch swipe, a size or shape of the touch input, the location of the touch on the display 104, a duration of the touch, the area covered by the touch, the direction and speed of movement of the touch, or any combination of these.
  • the SoC 105 can store computational interaction models representing areas of the display that are concealed or likely concealed from the user during user interaction with the display 104.
  • the interaction models can include heuristic models, machine learning models, or both.
  • the memory 306 stores heuristic (i.e., rule-based) interaction models.
  • the processor 304 can access the heuristic models in the memory 306 to determine whether or not to adjust display performance and to determine how to adjust display performance.
  • the processor 304 can select a particular heuristic model from the memory 306 based on the ambient brightness data 330, the accelerometer data 332, the application data 336, the touch input signal 340, or any combination thereof.
  • the processor 304 can use the particular heuristic model to identify areas of the display 104 in which display performance is to be reduced, to determine an extent of display performance reduction for the identifies areas of the display, to determine a mechanism for reducing the display performance, to determine a duration for reducing the display performance, or any of these.
  • the first heuristic model can specify a mechanism for reducing the display performance (e.g., by reducing pixel brightness).
  • the first heuristic model can specify an amount by which to reduce the display performance (e.g., a percentage brightness reduction).
  • the first heuristic model can specify a duration for reducing the display performance (e.g., until at least one second has passed after detecting a touch input).
  • the NPU 311 stores machine learning interaction models.
  • the processor 304 can access the machine learning models in the NPU 311 to determine whether or not to adjust display performance and to determine how to adjust display performance.
  • the processor 304 provides input to the machine learning interaction models such as the ambient brightness data 330, the accelerometer data 332, the application data 336, the touch input signal 340, or any combination thereof.
  • the processor 304 can obtain output from the machine learning models indicating areas of the display 104 in which display performance is to be reduced, to determine an extent of display performance reduction for the identifies areas of the display, to determine a mechanism for reducing the display performance, to determine a duration for reducing the display performance, or any of these.
  • the processor 304 generates modified image data 302.
  • the processor can modify the image data 310 based on heuristic and/or machine learning models to produce the modified image data 302.
  • the modified image data 302 when presented by the display 104, presents the same content as the image data 310 but with a reduced display performance for at least part of the content.
  • the image data 310 can include any combination of user interface elements, icons, text, photographs, graphs, animations, and videos.
  • the modified image data 302 presents the same user interface elements, icons, text, photographs, graphs, animations, and videos as the image data 310.
  • the modified image data 302 can be modified in various ways to present the visual content with reduced display performance in order to reduce power consumption.
  • the processor 304 can modify the image data 310 by applying a brightness dimming mask to the image data 310. After applying a brightness dimming mask to the image data 310, a portion of a frame of the modified image data 302 is dimmer than the same portion of the same frame of the image data 310, for example, while another portion of the frame of modified image data 302 is the same as a corresponding portion of the frame of the image data 310.
  • the processor 304 can modify the image data 310 by adjusting bit ranges of the image data 310. After adjusting the bit ranges of the image data 310, at least a portion of a frame of the modified image data 302 has a reduced bit range compared to the same portion of the same frame of the image data 310. Therefore, for a frame of modified image data 302, some pixels of the frame have a greater bit range (e.g., ten bits) compared to the bit range of other pixels of the frame (e.g., six bits).
  • the processor 304 can modify the image data 310 by adjusting a spatial resolution of the image data 310. After adjusting the spatial resolution of the image data 310, at least a portion of a frame of the modified image data 302 has a reduced spatial resolution compared to the same portion of the same frame of the image data 310. Therefore, for a frame of modified image data 302, some areas of the frame include clusters of pixels addressed with the same pixel values, and other areas of the frame include individual pixels that are each addressed with a respective pixel value.
  • the SoC 105 can send instructions to the DDIC 106 that adjust the display performance when the display 104 presents the image data 310.
  • the SoC 105 can instruct the DDIC to provide adjusted data values to a subset of pixels that are within the selected region of the display 104 in order to reduce the current provided to the subset of the pixels.
  • the SoC 105 can instruct the DDIC to reduce a refresh rate of a subset of pixels that are within the selected region of the display 104.
  • the SoC 105 can instruct the DDIC to reduce a refresh rate of the subset of pixels from 240 Hz to 120 Hz.
  • the subset of pixels that are within the selected region will refresh at a reduced frequency of 120 Hz, while pixels outside of the selected region will refresh at a nonreduced frequency of 240 Hz.
  • FIGS. 6A-C show examples of using interaction models to identify a selected region of a touchscreen display based on user input.
  • the display 104 is divided into multiple zones represented by a grid of multiple grid segments.
  • Each grid segment represents a region of the display 104 that presents a corresponding portion of frames of visual content.
  • each grid segment is a square or rectangular area of the display.
  • Each grid segment can include a group of pixels of the display 104 that are within the same region or zone of the display 104.
  • the SoC 105 receives, as input data, a touch input signal 340 indicating occurrence of the touch and a location of the touch.
  • the touch input signal 340 can also indicate the shape and/or orientation of the touch (e.g., as determined by an algorithm executed by the SoC 105).
  • the SoC 105 can receive additional input data such as the accelerometer data 332, the ambient brightness data 330, and the application data 336.
  • the processor 304 accesses an interaction model from the memory 308 and/or the NPU 311.
  • the processor 304 can send an indication of the user input that contacted the touchscreen display to a computational interaction model which can be a heuristic mode or a machine learning model.
  • the processor 304 receives, from the computational model, an indication of the selected region in which display performance is to be reduced.
  • the processor 304 receives, from the computational model, information that the processor 304 uses to identify the selected region in which display performance is to be reduced.
  • the processor 304 selects a particular heuristic interaction model based on the input data, and the particular interaction model indicates a heat map of the display. Areas of higher usage and lower usage can be identified with the heat map, where areas of high usage are classified as “hot” and areas of lower usage are classified as “cool.” For example, each grid segment can be associated with a percentage of likelihood that the grid segment is under the user’s hand or finger, and classified as hot or cool based on the percentage of likelihood. In some examples, heat maps of high and low usage can be stored in the memory 306. The heat maps can be associated with different applications, user interfaces, orientations, movements, or any combination of these.
  • the processor 304 provides the input data to a machine learning interaction model and receives a heat map of the display as output from the machine learning interaction model.
  • machine learning models can record user behavior and determine highly used areas of the touchscreen display. The user behavior can be dependent on types of applications in use and/or types of user interfaces being presented on the display 104.
  • the computing system may store a first heat map to use when the user device is presenting a user interface for a video-streaming application in landscape mode (e.g., showing “hot” areas at the left and right sides of the display at which thumbs overlap the display).
  • the same computing system may store a second heat map to use when the user device is presenting a user interface for a web browser in portrait mode, and when user interaction has been recently received at a right edge of the display (e.g., showing a hot area along a right edge of the display where a vertical scroll bar of the web browser is presented).
  • the device 190 determines an indication of an orientation of the finger 101 that touched the display 104.
  • the device 190 can determine the indication of the orientation based on analysis of a shape of an area of the user contact.
  • the device 190 identifies the selected region based at least in part on the determined orientation of the user finger. For example, the orientation of the finger 101 in FIG. 6A is diagonal from a top left comer of the display 104 towards a bottom right comer of the display 104. The touch location of the finger 101 is nearer to the bottom of the display 104 than to the top of the display 104. Therefore, the processor 304 can determine a higher likelihood of a bottom right comer region of the display 104 being concealed from the user compared to other regions of the display 104.
  • Step 612 shows an example heat map, with darker patterned grid segments representing areas of the display that are more likely concealed from the user, and lighter patterned grid segments representing areas of the display that are less likely concealed from the user.
  • the grid segment 602 is the darkest grid segment because the grid segment 602 corresponds to the touch location of the finger 101.
  • Grid segments around the grid segment 602 and between the grid segment 602 and the edge of the display 104 are shaded with patterns of various darkness.
  • the heat map specifies, for each grid segment, a probability that the grid segment is concealed from the user.
  • the processor 304 selects a region of the display 104 based on the heat map. In some examples, the processor 304 selects grid segments for which the probability that the grid segment is concealed from the user satisfies a threshold probability. In the example of FIG. 6A, the processor 304 selects grid segments within boundary 604.
  • the boundary 604 encompasses the grid segment 602 at which the finger 101 touched the display 104.
  • the boundary 604 encompasses an area of the display 104 that is larger than the area of the touch by the finger 101.
  • the area encompassed by the boundary 604 can be considered a “shadow area” of the display 104.
  • the device 190 presents visual content using different levels of display performance for region 606, which is within the boundary 604, and for region 608, which is outside of the boundary 604.
  • the device 190 presents the same visual content in the region 606 as would be presented without adjusting the display performance.
  • the image presented by the display 104 is the same when using the different levels of display performance as when using the same levels of display performance.
  • the display performance used to present the visual content in the region 606 is reduced compared to the display performance used to present the visual content in the region 608.
  • the visual content presented in the region 606 can have a reduced brightness, a reduced color depth, a reduced spatial resolution, a reduced refresh rate, or any of these, compared to the region 608.
  • FIG. 6B provides another example of using an interaction model to identify a selected region of a touchscreen display based on user input., where the interaction model specifies a gradient indicating an amount by which display performance is to be reduced throughout the selected region.
  • the SoC 105 receives, as input data, a touch input signal 340 indicating occurrence of the touch and a location of the touch.
  • the touch input signal 340 can also indicate the shape and/or orientation of the touch.
  • the SoC 105 can receive additional input data such as the accelerometer data 332, the ambient brightness data 330, and the application data 336.
  • the processor 304 accesses an interaction model from the memory 308 and/or the NPU 311. In some examples, the processor 304 selects a particular heuristic interaction model based on the input data, and the particular interaction model indicates a region 626 of the display in which visual content is to be presented with a reduced display performance.
  • the processor 304 provides the input data to a machine learning interaction model and receives, as output from the machine learning interaction model, an identified region 626 of the display in which visual content is to be presented with a reduced display performance.
  • the interaction model includes a heat map that indicates an amount by which display performance is to be reduced in each grid segment. For example, segment 624, represented at step 622 by black shading, is to have a greatest amount of display performance reduction. Other grid segments within the region 626 are to have reduced display performance by varying amounts represented by the amount of shading in each grid segment.
  • the device 190 thus presents content in the region 626 with degraded display performance according to the heat map.
  • the device 190 presents content in the region 628, outside of the region 626, without reducing the display performance.
  • FIG. 6C provides another example of using an interaction model to identify a selected region of a touchscreen display based on user input., where the interaction model specifies a uniform reduction in display performance throughout the selected region.
  • the SoC 105 receives, as input data, a touch input signal 340 indicating occurrence of the touch and a location of the touch.
  • the touch input signal 340 can also indicate the shape and/or orientation of the touch.
  • the SoC 105 can receive additional input data such as the accelerometer data 332, the ambient brightness data 330, and the application data 336.
  • the processor 304 accesses an interaction model from the memory 308 and/or the NPU 311. In some examples, the processor 304 selects a particular heuristic interaction model based on the input data, and the particular interaction model indicates a region 636 of the display in which visual content is to be presented with a reduced display performance. [00125] In some examples, the processor 304 provides the input data to a machine learning interaction model and receives, as output from the machine learning interaction model, an identified region 636 of the display in which visual content is to be presented with a reduced display performance.
  • the level of display performance is adjusted uniformly throughout the region 636.
  • the level of display performance within the region 636 can be reduced by the same amount throughout the region 636.
  • the device 190 thus presents visual content in the region 636 with uniformly degraded display performance.
  • the device 190 presents visual content in the region 638, outside of the region 636, without reducing the display performance.
  • FIG. 7 shows a flowchart of a process 700 for operating a power saving device with variable touchscreen display performance.
  • the process may be implemented by a display device, a computing device that includes the display device, or one or more components thereof or in communication with the computing device or display device.
  • a computing system receives display content.
  • the computing system can be the device 190, described with respect to FIGS. 1-5, which can receive image data 310 including display content for presentation on the display 104 of the device 190.
  • the computing system may receive the image data 310 by generating the image data 310 and transferring the image data 310 from one component of the computing system to another component of the computing system.
  • the computing system receives user input that contacts a touchscreen display.
  • the device 190 can receive a touch input signal 340 representing contact with the display 104 by an object such as the user’s finger 101.
  • the user input includes first user contact at a first location of the touchscreen display 104.
  • the computing system detects the user input over a period of time during which the computing system presents multiple frames on the touchscreen display.
  • the device 190 can detect a first user contact that extends for a duration of presenting multiple frames.
  • the user input includes (in addition to the first user contact) second user contact at a second location of the touchscreen display that is different from the first location.
  • the device 190 can detect a first touch at a first location of the display 104 and a second touch at a second location of the display 104. The first location and the second location may correspond to locations at which the display 104 presented different user elements.
  • the computing system identifies a selected region of the touchscreen display based on a location of the user input.
  • the device 190 can identify the selected region of the touchscreen display 104 based on having received the touch by the user’s finger 101.
  • the selected region can include the first location at which the user input contacted the touchscreen display and can be larger than an area of the first user contact at the first location of the touchscreen display.
  • the selected regions 606, 626, and 636 are each larger than an area of the user contact with the finger 101.
  • the computing system generates data that represents a frame of visual content.
  • the SoC 105 of the device 190 can generate, or receive from another component of the computing system, image data 310 that represents a frame of visual content.
  • the computing system identifies a first portion of the frame that corresponds to an area outside of the selected region. For example, referring to FIG. 6 A, region 608 is outside of the boundary 604 of the selected region 606.
  • the computing device 190 identifies a first portion of the frame of visual content that corresponds to the region 608.
  • the computing system identifies a second portion of the frame that corresponds to an area within the selected region. For example, referring to FIG. 6A, region 606 is within the boundary 604.
  • the computing device 190 identifies a second portion of the frame of visual content that corresponds to the region 606.
  • the computing system presents the frame of visual content.
  • the computing system presents the frame of visual content using different levels of display performance to depict different portions of the visual content.
  • the device 190 can present the second portion of the frame corresponding to the region 606 using a reduced level of display performance than the first portion of the frame corresponding to the region 608.
  • the computing system presents the first portion of the frame with a first level of display performance.
  • the first portion of the frame of visual content is displayed by the area outside of the selected region of the touchscreen display.
  • the first portion of the frame can be displayed by the region 608.
  • the computing system presents the second portion of the frame with a second level of display performance.
  • the second portion of the frame is displayed by the area within the selected region of the touchscreen display.
  • the second portion of the frame can be displayed by the region 606.
  • the second level of display performance is lower than the first level of display performance.
  • the different levels of display performance include different brightness levels for the different portions of the frame of visual content.
  • the first portion of the frame of visual content is presented with a first level of brightness
  • the second portion of the frame is presented with a second level of brightness that is less than the first level of brightness.
  • the second portion of the frame presented by the region 606 can have a reduced brightness compared to the first portion of the frame presented by the region 608.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

Methods, systems, and program products for presenting display content on a display of a computing system are disclosed. A method includes receiving user input that contacts the touchscreen display, including first user contact at a first location of the touchscreen display; identifying a selected region of the touchscreen display based on the user input, the selected region including the first location; generating data that represents a frame of visual content for presentation by the touchscreen display; and presenting the frame of visual content by: (i) presenting a first portion of the frame of visual content with a first level of display performance, the first portion of the frame corresponding to an area outside the selected region; and (i) presenting a second portion of the frame of visual content with a second level of display performance, the second portion of the frame corresponding to an area within the selected region.

Description

POWER SAVING DEVICE WITH VARIABLE TOUCHSCREEN DISPLAY PERFORMANCE
TECHNICAL FIELD
[0001] This document generally relates to display devices.
BACKGROUND
[0002] Electronic devices can include display devices on which visual images are shown. Higher levels of display performance can enhance user experience. Higher levels of display performance generally result in high power consumption and reduced battery life.
SUMMARY
[0003] This document describes techniques, methods, systems, and other mechanisms for providing a power saving device with variable touchscreen display performance. Users interact with applications operating on the device by contacting the touchscreen display. When contacting the touchscreen display with a hand, some regions of the touchscreen display will be concealed from the user. Areas of the touchscreen display that are under the shadow of the user’s hands and fingers are therefore less visible to the user than other areas of the touchscreen display.
[0004] The areas of the touchscreen display that are under the hands and fingers of the user can vary over time while different applications are in use and while different user interfaces are presented on the touchscreen. The regions of the touchscreen display that are concealed from the user can also vary depending on the angle or orientation of the device and the location of the device relative to the user. For example, users can contact the touchscreen display while holding the device in one hand, while holding the device in two hands, and while the device is held by a support structure such as vehicle dashboard mount.
[0005] Visual content presented on areas of the touchscreen display that are under a user’s hand or fingers can be presented with a lesser display performance compared to other areas of the touchscreen display. The size and shape of the area in which the display performance is reduced can vary based on various factors such as a location of the user’s touch, a size and shape of the touch, an orientation or movement of the device, a type of user interface being presented by the touchscreen display, or any combination thereof. The displayperformance of a region of the display can be reduced, for example, by decreasing display brightness, by decreasing color depth, by reducing refresh rate, and/or by reducing spatial resolution of the region of the display. Display performance of the region of the display can be reduced while presenting the same visual content that would be presented at higher levels of display performance.
[0006] Varying display performance can reduce power consumption of display devices, while maintaining image clarity and readability. The disclosed techniques can be used to reduce power consumption while maintaining or enhancing display quality experienced by a user. Reducing display performance around an area of the display where the user is touching the display is likely imperceptible to the user. Variably reducing display performance in some regions of the display can result in reduced power consumption and extended battery life, for example, while providing a same user experience.
[0007] As additional description to the embodiments described below, the present disclosure describes the following embodiments.
[0008] Embodiment 1 is directed to a computer-implemented method, comprising: receiving, by a computing device with a touchscreen display, user input that contacts the touchscreen display, including first user contact at a first location of the touchscreen display; identifying, by the computing device, a selected region of the touchscreen display based on having received the user input, the selected region including the first location at which the user input contacted the touchscreen display; generating, by the computing device, data that represents a frame of visual content for presentation by the touchscreen display; and presenting, by the computing device on the touchscreen display, the frame of visual content, using different levels of display performance to depict different portions of the visual content, including by: (i) presenting a first portion of the frame of visual content with a first level of display performance, the first portion of the frame corresponding to and displayed by an area outside the selected region of the touchscreen display; and (i) presenting a second portion of the frame of visual content with a second level of display performance that is lower than the first level of display performance, the second portion of the frame corresponding to and displayed by an area within the selected region of the touchscreen display.
[0009] Embodiment 2 is the computer-implemented method of embodiment 1, wherein the computing device is configured, such that the computing device would have consumed more power presenting the second portion of the frame of visual content with the first level of display performance, than the computing device consumed presenting the second portion of the visual content with the second level of display performance.
[0010] Embodiment 3 is the computer-implemented method of any one of the embodiments 1 or 2, comprising: detecting, by the computing device, the user input over a period of time during which the computing device presents multiple frames on the touchscreen display, the user input including: (i) the first user contact at the first location of the touchscreen display; and (ii) a second user contact at a second location of the touchscreen display that is different from the first location, the second user contact initiating contact with the touchscreen display after the first user contact released from being in contact with the touchscreen display, such that during a time between the first user contact ending and the second user contact beginning, there was no user contact with the touchscreen display, wherein the selected region contiguously includes an area of the first user contact at the first location and an area of the second user contact at the second location.
[0011] Embodiment 4 is the computer-implemented method of any one of the embodiments 1 through 3, comprising: presenting, by the computing device, multiple additional frames of visual content using the different levels of display performance, after having presented the frame of visual content using the different levels of display performance.
[0012] Embodiment 5 is the computer-implemented method of any one of the embodiments 1 through 4, wherein: the computing device presents the multiple additional frames of visual content over a period of time during which additional user input both contacts the selected region of the touchscreen display and does not contact the touchscreen display, such that the computing device presents frames of visual content using the different levels of display performance through a period of time during which no user input is contacting the touchscreen display, following an end of a first user contact and a beginning of a second user contact.
[0013] Embodiment 6 is the computer-implemented method of any one of the embodiments 1 through 5, wherein the selected region of the touchscreen display includes and is larger than an area of the first user contact at the first location of the touchscreen display.
[0014] Embodiment 7 is the computer-implemented method of any one of the embodiments 1 through 6, wherein: the method comprises determining, by the computing device, an indication of an orientation of a user finger that provided the first user contact at the first location of the touchscreen display, based on analysis of a shape of an area of the first user contact at the first location; and identifying the selected region of the touchscreen display includes identifying the selected region based on the determined orientation of the user finger; and the selected region includes: (1) the area of the first user contact at the first location; and (2) a second area outside the first user contact at the first location, and that is between the area of the first user contact at the first location and an edge of the touchscreen display along the determined orientation of the user finger, based on the determined orientation of the user finger, such that the second area represents area under the user finger and not contacted by the user finger during the first user contact.
[0015] Embodiment 8 is the computer-implemented method of any one of the embodiments 1 through 7, wherein identifying the selected region of the touchscreen display includes: sending, to a computational model, an indication of the user input that contacted the touchscreen display; and receiving, from the computational model, an indication of the selected region, or information that the computing device uses to identify the selected region. [0016] Embodiment 9 is the computer-implemented method of any one of the embodiments 1 through 8, wherein the computing device identifies the selected region of the touchscreen display based on: (i) locations at which the user input contacted the touchscreen display; and (ii) data received from orientation and/or movement sensors of the computing device during receipt of the user input, such that the computing device is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of different data being received from the orientation and/or movement sensors during receipt of the user input.
[0017] Embodiment 10 is the computer-implemented method of any one of the embodiments 1 through 9, wherein the computing device identifies the selected region of the touchscreen display based on: (i) locations at which the user input contacted the touchscreen display; and (ii) a type of user interface being presented by the touchscreen display during receipt of the user input, such that the computing device is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of a different type of user interface being presented during receipt of the user input.
[0018] Embodiment 11 is the computer-implemented method of any one of the embodiments 1 through 10, wherein: the different levels of display performance include using different brightness levels for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance to depict the different portions of the visual content includes: (i) presenting the first portion of the frame of visual content with a first level of brightness; and (ii) presenting the second portion of the frame of visual content with a second level of brightness that is less than the first level of brightness. [0019] Embodiment 12 is the computer-implemented method of any one of the embodiments 1 through 11, wherein: the method comprises identifying data that specifies different levels of intensity for pixels within the selected region of the touchscreen display the different levels of intensity producing a gradient across at least part of the selected region; and presenting the second portion of the frame of visual content with the second level of brightness includes presenting the second portion of the frame with varying reductions in brightness, with respect to the first level of brightness, based on the data that specifies the different levels of intensity for the pixels within the selected region of the touchscreen display producing a gradient of brightness deviation from the first level of brightness within the selected region of the touchscreen display.
[0020] Embodiment 13 is the computer-implemented method of any one of the embodiments 1 through 12, wherein: the different levels of display performance include different color depths for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance for the different portions of the frame of visual content includes: (i) presenting the first portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a first number of bits; and (ii) presenting the second portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a second number of bits that is lower than the first number of bits.
[0021] Embodiment 14 is the computer-implemented method of any one of the embodiments 1 through 13, comprising: determining, by the computing device, that the user input has ended its interaction with the touchscreen display; and presenting, by the computing device after having determined that the user input ended its interaction with the touchscreen display, another frame of visual content on the touchscreen display, using a same level of display performance for the first portion of the another frame and the second portion of the another frame, as a result of having determined that the user input ended its interaction with the touchscreen display.
[0022] Embodiment 15 is a computing system, comprising: a touchscreen display configured to present visual content; one or more processors; and one or more computer- readable devices including instructions that, when executed by the one or more processors, cause the computing system to perform operations that include: receiving user input that contacts the touchscreen display, including first user contact at a first location of the touchscreen display; identifying a selected region of the touchscreen display based on having received the user input, the selected region including the first location at which the user input contacted the touchscreen display; generating data that represents a frame of visual content for presentation by the touchscreen display; and presenting the frame of visual content on the touchscreen display, using different levels of display performance to depict different portions of the visual content, including by: (i) presenting a first portion of the frame of visual content with a first level of display performance, the first portion of the frame corresponding to and displayed by an area outside the selected region of the touchscreen display; and (i) presenting a second portion of the frame of visual content with a second level of display performance that is lower than the first level of display performance, the second portion of the frame corresponding to and displayed by an area within the selected region of the touchscreen display.
[0023] Embodiment 16 is the computing system of embodiment 15, wherein the computing system is configured, such that the computing system would have consumed more power presenting the second portion of the frame of visual content with the first level of display performance, than the computing system consumed presenting the second portion of the visual content with the second level of display performance.
[0024] Embodiment 17 is the computing system of any one of the embodiments 15 or 16, the operations including: detecting, by the computing system, the user input over a period of time during which the computing system presents multiple frames on the touchscreen display, the user input including: (i) the first user contact at the first location of the touchscreen display; and (ii) a second user contact at a second location of the touchscreen display that is different from the first location, the second user contact initiating contact with the touchscreen display after the first user contact released from being in contact with the touchscreen display, such that during a time between the first user contact ending and the second user contact beginning, there was no user contact with the touchscreen display, wherein the selected region contiguously includes an area of the first user contact at the first location and an area of the second user contact at the second location.
[0025] Embodiment 18 is the computing system of any one of the embodiments 15 through 17, the operations comprising: presenting, by the computing system, multiple additional frames of visual content using the different levels of display performance, after having presented the frame of visual content using the different levels of display performance.
[0026] Embodiment 19 is the computing system of any one of the embodiments 15 through 18, wherein: the computing system presents the multiple additional frames of visual content over a period of time during which additional user input both contacts the selected region of the touchscreen display and does not contact the touchscreen display, such that the computing system presents frames of visual content using the different levels of display performance through a period of time during which no user input is contacting the touchscreen display, following an end of a first user contact and a beginning of a second user contact.
[0027] Embodiment 20 is the computing system of any one of the embodiments 15 through 19, wherein the selected region of the touchscreen display includes and is larger than an area of the first user contact at the first location of the touchscreen display.
[0028] Embodiment 21 is the computing system of any one of the embodiments 15 through 20, wherein: the operations comprise determining, by the computing system, an indication of an orientation of a user finger that provided the first user contact at the first location of the touchscreen display, based on analysis of a shape of an area of the first user contact at the first location; and identifying the selected region of the touchscreen display includes identifying the selected region based on the determined orientation of the user finger; and the selected region includes: (1) the area of the first user contact at the first location; and (2) a second area outside the first user contact at the first location, and that is between the area of the first user contact at the first location and an edge of the touchscreen display along the determined orientation of the user finger, based on the determined orientation of the user finger, such that the second area represents area under the user finger and not contacted by the user finger during the first user contact.
[0029] Embodiment 22 is the computing system of any one of the embodiments 15 through 21, wherein identifying the selected region of the touchscreen display includes: sending, to a computational model, an indication of the user input that contacted the touchscreen display; and receiving, from the computational model, an indication of the selected region, or information that the computing system uses to identify the selected region. [0030] Embodiment 23 is the computing system of any one of the embodiments 15 through 22, wherein the computing system identifies the selected region of the touchscreen display based on: (i) locations at which the user input contacted the touchscreen display; and (ii) data received from orientation and/or movement sensors of the computing system during receipt of the user input, such that the computing system is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of different data being received from the orientation and/or movement sensors during receipt of the user input. [0031] Embodiment 24 is the computing system of any one of the embodiments 15 through 23, wherein the computing system identifies the selected region of the touchscreen display based on: (i) locations at which the user input contacted the touchscreen display; and (ii) a type of user interface being presented by the touchscreen display during receipt of the user input, such that the computing system is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of a different type of user interface being presented during receipt of the user input.
[0032] Embodiment 25 is the computing system of any one of the embodiments 15 through 24, wherein: the different levels of display performance include using different brightness levels for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance to depict the different portions of the visual content includes: (i) presenting the first portion of the frame of visual content with a first level of brightness; and (ii) presenting the second portion of the frame of visual content with a second level of brightness that is less than the first level of brightness.
[0033] Embodiment 26 is the computing system of any one of the embodiments 15 through 25, wherein: the operations comprise identifying data that specifies different levels of intensity for pixels within the selected region of the touchscreen display the different levels of intensity producing a gradient across at least part of the selected region; and presenting the second portion of the frame of visual content with the second level of brightness includes presenting the second portion of the frame with varying reductions in brightness, with respect to the first level of brightness, based on the data that specifies the different levels of intensity for the pixels within the selected region of the touchscreen display producing a gradient of brightness deviation from the first level of brightness within the selected region of the touchscreen display.
[0034] Embodiment 27 is the computing system of any one of the embodiments 15 through 26, wherein: the different levels of display performance include different color depths for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance for the different portions of the frame of visual content includes: (i) presenting the first portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a first number of bits; and (ii) presenting the second portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a second number of bits that is lower than the first number of bits.
[0035] Embodiment 28 is the computing system of any one of the embodiments 15 through 27, the operations comprising: determining, by the computing system, that the user input has ended its interaction with the touchscreen display; and presenting, by the computing system after having determined that the user input ended its interaction with the touchscreen display, another frame of visual content on the touchscreen display, using a same level of display performance for the first portion of the another frame and the second portion of the another frame, as a result of having determined that the user input ended its interaction with the touchscreen display.
[0036] Implementations of the above techniques include methods, apparatus, systems, and computer program products. One such computer program product is suitably embodied in a non-transitory machine-readable medium that stores instructions executable by one or more processors. The instructions are configured to cause the one or more processors to perform the above-described actions.
[0037] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.
DESCRIPTION OF DRAWINGS
[0038] FIG. 1 shows an example electronic device with a display.
[0039] FIG. 2 shows a diagram of an example display system of the electronic device.
[0040] FIG. 3 shows a diagram of a pixel circuit of a display device.
[0041] FIG. 4 shows a timing diagram of the pixel circuit of the display device.
[0042] FIG. 5 shows a block diagram of a system for varying display performance.
[0043] FIGS. 6A-C show examples of identifying a selected region of a touchscreen display based on user input.
[0044] FIG. 7 shows a flowchart of a process for operating a power saving device with variable touchscreen display performance.
[0045] FIG. 8 shows a block diagram of computing devices that may be used to implement the systems and methods described in this document, as either a client or as a server or plurality of servers.
[0046] Like reference symbols in the various drawings indicate like elements. DETAILED DESCRIPTION
[0047] This document generally describes mechanisms for providing a power saving device with variable touchscreen display performance. Display performance can be reduced in areas of the display that are concealed or likely concealed by an object such as a user’s hand. Display performance can be reduced in a portion of the display, for example, by adjusting a spatial resolution of the portion of the display, by adjusting a refresh rate of the portion of the display, by adjusting a color depth of the portion of the display, by adjusting a brightness of the portion of the display, or any combination of these. Reducing display performance in a portion of the display can reduce the amount of power consumed by the display and improve battery life of the power saving device.
[0048] The following discussion of the figures provides additional detail regarding such mechanisms to vary performance of a touchscreen display. The discussion of FIGS. 1-4 provides an overview of operation of a display device and components therein, with FIGS. 5 to 8 describing how such components can be operated to reduce power consumption by varying display performance.
[0049] FIG. 1 shows an example electronic device 190 with a display 104. As a few examples, the electronic device 190 can be a smart phone, a tablet computer, a laptop computer, a television, a smart watch, or a handheld game console. In this example, the electronic device 190 is a mobile telephone and the display 104 is a display panel of the mobile telephone.
[0050] The display 104 is configured to sense contact by a user. For example, FIG. 1 illustrates an individual contacting the display 104 with finger 101. The display 104 can be configured to receive a touch input and generate a touch input signal based on the location of the touch. The display 104 can also generate the touch input signal based on other characteristics of the touch, e.g., a number of touches, a path of a touch swipe, a size or shape of the touch input, etc. The touch input signal can indicate various properties of the touch, such as the location on the display 104, the area covered by the touch, the direction and speed of movement of the touch (e.g., the path or trajectory), and so on.
[0051] The display 104 presents a user interface 164. The user interface 164 includes user interface elements such as play button 166. In addition to buttons, user interface elements can include scroll bars, text input fields, drop-down menus, selectable icons, and links. The user can interact with the user interface elements of the user interface 164 in order to provide input to the electronic device. An electronic control module of the electronic device 190 can be configured to receive the touch input signal and to perform an action based on the received touch input signal. For example, based on the received touch input signal, the electronic control module may perform an action such as illuminating the display, opening an application, or scrolling the content displayed on the electronic device 190.
[0052] The display 104 includes an array of light-emitting pixels. In operation, the display 104 can display an image by illuminating the light-emitting pixels. The display 104 may be, for example, a light-emitting diode (LED) display, a mini-LED display, a microLED display, an organic light-emitting diode (OLED) display, a passive-matrix OLED display, an active-matrix OLED display, etc. In some examples, the display is a cathode ray tube (CRT) or liquid crystal display (LCD).
[0053] While the example electronic device 190 shown in FIG. 1 includes a single display 104, in general, the electronic device 190 may have multiple touch-sensitive surfaces. For example, a top half of the computing device may have a first touch-sensitive surface and the bottom half of the computing device may have a second touch-sensitive surface. As another example, the rear side of the computing device may have a first touch-sensitive surface and the side surfaces of the computing device may have a second touch-sensitive surface. While the electronic device 190 includes a flat display, in some implementations the display 104 may be curved.
[0054] FIG. 2 is a diagram of an example display system 100 of computing device 190. The device 190 includes a display 104 housed in a chassis 109. A region of the device 190 between the edge of the display 104 and the edge of the chassis is a bezel region 103.
[0055] The display 104 is an OLED display that includes an array 112 of light emitting pixels. Each light emitting pixel includes an OLED. The OLED display is driven by drivers, including SCAN/EM drivers 108 and data drivers 110. The SCAN/EM drivers 108 can be integrated, i.e., stacked, row line drivers. In general, the data drivers 110 provide data signals (e.g., voltage data (VDATA)) to the data lines (e.g., D1-D3), the SCAN/EM Drivers 108 provides a SCAN signal to a selected one of the scan lines (e.g., SCAN1) move the data signals from the data lines to the pixels in the selected scan line, and the SCAN/EM Drivers 108 provide an EMISSION signal to a selected one of the emission lines (e.g., El) to light the OLEDs in the selected row according to image data specified by the data signals. Although FIG. 1 illustrates the display system 100 having the SCAN/EM drivers 108 on a single side of the display, the SCAN/EM drivers 108 can be placed on both left and right sides of the display to improve driving performance (e.g., increasing speed by having SCAN drivers on the left side of the display and the EM drivers on the right side of the display). [0056] The pixel array 112 includes a plurality of light emitting pixels, for example, the pixels Pl 1 through P34. A pixel is a small element of a display that can change color based on the image data supplied to the pixel. Each pixel includes an OLED and circuitry to address the OLED with a data value, store the data value, and drive the OLED at an intensity based on the data value (e g., the components shown in FIG. 3. Each pixel within the pixel array 112 can be addressed individually to produce various intensities of a color produced by the pixel. Each pixel maintains a mostly steady luminance throughout a frame time, displaying light corresponding to the supplied image data. Luminance is the amount of light emitted by the surface area of a light source such as a pixel or a display. Display luminance is the luminous intensity coming from the surface of the display. Luminance can be measured in units such as candelas per square meter (cd/m2), which are also referred to as “nits.”
[0057] A frame time, or frame period, is an amount of time between a start of a frame and a start of a next frame. The frame time can be the inverse of a frame rate of a display system. For example, a frame rate of 60 frames per second (fps) corresponds to a frame time of one- sixtieth of a second, or 0.0167 seconds.
[0058] The pixel array 112 extends in a plane and includes rows and columns. Each row extends horizontally across the pixel array 112. For example, the first row 120 of the pixel array 112 includes pixels Pl 1, P21, and P31. Each column extends vertically down the pixel array 112. For example, the first column 130 of the pixel array 112 includes pixels Pl 1, P12, P13, and P14. Only a few pixels are shown in FIG. 1 for simplicity. In practice, there may be thousands or millions of pixels in the pixel array 112. Increasing the numbers of pixels in a display that remains the same size results in a higher image resolution.
[0059] The display system 100 includes a display driver integration circuit (DDIC) 106 that receives display input data 102. The display input data 102 can include color values for each pixel of the pixel array 112. The color value for a pixel corresponds with a color to be emitted by the pixel. In some examples, the display input data 102 can include brightness values for each pixel of the pixel array 112. The brightness value for a pixel corresponds with a brightness of the light to be emitted by the pixel.
[0060] In some examples, the display input data 102 includes a pixel value that incorporates both color data and brightness data. The RGB values of typical digital images do not directly correspond to the physical light intensities, but are rather compressed by a gamma correction function. This transformation better utilizes the limited number of bits in the encoded image by choosing a gamma value that matches the non-linear human perception of luminance. For example, the display input data 102 can include a gamma corrected pixel value for each subpixel of each pixel of the array 112. Addressing a pixel using the gamma corrected pixel value causes the pixel to emit light at the color and brightness specified by the gamma corrected pixel value.
[0061] In some examples, the DDIC 106 receives the display input data 102 from a system-on-chip (SoC) 105. The SoC 105 is a microchip with all the necessary electronic circuits and parts for a given system, such as a smartphone or wearable computer, on a single integrated circuit (IC). The SoC 105 is an integrated circuit that includes multiple components on a single chip. The SoC 105 can include, for example, a processor, a memory 306, and input/output (I/O) ports. The SoC 105 can be implemented on a single substrate, such as silicon. The SoC 105 can process digital signals, analog signals, and mixed signals. [0062] The DDIC 106 can be, for example, a semiconductor integrated circuit or a state machine. The DDIC 106 generates signals with suitable voltage, current, timing, and demultiplexing to cause a display 104 to show images according to display input data 102. In some examples, the DDIC 106 can be a microcontroller and may incorporate RAM, Flash memory, EEPROM, ROM, etc.
[0063] The DDIC 106 drives the pixel array 112 to emit light according to the display input data 102. For example, the data signal generator 138 of the DDIC 106 generates image data signals 144 from the display input data 102 and provides the image data signals 144 to the data drivers 110. The image data signals 144 can include voltages for each subpixel of the pixel array 112 to drive the subpixels to emit light at a color and brightness specified by the display input data 102.
[0064] The DDIC 106 includes a timing controller 134, a clock signal generator 136, and a data signal generator 138. The DDIC 106 generates control signals 142. The control signals 142 can include, for example, signals that control a display frame start time and a display frame stop time of each frame presented by the display 104, where a frame represents a single image in a sequence of images that are presented by the display 104. In examples in which each frame presented by the display panel includes multiple emission cycles, the control signals 142 or other signals not illustrated in FIG. 1 can control a display emission start time and a display emission stop time of each emission cycle of the display 104.
[0065] In some examples, the SCAN/EM drivers 108, the data drivers 110, or both, can be integrated with the DDIC 106. The SCAN/EM drivers supply SCAN and EM signals to rows of the pixel array 112. For example, the SCAN/EM drivers 108 supply scan signals via scan lines SI to S4, and EM signals via EM lines El to E4, to the rows of pixels, with each row of pixels in the pixel array 112 being addressed by a scan line and a corresponding emission line. For example, the first row 120 of the pixel array 112 is addressed by scan line SCAN1 and emission line El.
[0066] The data drivers 110 supply signals to columns of the pixel array 112. For example, based on the image data signal 144 from the data signal generator 138, the data drivers 110 output data values via source amp output signal lines SAN (e.g., a set of source amp signal lines SAI, SA2, and SA3) to a set of multiplexers 114 in the display 104. The set of multiplexers 114 in the display 104 receive data values from a corresponding set of source amp output signal lines SAN, and route the received data values among a greater number of data lines. For example, FIG. 1 illustrates a single MUX 114 that is configured to receive a stream of data values from the data driver 110 via the source output signal line SAI, and distribute the stream of data values one at a time among the data signal lines DI -3. In practice there would likely be multiple MUXs, each being fed with data values from the data drivers 110 via a corresponding source control signal line. Operations of the multiplexers 114 are described in greater detail with reference to FIG. 3A.
[0067] The data drivers 110 supply data voltages via the data lines D 1 to D3. In some examples, each of the data lines DI to D3 represent multiple data lines. For example, the pixel P 11 can include three subpixels (e. g. , P 11 R for a red subpixel, P 11 G for a green subpixel, and Pl IB for a blue subpixel), and the data line DI can represent three corresponding data lines, each addressing a corresponding subpixel of pixel Pl 1.
[0068] The control signals 142 can be used to drive the SCAN/EM drivers 108 and the data drivers 110. Thus, the DDIC 106 controls the timing of the scan signals, EM signals, and data signals.
[0069] The display system 100 includes a power supply 150. The power supply 150 provides a first supply voltage ELVDD and a second supply voltage ELVSS, both of which are provided to each pixel in the pixel array 112. In some examples, the power supply 150 can be integrated with the DDIC 106.
[0070] Each pixel in the pixel array 112 is addressable by a horizontal scan line, a horizontal EM line, and a vertical data line. For example, the pixel Pl 1 is addressable by the data line DI, the scan line SI, and the EM line El. In another example, the pixel P23 is addressable by the data line D2, the scan line S3, and the EM line E3.
[0071] The scan lines are addressed sequentially for each frame. A scan direction determines an order in which the scan lines are addressed (e.g., a direction in which rows of pixels receive data values and then light up at intensities based on the received data values). In the display system 100, the scan direction is from a top of the pixel array 112 to a bottom of the pixel array 112. For example, the scan line SI is addressed first, followed by the scan line S2, then S3, etc. In some implementations, all rows of pixels are programmed with data values using SCAN signals (one row at a time), before the display device activates all rows of pixels at intensities based on the programmed data values. In some implementations, a display device may activate rows of pixels while other rows of pixels are still being programmed, such that there is a gap of a few rows between a row currently receiving a SCAN signal and a row of pixels that is activated and begins emitting light.
[0072] While FIG. 2 illustrates that each row is addressed by a single scan line, each row may be addressed by multiple scan lines (e g., nSCAN and pSCAN). Although FIG. 1 illustrates example components of an OLED display, the described techniques may be applied to other flat panel display technologies that include an array of pixels. The techniques can be applied to curved displays, flexible displays, foldable displays, and rollable displays. For example, the technology may be applied to LED displays, LCD displays, plasma display panels (PDP), and CRT displays.
[0073] The technology can also be applied to projectors (e.g., digital light processing projectors) to reduce power consumption and to reduce the amount of heat absorbed and dissipated by the projector.
[0074] FIG. 3 shows a diagram of a pixel circuit of a display device, which pixel circuit includes an LED and corresponding drive circuitry for the pixel circuit. FIG. 3 may illustrate a more detailed view of a single pixel from the array of pixels shown in FIG. 1. While this disclosure sometimes refers to the components shown in FIG. 3 as a “pixel circuit”, this disclosure may also refer to such components as simply a “pixel.” Further, the pixel shown in FIG. 3 can represent a sub-pixel.
[0075] The pixel circuit may be an active matrix OLED (AMOLED) pixel circuit. The pixel circuit receives an emission signal (EM) on an emission line, SCAN signals on scan signal lines, and a data voltage (VDATA) signal on a data line. The pixel circuit 200 receives a first supply voltage ELVDD on a first voltage supply line, a second supply voltage ELVSS on a second voltage supply line, and an initial reference voltage VINIT on an initial voltage supply line.
[0076] The pixel circuit includes an organic light-emitting diode (OLED). The OLED includes a layer of an organic compound that emits light in response to an electric current, IOLED. The organic layer is positioned between two electrodes: an anode and a cathode. The OLED is driven by a driving transistor Tl, which receives the supply voltage ELVDD and acts as a current source that drives the OLED to emit light. [0077] The pixel also includes a storage capacitor CST and transistors T2 through T7. The operation of the pixel is defined by states of the control signals SCAN, EM, and VDATA. An amount/level of the OLED current (IOLED) is set by a voltage present at a gate terminal of the driving transistor Tl, referred to herein as the “G” node.
[0078] The driving transistor Tl has a threshold voltage VTH between the gate terminal of the driving transistor Tl and a source terminal of the driving transistor Tl. If the voltage between the gate terminal and the source terminal is above the threshold voltage VTH, the driving transistor Tl creates a conducting path from the source terminal to the drain terminal. An amount of current IOLED that flows through the conducting path through the driving transistor Tl corresponds to an amount that the voltage between the gate terminal and the source terminal is above the threshold voltage VTH.
[0079] FIG. 4 shows a timing diagram of the control signals provided to and received by the pixel shown in FIG. 3. These control signals repeatedly transition during operation of the display system 100 between an initialization stage, a programming stage, and an emission stage.
[0080] At an end of an emission stage, the EM signal transitions to an off state (e.g., by changing from a low state to a high state). This transition turns off transistors T5 and T6, which interrupts current being provided from ELVDD to the OLED, therefore stopping light emission by the OLED. Since the EM signal may be provided to an entire line of pixels, this transition can turn off all pixels in the line of pixels.
[0081] During the initialization stage, the SCAN[n-l] signal turns to an on state (e.g., by changing from a high state to a low state), which turns on transistor T4 for a period of time and initializes the G node to the initialization voltage VINIT. Since the SCAN[n-l] signal may be provided to an entire line of pixels, this initialization stage can erase the data values that were previously stored at each pixel in the line of pixels. The SCAN[n-l] signal may be the SCAN[n] signal provided to a preceding row by a state machine of the SCAN/EM drivers 108.
[0082] During the programming stage, the SCAN[n] signal turns to an on state (e.g., by going low), which turns on transistors T2, T3, and T7 for a period of time. This causes the voltage value at the voltage data VDATA line to pass through transistors T2, Tl, and T3 to the G node, setting the G node to a value based on the VDATA line (e.g., the voltage at VDATA minus an effect of transistor threshold voltages). Since the SCAN signal may be provided to an entire line of pixels, this programming stage can cause each pixel in the line of pixels to move data voltage values from each pixel’s respective data line to the G node of the respective pixel.
[0083] During the emission stage, the EM signal turns to an on state (e.g., by going low), which turns on transistors T5 and T6. Current flows from ELVDD through transistors T5, Tl, and T6 to an anode of the OLED. Since the EM signal is provided to an entire line of pixels, all pixels in the line of pixels may activate.
[0084] A current level provided to the OLED in each pixel is determined by the voltage present at the Gnode of the pixel (e.g., with the G node voltage level having been programmed by the voltage data VDATA line). An intensity or brightness of light emitted by the OLED directly correlates to an amount of electrical current IOLED applied to the OLED, with higher current corresponding to a greater intensity of light than a lower current. The storage capacitor CST maintains the voltage at the G node, so that the OLED continues to emit light at roughly the same level for a duration of the emission stage.
[0085] The voltage at the G node may decrease slightly during the emission stage. As such, the current IOLED applied to the OLED and the intensity of light emitted by the OLED may decrease or increase slightly during the emission stage, depending on a type of pixel circuit design (e.g., with p-channel transistors in the pixel circuit, lower voltage levels at the G node cause higher IOLED and higher intensity of OLED light).
[0086] FIG. 5 shows a block diagram of a system 500 for operating a power saving device with variable touchscreen display performance. The system 500 includes the SoC 105 and the DDIC 106 of the display system 100. The SoC 105 includes a memory 306, a processor 304, and a NPU 311.
[0087] The processor 304 can be, for example, a graphical processing unit (GPU), a data processing unit (DPU), or a central processing unit (CPU). The processor 304 can include, for example, a bus interface, a power management unit, a video processing unit, a graphics memory controller, a display interface, or any combination of these. The processor 304 can include a digital signal processor (DSP). The DSP can perform signal processing operations such as data collection and data processing.
[0088] The neural processing unit (NPU) 311 can be, for example, a tensor processing unit (TPU). The NPU 311 is a hardware accelerator for executing neural network tasks. The NPU 311 performs neural network tasks such as matrix multiplications or other tensor operations.
[0089] When generating and displaying images on the display 104 of the device 190, the processor 304 can generate visual content data, such as a frame of a video. The visual content data may be for a video sequence that is pre-rendered, e.g., for a fdm. The visual content data may be for a video sequence that is dynamically generated, e.g., for a video game or for user navigation through various operating system screens and menus. In some examples, the visual content data can be compressed, using any appropriate method. In some examples, the visual content data can be uncompressed. The processor 304 can store the generated visual content data in the memory 306. The memory 306 may be any appropriate type of memory. For instance, the memory 306 can be a random access memory (RAM).
[0090] The SoC 105 can identify image data 310 (e.g., by receiving the image data 310 from another source or by generating the image data 310). The image data 310 can be, for example, image data for an image frame. The SoC 105 can receive, as input, ambient brightness data 330, accelerometer data 332, application data 336, or any combination of these.
[0091] The ambient brightness data 330 can indicate brightness of the environment in which the device 190 is located. In some examples, the device 190 includes a light sensor configured to detect ambient brightness. The SoC 105 can receive an indication of an amount of light sensed by a light sensor of the device 190, and can determine the ambient brightness using the indication of the amount of light sensed by the light sensor. In some examples, the SoC 105 modifies a display brightness setting based on the amount of light sensed by the light sensor, for example, to increase an overall brightness of the display when in high ambient light environments. The ambient brightness data 330 can affect the amount by which the image data 310 is modified by the SoC 105. For example, at higher ambient brightness levels, the SoC 105 can select a larger area of the display within which to present visual content using a reduced level of display performance. In some examples, at higher ambient brightness levels, the SoC 105 can modify the image data 310 by a greater amount in order to achieve a greater amount of power savings. In some examples, the SoC 105 only presents visual content using a reduced level of display performance when ambient brightness and/or a display brightness level exceeds a threshold (e.g., such that image data is modified only when the display is operating at a high brightness level when image modification has a greatest effect).
[0092] The accelerometer data 332 can include data indicating device orientation, device movement, or both. The accelerometer data 332 can be generated by an accelerometer of the device 190 or any movement sensor or orientation sensor of the device 190. An accelerometer is an electromechanical device that measures the force of acceleration caused by movement, by acceleration (such as acceleration due to gravity), and by vibration. In some examples, the accelerometer is a dynamic accelerometer that detects a change in orientation or tilt of the display. In response to detecting the change in orientation of the display, the display can change between portrait mode and landscape mode. The accelerometer can also detect movement such as movement of a user walking while carrying the device, or movement of a vehicle in which the device is located. The accelerometer data 332 can affect the portions of the display that are visible to the user. For example, different areas of the display may be visible to the user when the user holds the device 190 in a landscape orientation compared to a portrait orientation. Additionally, different areas of the display may be visible to the user when the device 190 is mounted to a dashboard of a moving vehicle compared to when the device 190 is placed on a stationary desk.
[0093] The application data 336 can include data indicating applications operated by the device 190. In some examples, the application data 336 includes data indicating a user interface presented on the display 104. For example, the application data 336 can indicate that the device 190 is operating an application that presents videos, and that the display 104 is presenting a display that shows a list of videos and a scrolling user interface element. In another example, the application data 336 can indicate that the device 190 is operating an application that presents videos, and that the display 104 is presenting a display that shows a single video and user interface elements for controlling the video. The application data 336 can affect the portions of the display that are likely to be touched by the user, and the orientation of the user’s finger when touching the display. Thus, the application data 336 can affect the portions of the display that are visible to the user.
[0094] The touch input signal 340 indicates that a touch input was received by the display 104. The touch input signal can indicate characteristics of the touch such as a number of touches, a path of a touch swipe, a size or shape of the touch input, the location of the touch on the display 104, a duration of the touch, the area covered by the touch, the direction and speed of movement of the touch, or any combination of these.
[0095] The SoC 105 can store computational interaction models representing areas of the display that are concealed or likely concealed from the user during user interaction with the display 104. The interaction models can include heuristic models, machine learning models, or both.
[0096] In some examples, the memory 306 stores heuristic (i.e., rule-based) interaction models. The processor 304 can access the heuristic models in the memory 306 to determine whether or not to adjust display performance and to determine how to adjust display performance. In some examples, the processor 304 can select a particular heuristic model from the memory 306 based on the ambient brightness data 330, the accelerometer data 332, the application data 336, the touch input signal 340, or any combination thereof. The processor 304 can use the particular heuristic model to identify areas of the display 104 in which display performance is to be reduced, to determine an extent of display performance reduction for the identifies areas of the display, to determine a mechanism for reducing the display performance, to determine a duration for reducing the display performance, or any of these.
[0097] In an example, the device 190 operates a video-playing application in a bright environment in landscape mode. A user interacts with a sliding user interface element that enables the user to seek forward and backward through a video. The sliding user interface element is located under the video on the display 104. The processor 304 selects a first heuristic model from the memory 306 based on the ambient brightness data 330 that indicates the bright environment, the accelerometer data 332 that indicates the landscape mode, the application data 336 that indicates the user interface displayed by the video-playing application, and/or the touch input signal 340 that indicates the user interaction with the sliding user interface element. The first heuristic model can specify that display performance of an area of the display 104 that includes the sliding user interface element is to be reduced. The first heuristic model can specify a mechanism for reducing the display performance (e.g., by reducing pixel brightness). The first heuristic model can specify an amount by which to reduce the display performance (e.g., a percentage brightness reduction). The first heuristic model can specify a duration for reducing the display performance (e.g., until at least one second has passed after detecting a touch input).
[0098] In some examples, the NPU 311 stores machine learning interaction models. The processor 304 can access the machine learning models in the NPU 311 to determine whether or not to adjust display performance and to determine how to adjust display performance. In some examples, the processor 304 provides input to the machine learning interaction models such as the ambient brightness data 330, the accelerometer data 332, the application data 336, the touch input signal 340, or any combination thereof. The processor 304 can obtain output from the machine learning models indicating areas of the display 104 in which display performance is to be reduced, to determine an extent of display performance reduction for the identifies areas of the display, to determine a mechanism for reducing the display performance, to determine a duration for reducing the display performance, or any of these. [0099] In some examples, the processor 304 generates modified image data 302. For example, the processor can modify the image data 310 based on heuristic and/or machine learning models to produce the modified image data 302. The modified image data 302, when presented by the display 104, presents the same content as the image data 310 but with a reduced display performance for at least part of the content. For example, the image data 310 can include any combination of user interface elements, icons, text, photographs, graphs, animations, and videos. The modified image data 302 presents the same user interface elements, icons, text, photographs, graphs, animations, and videos as the image data 310. As described below, the modified image data 302 can be modified in various ways to present the visual content with reduced display performance in order to reduce power consumption.
[00100] In some examples, the processor 304 can modify the image data 310 by applying a brightness dimming mask to the image data 310. After applying a brightness dimming mask to the image data 310, a portion of a frame of the modified image data 302 is dimmer than the same portion of the same frame of the image data 310, for example, while another portion of the frame of modified image data 302 is the same as a corresponding portion of the frame of the image data 310.
[00101] In some examples, the processor 304 can modify the image data 310 by adjusting bit ranges of the image data 310. After adjusting the bit ranges of the image data 310, at least a portion of a frame of the modified image data 302 has a reduced bit range compared to the same portion of the same frame of the image data 310. Therefore, for a frame of modified image data 302, some pixels of the frame have a greater bit range (e.g., ten bits) compared to the bit range of other pixels of the frame (e.g., six bits).
[00102] In some examples, the processor 304 can modify the image data 310 by adjusting a spatial resolution of the image data 310. After adjusting the spatial resolution of the image data 310, at least a portion of a frame of the modified image data 302 has a reduced spatial resolution compared to the same portion of the same frame of the image data 310. Therefore, for a frame of modified image data 302, some areas of the frame include clusters of pixels addressed with the same pixel values, and other areas of the frame include individual pixels that are each addressed with a respective pixel value.
[00103] In some examples, instead of or in addition to generating the modified image data 302, the SoC 105 can send instructions to the DDIC 106 that adjust the display performance when the display 104 presents the image data 310. For example, the SoC 105 can instruct the DDIC to provide adjusted data values to a subset of pixels that are within the selected region of the display 104 in order to reduce the current provided to the subset of the pixels.
[00104] In some examples, the SoC 105 can instruct the DDIC to reduce a refresh rate of a subset of pixels that are within the selected region of the display 104. For example, the SoC 105 can instruct the DDIC to reduce a refresh rate of the subset of pixels from 240 Hz to 120 Hz. As a result, the subset of pixels that are within the selected region will refresh at a reduced frequency of 120 Hz, while pixels outside of the selected region will refresh at a nonreduced frequency of 240 Hz.
[00105] The SoC 105 provides display input data 102 to the DDIC 106. The display input data 102 can include the modified image data 302 generated by the processor 304. Although FIG. 3 shows the SoC 105 generating the modified image data 302 from the image data 310, other implementations are possible. For example, in some implementations, the SoC 105 provides an interaction model and the image data 310 to the DDIC 106, and the DDIC 106 modifies the image data 310 using the interaction model.
[00106] FIGS. 6A-C show examples of using interaction models to identify a selected region of a touchscreen display based on user input. In FIGS. 6A-C, the display 104 is divided into multiple zones represented by a grid of multiple grid segments. Each grid segment represents a region of the display 104 that presents a corresponding portion of frames of visual content. In some examples, each grid segment is a square or rectangular area of the display. Each grid segment can include a group of pixels of the display 104 that are within the same region or zone of the display 104.
[00107] FIG. 6A provides an example of using an interaction model to identify a selected region of a touchscreen display based on user input., where the selected region includes segments of the touchscreen display that have at least a threshold probability of being concealed from the user.
[00108] Referring to FIG. 6A, at step 610, the user’s finger 101 touches the display 104. Referring to FIG. 5, the SoC 105 receives, as input data, a touch input signal 340 indicating occurrence of the touch and a location of the touch. The touch input signal 340 can also indicate the shape and/or orientation of the touch (e.g., as determined by an algorithm executed by the SoC 105). The SoC 105 can receive additional input data such as the accelerometer data 332, the ambient brightness data 330, and the application data 336.
[00109] At step 612, the processor 304 accesses an interaction model from the memory 308 and/or the NPU 311. The processor 304 can send an indication of the user input that contacted the touchscreen display to a computational interaction model which can be a heuristic mode or a machine learning model. In some examples, the processor 304 receives, from the computational model, an indication of the selected region in which display performance is to be reduced. In some examples, the processor 304 receives, from the computational model, information that the processor 304 uses to identify the selected region in which display performance is to be reduced.
[00110] In some examples, the processor 304 selects a particular heuristic interaction model based on the input data, and the particular interaction model indicates a heat map of the display. Areas of higher usage and lower usage can be identified with the heat map, where areas of high usage are classified as “hot” and areas of lower usage are classified as “cool.” For example, each grid segment can be associated with a percentage of likelihood that the grid segment is under the user’s hand or finger, and classified as hot or cool based on the percentage of likelihood. In some examples, heat maps of high and low usage can be stored in the memory 306. The heat maps can be associated with different applications, user interfaces, orientations, movements, or any combination of these.
[00111] In some examples, the processor 304 provides the input data to a machine learning interaction model and receives a heat map of the display as output from the machine learning interaction model. In some examples, machine learning models can record user behavior and determine highly used areas of the touchscreen display. The user behavior can be dependent on types of applications in use and/or types of user interfaces being presented on the display 104.
[00112] As an illustration, the computing system may store a first heat map to use when the user device is presenting a user interface for a video-streaming application in landscape mode (e.g., showing “hot” areas at the left and right sides of the display at which thumbs overlap the display). The same computing system may store a second heat map to use when the user device is presenting a user interface for a web browser in portrait mode, and when user interaction has been recently received at a right edge of the display (e.g., showing a hot area along a right edge of the display where a vertical scroll bar of the web browser is presented).
[00113] In some examples, the device 190 determines an indication of an orientation of the finger 101 that touched the display 104. The device 190 can determine the indication of the orientation based on analysis of a shape of an area of the user contact. In some examples, the device 190 identifies the selected region based at least in part on the determined orientation of the user finger. For example, the orientation of the finger 101 in FIG. 6A is diagonal from a top left comer of the display 104 towards a bottom right comer of the display 104. The touch location of the finger 101 is nearer to the bottom of the display 104 than to the top of the display 104. Therefore, the processor 304 can determine a higher likelihood of a bottom right comer region of the display 104 being concealed from the user compared to other regions of the display 104.
[00114] Step 612 shows an example heat map, with darker patterned grid segments representing areas of the display that are more likely concealed from the user, and lighter patterned grid segments representing areas of the display that are less likely concealed from the user. For example, the grid segment 602 is the darkest grid segment because the grid segment 602 corresponds to the touch location of the finger 101. Grid segments around the grid segment 602 and between the grid segment 602 and the edge of the display 104 are shaded with patterns of various darkness. In some examples, the heat map specifies, for each grid segment, a probability that the grid segment is concealed from the user.
[00115] At step 614, the processor 304 selects a region of the display 104 based on the heat map. In some examples, the processor 304 selects grid segments for which the probability that the grid segment is concealed from the user satisfies a threshold probability. In the example of FIG. 6A, the processor 304 selects grid segments within boundary 604. The boundary 604 encompasses the grid segment 602 at which the finger 101 touched the display 104. The boundary 604 encompasses an area of the display 104 that is larger than the area of the touch by the finger 101. The area encompassed by the boundary 604 can be considered a “shadow area” of the display 104.
[00116] At step 616, the device 190 presents visual content using different levels of display performance for region 606, which is within the boundary 604, and for region 608, which is outside of the boundary 604. The device 190 presents the same visual content in the region 606 as would be presented without adjusting the display performance. In other words, the image presented by the display 104 is the same when using the different levels of display performance as when using the same levels of display performance. The display performance used to present the visual content in the region 606 is reduced compared to the display performance used to present the visual content in the region 608. For example, the visual content presented in the region 606 can have a reduced brightness, a reduced color depth, a reduced spatial resolution, a reduced refresh rate, or any of these, compared to the region 608. [00117] FIG. 6B provides another example of using an interaction model to identify a selected region of a touchscreen display based on user input., where the interaction model specifies a gradient indicating an amount by which display performance is to be reduced throughout the selected region.
[00118] At step 620 in FIG. 6B, the user’s finger 101 touches the display 104. Referring to FIG. 5, the SoC 105 receives, as input data, a touch input signal 340 indicating occurrence of the touch and a location of the touch. The touch input signal 340 can also indicate the shape and/or orientation of the touch. The SoC 105 can receive additional input data such as the accelerometer data 332, the ambient brightness data 330, and the application data 336. [00119] At step 622, the processor 304 accesses an interaction model from the memory 308 and/or the NPU 311. In some examples, the processor 304 selects a particular heuristic interaction model based on the input data, and the particular interaction model indicates a region 626 of the display in which visual content is to be presented with a reduced display performance.
[00120] In some examples, the processor 304 provides the input data to a machine learning interaction model and receives, as output from the machine learning interaction model, an identified region 626 of the display in which visual content is to be presented with a reduced display performance.
[00121] In some examples, the interaction model includes a heat map that indicates an amount by which display performance is to be reduced in each grid segment. For example, segment 624, represented at step 622 by black shading, is to have a greatest amount of display performance reduction. Other grid segments within the region 626 are to have reduced display performance by varying amounts represented by the amount of shading in each grid segment. The device 190 thus presents content in the region 626 with degraded display performance according to the heat map. The device 190 presents content in the region 628, outside of the region 626, without reducing the display performance.
[00122] FIG. 6C provides another example of using an interaction model to identify a selected region of a touchscreen display based on user input., where the interaction model specifies a uniform reduction in display performance throughout the selected region.
[00123] Refernng to FIG. 6C, at step 640, the user’s finger 101 touches the display 104. Referring to FIG. 5, the SoC 105 receives, as input data, a touch input signal 340 indicating occurrence of the touch and a location of the touch. The touch input signal 340 can also indicate the shape and/or orientation of the touch. The SoC 105 can receive additional input data such as the accelerometer data 332, the ambient brightness data 330, and the application data 336.
[00124] At step 632, the processor 304 accesses an interaction model from the memory 308 and/or the NPU 311. In some examples, the processor 304 selects a particular heuristic interaction model based on the input data, and the particular interaction model indicates a region 636 of the display in which visual content is to be presented with a reduced display performance. [00125] In some examples, the processor 304 provides the input data to a machine learning interaction model and receives, as output from the machine learning interaction model, an identified region 636 of the display in which visual content is to be presented with a reduced display performance.
[00126] In some examples, the level of display performance is adjusted uniformly throughout the region 636. For example, the level of display performance within the region 636 can be reduced by the same amount throughout the region 636. The device 190 thus presents visual content in the region 636 with uniformly degraded display performance. The device 190 presents visual content in the region 638, outside of the region 636, without reducing the display performance.
[00127] FIG. 7 shows a flowchart of a process 700 for operating a power saving device with variable touchscreen display performance. The process may be implemented by a display device, a computing device that includes the display device, or one or more components thereof or in communication with the computing device or display device.
[00128] A computing system receives display content. For example, the computing system can be the device 190, described with respect to FIGS. 1-5, which can receive image data 310 including display content for presentation on the display 104 of the device 190. The computing system may receive the image data 310 by generating the image data 310 and transferring the image data 310 from one component of the computing system to another component of the computing system.
[00129] At box 710, the computing system receives user input that contacts a touchscreen display. For example, the device 190 can receive a touch input signal 340 representing contact with the display 104 by an object such as the user’s finger 101. The user input includes first user contact at a first location of the touchscreen display 104.
[00130] In some examples, the computing system detects the user input over a period of time during which the computing system presents multiple frames on the touchscreen display. For example, the device 190 can detect a first user contact that extends for a duration of presenting multiple frames. In some examples, the user input includes (in addition to the first user contact) second user contact at a second location of the touchscreen display that is different from the first location. For example, the device 190 can detect a first touch at a first location of the display 104 and a second touch at a second location of the display 104. The first location and the second location may correspond to locations at which the display 104 presented different user elements. [00131] At box 720, the computing system identifies a selected region of the touchscreen display based on a location of the user input. For example, the device 190 can identify the selected region of the touchscreen display 104 based on having received the touch by the user’s finger 101. The selected region can include the first location at which the user input contacted the touchscreen display and can be larger than an area of the first user contact at the first location of the touchscreen display. For example, referring to FIGS. 6A-C, the selected regions 606, 626, and 636 are each larger than an area of the user contact with the finger 101. [00132] At box 730, the computing system generates data that represents a frame of visual content. For example, referring to FIG. 5, the SoC 105 of the device 190 can generate, or receive from another component of the computing system, image data 310 that represents a frame of visual content.
[00133] At box 740, the computing system identifies a first portion of the frame that corresponds to an area outside of the selected region. For example, referring to FIG. 6 A, region 608 is outside of the boundary 604 of the selected region 606. The computing device 190 identifies a first portion of the frame of visual content that corresponds to the region 608. [00134] At box 750, the computing system identifies a second portion of the frame that corresponds to an area within the selected region. For example, referring to FIG. 6A, region 606 is within the boundary 604. The computing device 190 identifies a second portion of the frame of visual content that corresponds to the region 606.
[00135] At box 760, the computing system presents the frame of visual content. The computing system presents the frame of visual content using different levels of display performance to depict different portions of the visual content. For example, the device 190 can present the second portion of the frame corresponding to the region 606 using a reduced level of display performance than the first portion of the frame corresponding to the region 608.
[00136] At box 770, the computing system presents the first portion of the frame with a first level of display performance. The first portion of the frame of visual content is displayed by the area outside of the selected region of the touchscreen display. For example, the first portion of the frame can be displayed by the region 608.
[00137] At box 780, the computing system presents the second portion of the frame with a second level of display performance. The second portion of the frame is displayed by the area within the selected region of the touchscreen display. For example, the second portion of the frame can be displayed by the region 606. The second level of display performance is lower than the first level of display performance. [00138] In some examples, the different levels of display performance include different brightness levels for the different portions of the frame of visual content. In some examples, the first portion of the frame of visual content is presented with a first level of brightness, and the second portion of the frame is presented with a second level of brightness that is less than the first level of brightness. For example, the second portion of the frame presented by the region 606 can have a reduced brightness compared to the first portion of the frame presented by the region 608.
[00139] In some examples, the different levels of display performance include different color depths for the different portions of the frame of visual content. In some examples, the first portion of the frame of visual content is presented using image data that encodes each sub-pixel using a color depth of a first number of bits (e.g., twelve bits) and the second portion of the frame of visual content is presented using image data that encodes each subpixel using a color depth of a second number of bits (e.g., eight bits). The second number of bits is lower than the first number of bits. Thus, the dynamic color range of the second portion is less than the first portion. For example, the second portion of the frame presented by the region 606 can have a reduced color depth compared to the first portion of the frame presented by the region 608.
[00140] In some examples, the different levels of display performance include different refresh rates for the different portions of the frame of visual content. For example, the second portion of the frame presented by the region 606 can have a reduced refresh rate compared to the first portion of the frame presented by the region 608. At a reduced refresh rate, the second portion of the frame updates at a reduced frequency with a lower time resolution compared to the first portion of the frame.
[00141] In some examples, the different levels of display performance include different spatial resolutions for the different portions of the frame of visual content. For example, the second portion of the frame presented by the region 606 can have a reduced spatial resolution compared to the first portion of the frame presented by the region 608.
[00142] In some examples, the computing system identifies data that specifies different levels of intensity for pixels within the selected region of the touchscreen. The different levels of intensity can produce a gradient across at least part of the selected region. For example, referring to FIG. 6B, the interaction model produces a gradient across the region 626. The computing system can present the second portion of the frame with varying reductions in brightness, with respect to the first level of brightness, based on the data that specifies the different levels of intensity for the pixels within the selected region of the touchscreen. The computing system can thus produce a gradient of brightness deviation from the first level of brightness within the selected region of the touchscreen. For example, the device 190 can present the second portion of the frame in the region 626 with a brightness gradient specified by the interaction model.
[00143] The computing system can be configured such that the computing system would have consumed more power presenting the second portion of the frame of visual content with the first level of display performance than the computing device consumed presenting the second portion of the visual content with the second level of display performance.
[00144] In some examples, after presenting the frame of visual content using the different levels of display performance, the computing system can present multiple additional frames of visual content using the different levels of display performance.
[00145] In some examples, the computing system presents the multiple additional frames of visual content over a penod of time during which additional user input both contacts the selected region of the touchscreen display and does not contact the touchscreen display. The computing system can present frames of visual content using the different levels of display performance through a period of time during which no user input is contacting the touchscreen display, following an end of a first user contact and a beginning of a second user contact. For example, the user may touch and remove their finger 101 multiple times while typing and/or swiping on the display 104. The device 190 can continue to present frames of visual content using the different levels of display performance in between touches by the user.
[00146] In some examples, the computing system determines that the user input has ended its interaction with the touchscreen display. For example, the device 190 can detect that a threshold duration of time has passed since a touch input was received by the display 104. In response to detecting that the threshold duration of time has passed since a touch input was received by the display 104, the device 190 can determine to present subsequent content using the same level of display performance across the entire display 104. The device 190 can then present another frame of visual content (e.g., a next or subsequent frame) on the touchscreen display using a same level of display performance for the first portion of frame and the second portion of the frame. For example, referring to FIG. 6A, after determining that the user is not interacting with the display 104, the device 190 can present visual content on the display 104 using a same level of display performance for the region 606 and the region 608. [00147] In some examples, the device 190 can determine a duration of time during which display performance is to be adjusted based on the interaction model. For example, the interaction model can indicate a predicted time duration of user interaction with the display given a particular set of input data. The device 190 can present visual content using the different levels of display performance for the region 606 and the region 608 for the predicted time duration. After the predicted time duration has expired, the device 190 can present visual content on the display 104 using a same level of display performance for the region 606 and the region 608.
[00148] FIG. 8 is a block diagram of computing devices 800, 850 that may be used to implement the systems and methods described in this document, as either a client or as a server or plurality of servers. Computing device 800 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device 850 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations described and/or claimed in this document.
[00149] Computing device 800 includes a processor 802, memory 804, a storage device 806, a high-speed controller 808 connecting to memory 804 and high-speed expansion ports 810, and a low speed controller 812 connecting to low speed expansion port 814 and storage device 806. Each of the components 802, 804, 806, 808, 810, and 812, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor 802 can process instructions for execution within the computing device 800, including instructions stored in the memory 804 or on the storage device 806 to display graphical information for a GUI on an external input/ output device, such as display 816 coupled to high-speed controller 808. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices 800 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).
[00150] The memory 804 stores information within the computing device 800. In one implementation, the memory 804 is a volatile memory unit or units. In another implementation, the memory 804 is a non-volatile memory' unit or units. The memory 804 may also be another form of computer-readable medium, such as a magnetic or optical disk. [00151] The storage device 806 is capable of providing mass storage for the computing device 800. In one implementation, the storage device 806 may be or contain a computer- readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. A computer program product can be tangibly embodied in an information carrier. The computer program product may also contain instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory 804, the storage device 806, or memory on processor 802. [00152] The high-speed controller 808 manages bandwidth-intensive operations for the computing device 800, while the low speed controller 812 manages lower bandwidthintensive operations. Such allocation of functions is an example only. In one implementation, the high-speed controller 808 is coupled to memory 804, display 816 (e.g., through a graphics processor or accelerator), and to high-speed expansion ports 810, which may accept various expansion cards (not shown). In the implementation, low-speed controller 812 is coupled to storage device 806 and low-speed expansion port 814. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) may be coupled to one or more input/ output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter. [00153] The computing device 800 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 820, or multiple times in a group of such servers. It may also be implemented as part of a rack server system 824. In addition, it may be implemented in a personal computer such as a laptop computer 822. Alternatively, components from computing device 800 may be combined with other components in a mobile device (not shown), such as device 850. Each of such devices may contain one or more of computing devices 800, 850, and an entire system may be made up of multiple computing devices 800, 850 communicating with each other.
[00154] Computing device 850 includes a processor 852, memory 864, an input/output device such as a display 854, a communication interface 866, and a transceiver 868, among other components. The device 850 may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of the components 850, 852, 864, 854, 866, and 868, are interconnected using vanous buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate. [00155] The processor 852 can execute instructions within the computing device 850, including instructions stored in the memory 864. The processor may be implemented as a chipset of chips that include separate and multiple analog and digital processors.
Additionally, the processor may be implemented using any of a number of architectures. For example, the processor may be a CISC (Complex Instruction Set Computers) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor. The processor may provide, for example, for coordination of the other components of the device 850, such as control of user interfaces, applications run by device 850, and wireless communication by device 850.
[00156] Processor 852 may communicate with a user through control interface 858 and display interface 856 coupled to a display 854. The display 854 may be, for example, a TFT (Thin-Film-Transistor Liquid Cry stal Display) display or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology such as LCD, LED, and CRT. The display interface 856 may comprise appropriate circuitry for driving the display 854 to present graphical and other information to a user. The control interface 858 may receive commands from a user and convert them for submission to the processor 852. In addition, an external interface 862 may be provided in communication with processor 852, so as to enable near area communication of device 850 with other devices. External interface 862 may be provided, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.
[00157] The memory 864 stores information within the computing device 850. The memory 864 can be implemented as one or more of a computer-readable medium or media, a volatile memory' unit or units, or a non-volatile memory unit or units. Expansion memory 874 may also be provided and connected to device 850 through expansion interface 872, which may include, for example, a SIMM (Single In Line Memory Module) card interface. Such expansion memory 874 may provide extra storage space for device 850, or may also store applications or other information for device 850. Specifically, expansion memory 874 may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, expansion memory 874 may be provided as a security module for device 850, and may be programmed with instructions that permit secure use of device 850. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.
[00158] The memory may include, for example, flash memory and/or NVRAM memory, as discussed below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory 864, expansion memory 874, or memory' on processor 852 that may be received, for example, over transceiver 868 or external interface 862.
[00159] Device 850 may communicate wirelessly through communication interface 866, which may include digital signal processing circuitry where necessary. Communication interface 866 may provide for communications under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, through radio-frequency transceiver 868. In addition, short-range communication may occur, such as using a Bluetooth, WiFi, or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module 870 may provide additional navigation- and location- related wireless data to device 850, which may be used as appropriate by applications running on device 850.
[00160] Device 850 may also communicate audibly using audio codec 860, which may receive spoken information from a user and convert it to usable digital information. Audio codec 860 may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of device 850. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on device 850.
[00161] The computing device 850 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a cellular telephone 880. It may also be implemented as part of a smartphone 882, personal digital assistant, tablet, or other similar mobile device.
[00162] Additionally computing device 800 or 850 can include Universal Serial Bus (USB) flash drives. The USB flash drives may store operating systems and other applications. The USB flash drives can include input/output components, such as a wireless transmitter or USB connector that may be inserted into a USB port of another computing device. [00163] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
[00164] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly /machine language. As used herein, the terms “machine-readable medium” “computer-readable medium” refers to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.
[00165] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
[00166] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), peer-to-peer networks (having ad-hoc or static members), grid computing infrastructures, and the Internet.
[00167] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
[00168] Although a few implementations have been described in detail above, other modifications are possible. Moreover, other mechanisms for performing the systems and methods described in this document may be used. In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:
1. A computer-implemented method, comprising: receiving, by a computing device with a touchscreen display, user input that contacts the touchscreen display, including first user contact at a first location of the touchscreen display; identifying, by the computing device, a selected region of the touchscreen display based on having received the user input, the selected region including the first location at which the user input contacted the touchscreen display; generating, by the computing device, data that represents a frame of visual content for presentation by the touchscreen display; and presenting, by the computing device on the touchscreen display, the frame of visual content, using different levels of display performance to depict different portions of the visual content, including by:
(i) presenting a first portion of the frame of visual content with a first level of display performance, the first portion of the frame corresponding to and displayed by an area outside the selected region of the touchscreen display; and
(i) presenting a second portion of the frame of visual content with a second level of display performance that is lower than the first level of display performance, the second portion of the frame corresponding to and displayed by an area within the selected region of the touchscreen display.
2. The computer-implemented method of claim 1, wherein the computing device is configured, such that the computing device would have consumed more power presenting the second portion of the frame of visual content with the first level of display performance, than the computing device consumed presenting the second portion of the visual content with the second level of display performance.
3. The computer-implemented method of claim 1, comprising: detecting, by the computing device, the user input over a period of time during which the computing device presents multiple frames on the touchscreen display, the user input including:
(i) the first user contact at the first location of the touchscreen display; and
(ii) a second user contact at a second location of the touchscreen display that is different from the first location, the second user contact initiating contact with the touchscreen display after the first user contact released from being in contact with the touchscreen display, such that during a time between the first user contact ending and the second user contact beginning, there was no user contact with the touchscreen display, wherein the selected region contiguously includes an area of the first user contact at the first location and an area of the second user contact at the second location.
4. The computer-implemented method of claim 3, comprising: presenting, by the computing device, multiple additional frames of visual content using the different levels of display performance, after having presented the frame of visual content using the different levels of display performance.
5. The computer-implemented method of claim 4, wherein: the computing device presents the multiple additional frames of visual content over a period of time during which additional user input both contacts the selected region of the touchscreen display and does not contact the touchscreen display, such that the computing device presents frames of visual content using the different levels of display performance through a period of time during which no user input is contacting the touchscreen display, following an end of a first user contact and a beginning of a second user contact.
6. The computer-implemented method of claim 1, wherein the selected region of the touchscreen display includes and is larger than an area of the first user contact at the first location of the touchscreen display.
7. The computer-implemented method of claim 1, wherein: the method comprises determining, by the computing device, an indication of an orientation of a user finger that provided the first user contact at the first location of the touchscreen display, based on analysis of a shape of an area of the first user contact at the first location; and identifying the selected region of the touchscreen display includes identifying the selected region based on the determined orientation of the user finger; and the selected region includes:
(1) the area of the first user contact at the first location; and
(2) a second area outside the first user contact at the first location, and that is between the area of the first user contact at the first location and an edge of the touchscreen display along the determined orientation of the user finger, based on the determined orientation of the user finger, such that the second area represents area under the user finger and not contacted by the user finger during the first user contact.
8. The computer-implemented method of claim 1, wherein identifying the selected region of the touchscreen display includes: sending, to a computational model, an indication of the user input that contacted the touchscreen display; and receiving, from the computational model, an indication of the selected region, or information that the computing device uses to identify the selected region.
9. The computer-implemented method of claim 1, wherein the computing device identifies the selected region of the touchscreen display based on:
(i) locations at which the user input contacted the touchscreen display; and
(ii) data received from orientation and/or movement sensors of the computing device during receipt of the user input, such that the computing device is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of different data being received from the orientation and/or movement sensors during receipt of the user input.
10. The computer-implemented method of claim 1, wherein the computing device identifies the selected region of the touchscreen display based on:
(i) locations at which the user input contacted the touchscreen display; and
(ii) a type of user interface being presented by the touchscreen display during receipt of the user input, such that the computing device is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of a different type of user interface being presented during receipt of the user input.
11. The computer-implemented method of claim 1, wherein: the different levels of display performance include using different brightness levels for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance to depict the different portions of the visual content includes:
(i) presenting the first portion of the frame of visual content with a first level of brightness; and
(ii) presenting the second portion of the frame of visual content with a second level of brightness that is less than the first level of brightness.
12. The computer-implemented method of claim 11 , wherein: the method comprises identifying data that specifies different levels of intensity for pixels within the selected region of the touchscreen display the different levels of intensity producing a gradient across at least part of the selected region; and presenting the second portion of the frame of visual content with the second level of brightness includes presenting the second portion of the frame with varying reductions in brightness, with respect to the first level of brightness, based on the data that specifies the different levels of intensity for the pixels within the selected region of the touchscreen display producing a gradient of brightness deviation from the first level of brightness within the selected region of the touchscreen display.
13. The computer-implemented method of claim 1, wherein: the different levels of display performance include different color depths for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance for the different portions of the frame of visual content includes:
(i) presenting the first portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a first number of bits; and
(ii) presenting the second portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a second number of bits that is lower than the first number of bits.
14. The computer-implemented method of claim 1, comprising: determining, by the computing device, that the user input has ended its interaction with the touchscreen display; and presenting, by the computing device after having determined that the user input ended its interaction with the touchscreen display, another frame of visual content on the touchscreen display, using a same level of display performance for the first portion of the another frame and the second portion of the another frame, as a result of having determined that the user input ended its interaction with the touchscreen display.
15. A computing system, comprising: a touchscreen display configured to present visual content; one or more processors; and one or more computer-readable devices including instructions that, when executed by the one or more processors, cause the computing system to perform operations that include: receiving user input that contacts the touchscreen display, including first user contact at a first location of the touchscreen display; identifying a selected region of the touchscreen display based on having received the user input, the selected region including the first location at which the user input contacted the touchscreen display; generating data that represents a frame of visual content for presentation by the touchscreen display; and presenting the frame of visual content on the touchscreen display, using different levels of display performance to depict different portions of the visual content, including by:
(i) presenting a first portion of the frame of visual content with a first level of display performance, the first portion of the frame corresponding to and displayed by an area outside the selected region of the touchscreen display; and
(i) presenting a second portion of the frame of visual content with a second level of display performance that is lower than the first level of display performance, the second portion of the frame corresponding to and displayed by an area within the selected region of the touchscreen display.
16. The computing system of claim 15, wherein the computing system is configured, such that the computing system would have consumed more power presenting the second portion of the frame of visual content with the first level of display performance, than the computing system consumed presenting the second portion of the visual content with the second level of display performance.
17. The computing system of claim 15, the operations including: detecting, by the computing system, the user input over a period of time during which the computing system presents multiple frames on the touchscreen display, the user input including:
(i) the first user contact at the first location of the touchscreen display; and
(ii) a second user contact at a second location of the touchscreen display that is different from the first location, the second user contact initiating contact with the touchscreen display after the first user contact released from being in contact with the touchscreen display, such that during a time between the first user contact ending and the second user contact beginning, there was no user contact with the touchscreen display, wherein the selected region contiguously includes an area of the first user contact at the first location and an area of the second user contact at the second location.
18. The computing system of claim 17, the operations comprising: presenting, by the computing system, multiple additional frames of visual content using the different levels of display performance, after having presented the frame of visual content using the different levels of display performance.
19. The computing system of claim 18, wherein: the computing system presents the multiple additional frames of visual content over a period of time during which additional user input both contacts the selected region of the touchscreen display and does not contact the touchscreen display, such that the computing system presents frames of visual content using the different levels of display performance through a period of time during which no user input is contacting the touchscreen display, following an end of a first user contact and a beginning of a second user contact.
20. The computing system of claim 15, wherein the selected region of the touchscreen display includes and is larger than an area of the first user contact at the first location of the touchscreen display.
21. The computing system of claim 15, wherein: the operations comprise determining, by the computing system, an indication of an orientation of a user finger that provided the first user contact at the first location of the touchscreen display, based on analysis of a shape of an area of the first user contact at the first location; and identifying the selected region of the touchscreen display includes identifying the selected region based on the determined orientation of the user finger; and the selected region includes:
(1) the area of the first user contact at the first location; and
(2) a second area outside the first user contact at the first location, and that is between the area of the first user contact at the first location and an edge of the touchscreen display along the determined orientation of the user finger, based on the determined orientation of the user finger, such that the second area represents area under the user finger and not contacted by the user finger during the first user contact.
22. The computing system of claim 15, wherein identifying the selected region of the touchscreen display includes: sending, to a computational model, an indication of the user input that contacted the touchscreen display; and receiving, from the computational model, an indication of the selected region, or information that the computing system uses to identify the selected region.
23. The computing system of claim 15, wherein the computing system identifies the selected region of the touchscreen display based on:
(i) locations at which the user input contacted the touchscreen display; and
(ii) data received from orientation and/or movement sensors of the computing system during receipt of the user input, such that the computing system is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of different data being received from the orientation and/or movement sensors during receipt of the user input.
24. The computing system of claim 15, wherein the computing system identifies the selected region of the touchscreen display based on:
(i) locations at which the user input contacted the touchscreen display; and
(ii) a type of user interface being presented by the touchscreen display during receipt of the user input, such that the computing system is configured to identify a different selected region of the touchscreen display for presentation of visual content at a lowered level of display performance, as a result of a different type of user interface being presented during receipt of the user input.
25. The computing system of claim 15, wherein: the different levels of display performance include using different brightness levels for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance to depict the different portions of the visual content includes:
(i) presenting the first portion of the frame of visual content with a first level of brightness; and
(ii) presenting the second portion of the frame of visual content with a second level of brightness that is less than the first level of brightness.
26. The computing system of claim 25, wherein: the operations comprise identifying data that specifies different levels of intensity for pixels within the selected region of the touchscreen display the different levels of intensity producing a gradient across at least part of the selected region; and presenting the second portion of the frame of visual content with the second level of brightness includes presenting the second portion of the frame with varying reductions in brightness, with respect to the first level of brightness, based on the data that specifies the different levels of intensity for the pixels within the selected region of the touchscreen display producing a gradient of brightness deviation from the first level of brightness within the selected region of the touchscreen display.
27. The computing system of claim 15, wherein: the different levels of display performance include different color depths for the different portions of the frame of visual content; and presenting the frame of visual content using the different levels of display performance for the different portions of the frame of visual content includes:
(i) presenting the first portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a first number of bits; and
(ii) presenting the second portion of the frame of visual content using image data that encodes each sub-pixel using a color depth of a second number of bits that is lower than the first number of bits.
28. The computing system of claim 15, the operations comprising: determining, by the computing system, that the user input has ended its interaction with the touchscreen display; and presenting, by the computing sy stem after having determined that the user input ended its interaction with the touchscreen display, another frame of visual content on the touchscreen display, using a same level of display performance for the first portion of the another frame and the second portion of the another frame, as a result of having determined that the user input ended its interaction with the touchscreen display.
EP23848749.0A 2023-12-28 2023-12-28 Power saving device with variable touchscreen display performance Pending EP4602448A1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2023/086136 WO2025144401A1 (en) 2023-12-28 2023-12-28 Power saving device with variable touchscreen display performance

Publications (1)

Publication Number Publication Date
EP4602448A1 true EP4602448A1 (en) 2025-08-20

Family

ID=89854345

Family Applications (1)

Application Number Title Priority Date Filing Date
EP23848749.0A Pending EP4602448A1 (en) 2023-12-28 2023-12-28 Power saving device with variable touchscreen display performance

Country Status (2)

Country Link
EP (1) EP4602448A1 (en)
WO (1) WO2025144401A1 (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5446624B2 (en) * 2009-09-07 2014-03-19 ソニー株式会社 Information display device, information display method, and program
US9377894B2 (en) 2014-05-22 2016-06-28 Sony Corporation Selective turning off/dimming of touch screen display region
CN110023881B (en) * 2016-12-01 2022-07-08 昆山云英谷电子科技有限公司 Region-based display data processing and transmission

Also Published As

Publication number Publication date
WO2025144401A1 (en) 2025-07-03

Similar Documents

Publication Publication Date Title
US11403984B2 (en) Method for controlling display and electronic device supporting the same
US10971074B2 (en) Displays with multiple scanning modes
KR102234523B1 (en) Pixel circuit and organic light emitting display device including the same
US10964262B1 (en) Systems and methods for reducing visual artifacts in displays due to refresh rate
US9368067B2 (en) Organic light-emitting diode display with dynamic power supply control
US10210801B2 (en) Electronic display driving scheme systems and methods
US12148382B2 (en) Image change sequence to prevent optical artifacts in low refresh rate AMOLED displays
TWI827231B (en) Pixel circuit and display device including the same
US10726811B2 (en) Electronic devices with displays
US20250218380A1 (en) Variable brightness dimming of display peripherals
WO2022028435A1 (en) Apparatus and method for driving display panel in power saving mode
CN119107888A (en) Display device, method of driving the same, and electronic device including the same
US12046193B2 (en) Variable frame rate display system
EP4602448A1 (en) Power saving device with variable touchscreen display performance
CN117174009A (en) Screen driving circuit, screen refreshing method and electronic device
KR102903705B1 (en) Method for operating display system at low power with mobile industry processor interface and display device using the same
KR102601613B1 (en) Display device and method for driving the same
KR102929088B1 (en) Method for driving a display device at lower power and display driver ic using the same
CN116529807A (en) Display device with reduced rounded corner bezel size
US20260087968A1 (en) Segmented Panel Regions for Driving Power Saving
US20260065824A1 (en) Display Strain Aware Frame Insertion Frame Rate Sequencing
US12062313B2 (en) Systems and methods for clock frequency control during low display refresh rates in electronic devices
WO2026066538A1 (en) Display method, electronic device, computer-readable storage medium and program product
CN120469659A (en) Image display method, electronic device, chip system, storage medium and program product
CN116153265A (en) Display device and method of operating display device

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20241108

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR