TW202013168A - Gui elements on a bistable display - Google Patents

Abstract

The present invention relates to a system (100). The system (100) includes a first display device (104), a second display device (106), and a memory (110) that stores the machine-executable instructions. The first display device (104) is a volatile display, while the second display device (106) is a bistable display. The system (100) further includes a processor (112) that controls the system (100) by means of execution of the instructions. The execution of the instructions makes the system (100) execute a first application. The execution of the first application generates a first graphic user interface element on the first display device (104) , which is the first GUI element (202). The execution of the instructions further makes the system (100) generate a graphic copy (400) of the first GUI element (202), and displays the graphic copy (400) on the second display device (106).

Description

GUI elements on bistable display

The present invention relates to systems including volatile displays and bistable displays, methods of operating such systems, and computer program products.

Mobile telecommunications devices usually have LCD, TFT, OLED or AMOLED displays to provide users with a touch screen sensitive visual interface. With the further development of mobile telecommunication devices, the trend is to additionally provide bistable displays to mobile telecommunication devices so that users can always access the information displayed on the bistable displays. The bistable display itself is known in the art and has the advantage that information can be permanently displayed on the bistable display by consuming minimal energy.

For example, US 2009/0085920 A1 describes a complementary dual display for electronic devices based on an application programming interface for providing native and non-native display utilities.

The object of the present invention is to provide a system including a volatile display and a bistable display, a method of operating such a system, and a computer program product.

The object of the present invention is solved by independent request items. The preferred embodiment is described by an attached request item.

A system including a first display, a second display, and memory storing machine-executable instructions is described, such as a mobile telecommunications device. The first display is preferably a volatile display, and the second display is a bistable display. The system further includes a processor for controlling the system by executing instructions, the execution of the instructions causes the system to execute a first application, and execution of the first application produces a first graphical use on the first display An interface element (first GUI element), execution of the instruction further causes the system to generate a graphical copy of the first GUI element and display the graphical copy on the second display.

In the following, a GUI element is understood as any graphical element that can be displayed on the first display. This can include graphical and textual information as well as active and inactive parts. The active part is understood as an area on the first GUI element that the user can select by, for example, touch or mouse click. The inactive part is an unselectable area on the first GUI element. The reason why the user can select or can be understood is that the first application can respond with a specific action in response to detecting the selection of the area on the first GUI element.

A volatile display is understood to be any display that requires a permanent supply of energy in order to maintain visual information displayed on the display. Examples are LCD, CRT, LED, OLED, PDP, AMOLED and ELD displays. Volatile displays need to refresh the primitives periodically to maintain their state, even for static images. This refresh usually occurs many times per second. If this is not done, then the primitives will gradually or immediately lose their coherent state and the image will disappear from the screen. In contrast, bistable displays can maintain static text and images for long periods of time, preferably indefinitely without power. An example is an electrophoretic display (EPD) that forms an image by rearranging charged pigment particles using an applied electric field. Another example is an electrowetting display (EWD) based on controlling the shape of a limited water/oil interface by the applied voltage. Other bistable displays can be electrofluidic displays, as well as interferometer modulators, such as "Mirasol" displays. In the context of this description, a bistable display has at least two stable states representing, for example, charged and uncharged pigment particles. However, it will be understood that bistable displays also encompass multi-stable displays that also have at least two stable display states.

Embodiments of the present invention may have the advantage that the display results of the first application originally intended specifically for the first display are now available on the second display. Because the second display is a bistable display, this can permanently show a copy of the first GUI element on the second display without supplying additional energy. In addition, because it is not necessary for the volatile display to refresh the information shown (as discussed above), the processor resources for displaying the first GUI element are minimized. Furthermore, for each first application that will display its graphical output on the second display, no special software development is required to enable the graphical output of the corresponding first application to be displayed on the second display. Therefore, a single first application is sufficient, and there is no need to install another first application dedicated for use in combination with the second display.

In summary, the present invention may have the advantage that the graphical output generated by the first application may be permanently shown on the second display, thus permanently providing information generated by the first application to the user. This can be achieved with minimal processor and memory resources. Therefore, the embodiments of the present invention can provide an enhanced level of information in an energy-saving manner.

For example, in the case of a mobile telecommunication device, the calendar can be permanently displayed on the bistable display, thus enabling the user to check without turning on the mobile telecommunication device (for example) by turning on the first display His calendar. However, the user does not need to install a separate calendar software dedicated to displaying the calendar on the bistable display of his device on his mobile telecommunication device.

Throughout this disclosure, a "graphic copy" is understood as a version of a first GUI element that may or may not be modified in terms of static and dynamic functionality of the first GUI element. The static functionality of the first GUI element is to display only any kind of information, while the dynamic functionality is possible to respond with actions associated with selecting a part of the first GUI element with dynamic functionality. The dynamic functionality therefore corresponds to the user selectivity mentioned above.

It must be noted that throughout this description, there may be more than two displays represented by the first and second displays. Thus, the system may include multiple displays, where one of the displays is a first display and the other of the displays is a second display.

According to an embodiment of the present invention, the memory further includes a second application, and execution of the instruction causes the system to execute the second application, and execution of the second application generates a graphical window on the first display. A first GUI element is included in the window, and executing the second application further causes the system to wait for an instruction command, the instruction command including a trigger, wherein the copy is generated and displayed on the second display after receiving the trigger.

The trigger may be a dedicated user command or user input, which may be that a specific time elapses after starting execution of the second application, or any user regarding the first GUI element or the second application when the first GUI element is displayed in the window operating.

Compared with the prior art (ie, US 2009/0085920 A1), embodiments according to the present invention do not require any modification of the first application originally dedicated to the specialized operation on the first volatile display in order to apply the corresponding application For the second bistable display. Based on the present disclosure, any application dedicated to the first volatile display can be applied to the second bistable display without modifying the corresponding first application. It is only the second application that specifically controls the generation of copies.

Embodiments can therefore have the advantage that the user can see and visualize the first GUI element in the "draft state" without any modification of the first application originally dedicated to the specialized operation on the first display. For example, the user can configure the appearance and functionality of the first GUI element in the manner that he will use in the case where the first GUI element dedicated to the first display is exclusively used.

For example, the instruction command is any one of the following: a size adjustment command for adjusting the size of the first GUI element, a positioning command for adjusting the specified position of the first GUI element on the second display , A design command for adjusting the design of the first GUI element, a configuration command for configuring the runtime behavior of the first GUI element when the first GUI element is displayed on the second display. The designated position of the first GUI element on the second display may be adjusted relative to the position of the first GUI element currently displayed in the window mentioned above.

In a practical example, the first GUI element may be a widget originally intended to be displayed on the first display. In another example, the second application may be a so-called'console manager application'. After launching the console manager application, the user may be able to select a specific widget and display the widget in the console manager application (ie, in the window provided by the console manager application on the first display) part. The user can then adjust, for example, any of the following: size, appearance, dynamic functionality. Thereafter, after a dedicated user command, the console manager application may generate an image related to the widget material suitable for the second display of the system (eg, mobile telecommunications device). Thereafter, the image of the widget may be displayed on the second display with the assistance of, for example, the EPD driver.

In general, the second display can be subdivided or organized in different consoles (ie, surfaces). Each console may be assigned to a specific first application (and corresponding copy of the first GUI element). The console may be displayed on the second display at the same time, or the console may be user-accessible and then displayed by a dedicated user selection on, for example, a touch-screen sensitive second display. For example, the user can swipe his finger across the second display to switch between consoles.

According to an embodiment of the present invention, the execution of the instruction causes the system to remove the first GUI element from the first display that generated the copy. The removal of the first GUI element may be completed automatically automatically after the copy is generated, or after the instruction command mentioned above has been received. Another possibility is to complete the removal of the first GUI element from the first display after closing the graphical window containing the first GUI element. Closing the graphics window can be user initiated.

This can have the advantage that it can provide cleaning of the first display. However, because the first GUI element is now displayed on the second display as some kind of copy, there is no or almost no loss of information that can be presented to the user of the system.

According to an embodiment of the invention, the copy may be any of the following:

-Screenshot

-A series of representations of a single second GUI element, the single second GUI element forming a first GUI element, the representation of the single second GUI element describes the corresponding single second GUI in any of the following aspects Elements: visual appearance, size, relative position with respect to each other, selectable or non-selectable, data values visually presented by a single second GUI element.

The screenshot can have the advantage of capturing the visual appearance of the first GUI element without distinguishing between possible static or dynamic functionality. Therefore, without any further explanation of the functionality of the first GUI element, a simple'picture' of the GUI element is presented on the second display. Optionally, the size of the picture can be adjusted according to the resolution of the second display, and the color depth of the picture can also be reduced according to the color depth capability of the second display.

Therefore, in the case of operating with a copy as a screenshot, providing the first GUI element on the second display can be done in a very fast and simple way, which is currently supported by a large number of operating systems, including Microsoft Windows , Android, Amazon Kindle, Chrome OS, HP WebOS, iOS, Linux and more operating systems.

In the case where the copy is a representation of a series of single second GUI elements as proposed above, this can have the advantage that the layout of the copy can be controlled in an extremely high way. For example, the highly complex graphical features of the first GUI element (ie, the second GUI element) are not presented, and the summary of the features can be used as a representation of a single second GUI element, which reduces complexity and permits The associated information is displayed on the second display, even if the resolution and color depth of the second display may be completely different from the resolution and color depth of the first display.

According to an embodiment of the invention, the execution of the instruction causes the system to:-determine the selection of the representation of the selectable second GUI element,-after determining the selection, execute the corresponding selectable second GUI element The actions associated with the selection.

This may have the advantage that the copy presented on the second display may mimic the dynamic functionality of the first GUI element in a complete or at least partial manner. For example, in the case where the first GUI element is a widget, after displaying a copy of the widget on the second display, the user can touch the widget at a specific position on the second display. Subsequently, the coordinates or gesture types of the touch area are compared with the coordinates of the active (ie, dynamic) part of the widget. In the case where the inactive part of the copy of the widget has been touched on the second display, nothing will happen. On the contrary, in the case where the active part of the copy of the widget has been touched on the second display, a specific action or situation can then be implemented, depending on the type of part of the widget touched.

According to an embodiment of the present invention, determining the selection of the selectable second GUI element includes: determining the spatial position of the selection on the second display, wherein the corresponding representation of the selectable second GUI element is currently displayed on In the case of the determined spatial position of the second display, the selectable second GUI element is regarded as being selected.

According to an embodiment of the present invention, the first GUI element is data represented graphically, wherein the action is any one of the following:-trigger or function implemented by the first application,-execution of another in the foreground or background An application,-a sub-program that executes the first application in the foreground or background.

The functions implemented by the first application may be, for example; switching functions, such as turning on or off the wireless LAN, turning on or off Bluetooth; performing the update of the data represented by the first GUI element; switching the page currently presented by the widget to Another page. The latter instance may be relevant in the case of a calendar widget, where the trigger may be to switch from a given month to the next month or the previous month. In general, the first application may be able to present information in multiple pages, each page corresponding to a corresponding first GUI element. By triggering the function'next' or'previous', any one of the pages represented by the corresponding first GUI element can be provided on the second display, thus completely or partially replacing the previously displayed page.

The sub-program that executes another application or the first application in the foreground or background can be multifaceted. For example, executing another application or sub-program in the foreground may change the first GUI element copied on the second display and/or it may display further information on the first display. Conversely, executing another application or a sub-program of the first application in the background may not immediately or at all cause a corresponding output on the first or second display. In a simple example of a clock widget, the subroutine may be a countdown or an alarm clock that is activated so that after the countdown ends or the alarm time is reached, a corresponding notification, such as an alarm sound, will be provided.

Therefore, the action may correspond to running the application to display the application or an explicit part thereof on the screen. It can run the application in the background or start some functions of the application in the background, which can correspond to pushing and displaying data to the widget itself after a certain period of time, the widget is copied on the second screen The widget or it may be a separate widget on the first screen. It can correspond to pushing and displaying data to the widget (on the first or second screen) itself after intent, or it can correspond to (for example) displaying several events if the widget is a calendar widget .

According to an embodiment of the present invention, the triggering or execution of another application or the sub-program of the first application will result in any of the following:-updating the data represented by the graphic copy, the execution of the command results in the use of the updated To update the graphic copy,-virtual graphic output, the execution of the instruction results in the display of the graphic representation of the virtual graphic output on the second display.

For example, the update of the material represented graphically by the graphical copy may be performed based on the background execution of the first application that caused the updated material. Subsequently, on the second screen, the specific graphic portion of the graphic copy representing the data to be updated is modified accordingly, so that the updated version of the data is finally shown as a substitute for the previously graphically represented data. This can have the advantage that computing resources are extremely effective when there is only a slight change in the updated information required.

The minor change may be, for example, in the case where the first GUI element is a digital clock widget, replacing the digital. Conversely, in the case of displaying more complex graphics information on the second display via a copy, the execution of the instruction may result in virtual graphics output, ie, it is completely generated in the system's memory but will not be effectively displayed on the first display Output. Subsequently, a representation of the virtual graphical output may be provided on the second display, again for example an image corresponding to a'screen shot', or again through the group representation of the single second GUI element mentioned above.

According to an embodiment of the present invention, the first GUI element graphically represents the data, and execution of the instruction causes the system to execute the first application in the background after generating the graphical copy, and executing the first application in the background results in the graphical representation of the Update the data, and the execution of the instruction causes the updated data to be used to update the graphic copy. This can have the advantage that there is no need to actually use the updated material to display the first GUI element on the first display, it is possible to update the copy on the second display accordingly, thus avoiding the use of the first display and thus saving energy. It must be noted here that operation, especially the display on mobile telecommunication devices, is often the main source of power consumption. This is a serious problem because the battery capacity of mobile telecommunication devices is limited to a certain extent. For example, both displays are powered by the system's battery. In general, the system is powered by any arbitrary power source.

According to an embodiment of the invention, the system is a telecommunications device, wherein the first display and the second display are located on opposite sides of the telecommunications device. The telecommunication device may be, for example, a mobile phone, a smart phone, or a tablet PC.

In general, for all embodiments, the system may be a phone, mobile phone, smart phone, tablet PC, laptop, netbook, media player, helmet, car display system.

In another aspect, the invention relates to a method of operating a system including a first display and a second display, the first display is a volatile display and the second display is a bistable display, the method includes:- Execute the first application, the execution of the first application will generate a first graphical user interface element (first GUI element) on the first display,-generate a graphical copy of the first GUI element,-display the graphic on the second display Copy.

In another aspect, the invention relates to a computer program product that includes computer executable instructions for performing the method described above.

In general, any of the power sources described in this document can be any of the following: batteries, supercapacitors, primary energy sources (eg, wind, water, solar energy), secondary energy sources (gases) , Biochemical, thermal and dynamic energy.

It should be understood that one or more of the foregoing embodiments of the present invention may be combined as long as the combined embodiments are not mutually exclusive.

Those skilled in the art will understand that various aspects of the present invention may be embodied as devices, methods, or computer program products. Therefore, various aspects of the present invention may take the form of a complete hardware embodiment, a complete software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects. Generally, all aspects may be referred to herein as "circuits," "modules," or "systems." In addition, aspects of the present invention may take the form of computer program products embodied in one or more computer readable media having the computer executable embodied in the above Code.

Any combination of one or more computer-readable media can be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. "Computer-readable storage medium" as used herein encompasses any tangible storage medium that can store instructions that can be executed by a processor of a computing device. The computer-readable storage medium may be referred to as a computer-readable non-transitory storage medium. Computer-readable storage media may also be referred to as tangible computer-readable media. In some embodiments, the computer-readable storage medium may also be capable of storing data that can be accessed by the processor of the computing device. Examples of computer readable storage media include (but are not limited to): floppy disks, magnetic hard drives, solid state drives, flash memory, USB thumb drives, random access memory (RAM), read only memory (ROM) ), CD, magneto-optical disc and the register file of the processor. Examples of optical disks include compact disks (CD) and digital versatile disks (DVD), such as CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R disks. The term computer-readable storage medium also refers to various types of recording media that can be accessed by a computer device via a network or communication link. For example, data can be retrieved via a modem, via the Internet, or via a local area network. The computer-executable code embodied on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing. The network communication protocols that can be used for transmission include: any IP protocol, including TCP, UDP, ICMP, HTTP, FTP, POP, SFTP, IMAP; and other protocols, such as Bluetooth, Fibre Channel, SSH, TLS, PPP, NNTP, etc. . Any standard can be used to complete the transmission, including wireless standards, such as Bluetooth, ZigBee, WirelessUSB, WLAN, WIMAX, LTE, GPRS, EDGE, etc.

The computer-readable signal medium may contain the propagated data signal in which the computer-executable code is embodied, for example, in baseband or as part of a carrier wave. Such propagated signals can take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. The computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can transmit, propagate, or transmit programs for use by or in conjunction with an instruction execution system, device, or device.

'Computer memory' or'memory' are examples of computer-readable storage media. Computer memory is any memory that the processor can directly access. 'Computer storage device' or'storage device' is another example of a computer-readable storage medium. The computer storage device is any non-volatile computer-readable storage medium. In some embodiments, the computer storage device may also be a computer memory or vice versa.

As used herein, a "processor" encompasses electronic components capable of executing programs or machine-executable instructions or computer-executable code. References to computing devices including "processors" should be interpreted as possibly containing more than one processor or processing core. The processor may be, for example, a multi-core processor. A processor can also refer to a collection of processors within a single computer system or distributed among multiple computer systems. The term computing device should also be interpreted as possibly referring to a collection or network of computing devices each including a processor. Computer executable code may be executed by multiple processors within the same computing device or even distributed across multiple computing devices.

Computer-executable code may include machine-executable instructions or programs that cause a processor to perform aspects of the invention. Computer program code for performing the operations of various aspects of the present invention can be written by any combination of one or more programming languages including object-oriented programming languages (eg, Java, Smalltalk, C++, etc.) And a conventional programming language, for example, "C" programming language or similar programming language, and the computer executable code can be compiled into machine executable instructions. This further includes other well-known languages. In some cases, computer-executable code may be in the form of a high-level language or in a pre-compiled form, and used in conjunction with an interpreter that generates machine-executable instructions during operation. You can use Java to encode on Android.

The computer executable code may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or completely remote Run on a computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network (including local area network (LAN) or wide area network (WAN)), or can be connected to an external computer (for example, using Internet services Provider via the Internet).

Various aspects of the invention are described with reference to flowchart illustrations and/or step diagrams of methods, devices (systems) and computer program products according to embodiments of the invention. It will be understood that, where applicable, each step of the flowchart, illustration, and/or step diagram, or a portion of the described steps, can be implemented by computer program instructions in the form of computer executable code. It should be further understood that when not mutually exclusive, combinations of steps in different flowcharts, diagrams, and/or step diagrams may be combined. These computer program instructions can be provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing equipment to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated for implementation The component of the function/action specified in the steps of the flowchart and/or step diagram.

These computer program instructions can also be stored in a computer-readable medium, which can lead the computer, other programmable data processing equipment, or other devices to function in a specific manner so that the instructions stored in the computer-readable medium are generated An article of manufacture containing instructions to implement the functions/actions specified in the steps of the flowchart and/or step diagram.

It is also possible to load computer program instructions onto a computer, other programmable data processing equipment or other device, so that a series of operation steps are performed on the computer, other programmable equipment or other device, thereby generating a computer-implemented process, making The instructions executed on the computer or other programmable equipment provide a process for implementing the functions/actions specified in the steps of the flowchart and/or step diagram.

100‧‧‧System

101‧‧‧Housing

104‧‧‧ First display

106‧‧‧Second display

110‧‧‧Memory

112‧‧‧ processor

114‧‧‧Power

200‧‧‧Console Manager

202‧‧‧The first GUI element

204‧‧‧ button

400‧‧‧Graphic copy

402‧‧‧Second GUI element

500‧‧‧Step

502‧‧‧Step

504‧‧‧Step

508‧‧‧Step

510‧‧‧Step

512‧‧‧Step

514‧‧‧Step

516‧‧‧Step

518‧‧‧Step

In the following, a preferred embodiment of the invention is described in more detail, wherein: FIG. 1 is a schematic view through a cross section of a mobile telecommunications device, FIG. 2 shows a first display with a console manager and a widget, and FIG. 3 A first display with an application running is shown, FIG. 4 shows a second display with a widget shown in a duplicated manner, and FIG. 5 is an exemplary method of operating a system including the first and second displays flow chart.

In the following, similar elements are designated by the same reference digits.

FIG. 1 is a schematic diagram of a cross-section through a system including a first display 104 and a second display 106. In the example of FIG. 1, the system is a mobile phone with two displays, one on the front side (display 104) and one on the back side (display 106). Without limitation on generality, it is assumed that the display 104 is an LCD display on the front side, and the display 106 is a bistable EPD display on the back side. The mobile phone 100 has a housing 101 carrying two displays 104 and 106.

The mobile phone 100 further includes a memory 110 and a processor 112, the memory including machine-executable instructions. The processor can execute these instructions stored in the memory 110 to control the mobile phone 100. The mobile phone 100 further includes a power source 114 (eg, battery 114) to power all electrical components of the mobile phone (eg, display, processor, and memory mentioned above).

The LCD display 104 can display graphical user interface elements with high quality, that is, with high color depth and high resolution. Without limitation on generality, the first GUI element to be displayed by the processor 112 executing the first application stored in the memory 110 is a widget. In general, the first GUI element can be related to any output that the first application running on the mobile phone 100 can produce on the LCD display 104.

Due to the special and special design of the first application for providing the first GUI element on the LCD display 104 in an optimized manner, execution of the first application would not otherwise result in the display of the first GUI element on the EPD display 106. However, displaying the first GUI element completely or partially on the EPD display 106 may be beneficial because the display 106 does not require permanent refresh of the power consumption of the displayed information due to the general characteristics of the bistable display. Therefore, power consumption is reduced. In order to avoid completely redesigning and reprogramming the first application to natively support the graphic output on the EPD display 106, the memory 110 further includes instructions capable of executing the following when executed by the processor 112:

A second application that generates the graphical window 200 on the LCD display 104 can be launched. This is depicted in FIG. 2, which shows the housing 101 carrying the LCD display 104. On the LCD display 104, a graphical window is shown as a'console manager' 200. Therefore, the second application provides the console manager. Within this graphical window 200 (ie, the console manager), a first GUI element 202 in the form of a widget is shown. This first GUI element is the result of executing the above-mentioned first application stored in the memory 110, which is natively designed only to provide graphical output to the LCD display 104.

In general, there may be several of these console managers, which may have widgets, headlines, and navigation bars on them. Widgets can be added with the assistance of the console manager. This then allows the user to use any kind of command command to interact with the displayed widget 202. For example, the user can adjust the size of the widget 202, he can position the widget at a specific location within the console manager, he can configure it after selecting a specific dynamic element provided by the widget The behavior of the widget, and he can configure the runtime behavior of the widget when this widget is displayed on the EPD display 106 later.

After the widget 202 is configured, arranged, and sized to satisfy the user, the user can trigger the display of a graphical copy of the widget on the EPD display 106. The trigger can be initiated by using a specific user command, for example, by pressing a button 204 displayed on the touch-sensitive LCD display 104 by the console manager.

Thereafter, the operating system or console manager of the mobile phone 100 or any other dedicated software can then generate a graphical copy of the configured widget 202 and send this copy for display to the EPD display 106. This can be done with the help of a dedicated EPD driver. In the simplest example, the copy may be a screenshot of the widget 202 currently displayed on the display 104. In a more complicated manner, the copy may be a representation of a series of single second GUI elements that form the first GUI element (ie, the widget 202). These single second GUI elements can be described by: their visual appearance, size, relative position with respect to each other, selectable or non-selectable (ie, when the user selects the corresponding second GUI element A subsequent reaction), the data value visually represented by a single second GUI element, etc.

A copy of the widget 202 can then be displayed on the EPD display 106. Thereafter, it is preferable that the console manager 200 can be closed and the LCD display 104 can be used for other purposes. However, the previously configured widget will remain permanently displayed on the EPD display 106.

Therefore, widgets can be added with the assistance of the front screen application console manager 200. This allows the user to make different actions on the widget on the back screen (ie, EPD display 106). The front screen application console manager can allow browsing of different widgets; select a single one of them; arrange them on the console and configure them; change the size; and of course, delete what is currently displayed on the EPD display 106 Widget. In addition, any one of the widgets (generally, the first GUI element) currently displayed on the EPD display 106 can be configured and rearranged accordingly using the console manager.

The user can drag and drop any widget from the widget list to the console manager window 200 when he needs it. The widget list can be provided by the operating system of the system or the console manager itself. After any action performed on the widget in the console manager, the corresponding changes will therefore be applied to the corresponding widget on the display 106 and will be based on these changes, for example with the assistance of the EPD controller, on the back Shown on the display.

The actual example can be a calendar widget: first, in the console manager, the calendar widget can be arranged according to its size and position and its visual appearance. Thereafter, the calendar widget itself can download the current date, event, etc. from the calendar application built in the mobile phone 100. This causes the current date, event, etc. to be displayed normally through the calendar widget on the LCD display, that is, in the window 200 of the console manager. It must be noted here that the selection of the size of the widget can be started after placing the calendar widget in its place or after configuring the type of calendar to be displayed (for example) of the calendar widget set up. The console manager 200 can then save calendar widget settings, events and dates, buttons (ie, dynamic graphic elements), and the selected size, and generate a copy in a manner suitable for display on the EPD display 106, ie, with The image related to the configured widget. Thereafter, the image of the calendar widget can be dragged on the EPD display with the assistance of the corresponding EPD driver of the mobile phone 100, for example.

In another example, the console manager can be used to configure the execution of the first application stored in the memory 110 in full-screen mode and provide its graphical output to the EPD display in a manner optimized for display on the EPD display 106 .

An example is given in FIG. 3, which indicates on the LCD display 104 that a chess game is currently effectively running. This results in a corresponding graphical output 202. This output may be a notification of the second application: the second application mentioned above is currently outputting the graphical content of the game on the back screen (ie, EPD display).

The output result is shown in FIG. 4, that is, a copy of the first GUI element generated by the chess application. Here, the EPD display 106 displays a representation of a series of single second GUI elements that together constitute the graphical output of a chess game (i.e., the first application currently being run by the processor 112). Here, many of these representations of a single second GUI element are user-selectable. For example, by selecting a representation of a single second GUI element 402 via the touch-sensitive EPD display 106, a corresponding action associated with the selection of the corresponding selectable second GUI element 402 will be performed. As is apparent from Fig. 4, selecting the'Undo' GUI element will result in an action where the last input (for example, moving a chess piece) will be cancelled.

Another action in this example may be that after selecting the'hint' graphic element, the chess application will be controlled in such a way that it will virtually generate a graphic related to the prompt for how best to proceed with the next chess move Output. Virtually here means, for example, that graphics output will be generated in the video memory only through a chess game, and will not be actually displayed on the LCD display 104. Instead, it is preferable to copy the output to the EPD display 106 again in a manner optimized for color depth and resolution. The second application may be configured to "intercept" the graphical output intended for the first display and provide a copy of it on the second display.

Another example of interacting with a widget displayed on the EPD display of a dual-screen device (eg, device 100) may be the following: First, display the display widget on the second screen 106 (ie, EPD display) Or generally speaking, any copy of the first graphical user interface element. After receiving the user's selection (for example, the user touches a specific area of the EPD display), the touch area is determined. In the case of widgets, determine the widget touch area. The coordinates of the touch area or gestures used as input are then determined and compared with the corresponding coordinates and/or gesture types stored with respect to the displayed widget. To this end, the memory 110 includes the following information: which of the displayed individual GUI elements on the EPD display 106 are considered selectable and thus dynamic, and which are non-selectable and static.

In the case where the user has touched the active part (i.e., selectable or dynamic part), a specific situation or action associated with the selection of that part is implemented. In the example of FIG. 4, the action may be to undo the last chess move, or to provide a specific reminder, or to move the selected piece to the desired position. On the contrary, in the case where the displayed graphical user interface element or the inactive part of the specific widget has been touched, nothing will happen.

In the event that the active part of the widget has been touched, this will cause a change in the visual appearance of the widget on the EPD display, which in turn means that the information displayed on the EPD display 106 must be refreshed. Therefore, the widget will appear on the EPD display 106 in a modified version.

Another example of widget selection may be the following: in the case where the widget is a calendar widget, usually either a single appointment is selected from the calendar or a single month on the calendar widget displayed on the second display Or to switch between dates. In both cases, the user needs to touch a specific area of the displayed calendar widget, that is, a user-specific selection must be made. In the case where the corresponding calendar widget is displayed on the EPD display 106, the operating system of the mobile telecommunication device 100 can detect the user's interaction with the EPD display in a specific touch area (for example, using a finger touch), or even detect along Gestures of a specific touch area of the EPD display 106. Gestures are movement trajectories of input members such as a stylus or a user's finger, and conversely, a single touch is just a single point area on the EPD display. The gesture usually has a line shape, and the single touch has a dot (circle) shape.

After determining the user input by a single touch or gesture on the EPD display 106, the coordinates or gesture of the touch area or the area where the gesture is input are compared with the stored coordinates of the active part of the widget. Depending on whether the active or inactive part of the widget has been touched or generally selected, different actions associated with the selection can be performed. In the case of a calendar widget, the action may correspond to launching a dedicated calendar application on the LCD display 104. This can leave the appearance of the widget on the EPD display 106 unaffected.

Another action may correspond to switching one month on the displayed widget on the EPD display 106 to the next month. In order to realize this function, the first application can be run in the background, and the widget can be virtually generated and updated by the first application in the background. This will result in a virtual graphical output, which is then displayed on the EPD display 106 as a graphical representation of the virtual graphical output, ie, a corresponding update of the calendar widget.

In general, on the LCD display 104 or on the EPD display 106, the above-mentioned selection can result in the transfer of full-screen application data, including any of the following: event list, month view with events and dates, Navigation buttons, meetings, and the ability to accept and reject them.

Another practical example may be the case where it can be assumed that the portable device 100 can have two or more touch displays, and it can run on the Android operating system. The first screen may be a touch LCD, LED, OLED, or another non-bistable display. The second screen is a bistable display, such as an electronic paper display (EPD) or Mirasol display.

The standard Android interface can be displayed on the first screen, and the bistable display can represent a special UI compatible with its features. For example, EPD displays have a low refresh rate and only two-color primitives, so two-color schemes and elements are widely used and the amount of animation is reduced. The second display can be divided into several working areas, so-called consoles. Each console can contain a navigation bar and can have a set of widgets on it. The user can navigate through the console and interact with the widget. Examples of widgets are calendar widgets, RSS widgets, SMS widgets, and social network widgets. They can intentionally refresh data within a certain period of time, and have control or display data from the network and any application.

Therefore, in general, any of the features described above with respect to FIGS. 1 to 4 can be applied to full-screen widgets, or individual widgets that can be placed as copies on the EPD display 106. In the latter case, these individual window widgets can be displayed on the EPD display 106 in a dedicated window (so-called'console'). The console manager 200 of FIG. 2 can manage the contents of individual consoles. The user can scroll through different consoles on the EPD display 106 through corresponding dedicated interactions.

Generally speaking, the user can change the appearance of the user interface of the second screen with the assistance of the first screen. The special application mentioned above (so-called console manager 200) allows the user to select, place, resize, delete, and perform other operations on the widget or any other graphical user interface element on the second screen .

The user can touch (for example) the arrow on the calendar widget on their second screen. When the user touches an arrow that navigates between different months, the coordinates of the touched area are compared with the coordinates of the widget. When it is determined that the user has touched the arrow, the month will change to the next month or the previous month on the second screen.

In another example, the user can touch the calendar itself, and this action will direct him to a full-screen application that will open on the first or second screen. The event list and current month can be displayed.

In the case of different widgets, the user can touch the play button (play widget) and it will open the music player in the background. Widgets such as RSS widgets can receive data intentionally or after a certain period of time. Different actions can be performed depending on the touch area of the widget. In the case of different posts displayed in the widget, the entire post will be opened after selecting its main body in the widget. In another example, in the case of an RSS widget, a tap on the ‘provide you more news...’ line will lead to a list of recent posts, etc.

After all actions performed by the widget and the application, the information displayed on the EPD display 106 may need to be updated. Special libraries and EPD drivers can be used here.

FIG. 5 is a flowchart for operating the system such as the mobile phone 100 depicted in FIG. 1. The method begins in step 500 by loading the appropriate widget into the window of the console manager on the first display (such as the LCD display 104). Thereafter, the console manager receives a specific configuration regarding the displayed widget in step 502, such as the size, the content to be displayed, and the selection behavior of the single second GUI element displayed by the widget.

After the final user selection, in step 504, an image of the configured widget is generated. This may be a screenshot or a representation of a series of individual second GUI elements that form the widget. The images in step 506 are then displayed on the EPD display 106 in a static manner.

In step 508, it is determined whether the user has selected a dynamic part of the copy by touching a specific area on the EPD display 106. This is possible whether the copy shown on the EPD display 106 is a simple screenshot or whether it is a representation of a series of single second GUI elements. In the case of receiving any selection, the corresponding coordinate of the selection, that is, the coordinate of the touch on the EPD display 106 is determined in step 510.

In step 512, it is determined whether these coordinates correspond to the active (ie, dynamic) part of the widget. In the case where the coordinates correspond to an active part, in step 514, a corresponding action associated with the active part is performed. The method then continues in step 508.

However, if it is determined in step 508 that no selection has been received, ie, the user has not touched the EPD display, then the method continues in step 516, where it is determined whether the displayed widget needs to be updated. An example could be a clock widget, which needs to constantly update its content, because time elapses continuously and therefore the displayed time must be updated accordingly.

If it is found in step 516 that an update is needed, then in step 518 the corresponding virtual graphics output is generated by the associated application configured to natively generate the widget on the LCD display 104. Virtual graphics output means that the updated output is not actually displayed on the display 104, but it is only calculated internally. Subsequently, this virtual output image is generated again in step 504, and the method continues to display the corresponding generated image in step 506.

If it is determined in step 516 that no update is currently required, then the method continues with step 508 again.

It must be noted here that the flowchart in FIG. 5 is only exemplary. For example, step 516 may generally occur immediately after step 506.

100‧‧‧System

101‧‧‧Housing

104‧‧‧ First display

106‧‧‧Second display

110‧‧‧Memory

112‧‧‧ processor

114‧‧‧Power

Claims (15)

A system (100) including a first display (104), a second display (106), and a memory (110) storing machine-executable instructions, the second display (106) is a bistable display , The system (100) further includes a processor (112) for controlling the system (100) by executing the instructions, the execution of the instructions causes the system (100) to execute the first application, the The execution of the first application generates a first graphical user interface (GUI) element (202) on the first display (104), and the execution of the instruction further causes the system (100) to generate the first GUI A graphical copy (400) of the element (202) and display the graphical copy (400) on the second display (106). According to the system (100) of claim 1, the memory (110) further includes a second application, and execution of the instruction causes the system (100) to execute the second application, the second application Execution generates a graphical window (200) on the first display (104), includes the first GUI element (202) in the window, and execution of the second application further causes the system (100) to wait An instruction command including a trigger, wherein after receiving the trigger, the copy (400) is generated and displayed on the second display (106). According to the system (100) described in claim 2, the instruction command is any one of the following: a size adjustment command for adjusting the size of the first GUI element (202), for adjusting the A positioning command of the specified position of the first GUI element (202) on the second display (106), a design command for adjusting the design of the first GUI element (202) (106) When the first GUI element (202) is displayed, a configuration command for configuring the runtime behavior of the first GUI element (202) is displayed. According to the system (100) of any one of the preceding request items, execution of the instructions causes the system (100) to remove all of the copies from the first display (104) after generating the copy (400) The first GUI element (202) is described. According to the system (100) of any one of the preceding request items, the copy (400) is any of the following:-a screenshot,-a representation of a series of single second GUI elements (402), The single second GUI element (402) forms the first GUI element (202), the representation of the single second GUI element (402) describes the corresponding single second in any of the following aspects GUI elements: visual appearance, size, relative position with respect to each other, selectable or non-selectable, data values visually presented by the single second GUI element. According to the system (100) of claim 5, the execution of the instruction causes the system (100)-to determine the selection of the representation of the selectable second GUI element,-after determining the selection, to execute The action associated with the selection of the representation of the selectable second GUI element. According to the system (100) of claim 6, the determination of the selection of the selectable second GUI element includes determining a spatial position of the selection on the second display (106), wherein, in the In the case where the representation of the selectable second GUI element is currently displayed on the determined spatial position of the second display (106), the selectable second GUI element is considered to be selected. According to the system (100) of claim 6 or 7, the first GUI element (202) graphically represents the data, and the action is any one of the following:-the trigger is implemented by the first application The function of-execute another application in the foreground or background,-execute the sub-program of the first application in the foreground or background. According to the system (100) of claim 8, the triggering or the execution of another application or the execution of the sub-program of the first application results in any of the following:-updating by the graphic copy ( 400) The data represented graphically, execution of the instruction causes the updated copy to be used to update the graphical copy (400),-virtual graphics output, execution of the instruction results in the second display The graphic representation of the virtual graphic output is displayed on (106). According to the system (100) of any one of the preceding request items, the first GUI element (202) graphically represents the material, and execution of the instruction causes the system (100) to generate the graphical copy (400) ) And then execute the first application in the background, executing the first application in the background results in the updated data represented graphically by the graphical copy (400), and execution of the instruction results in the use of the updated data to Update the graphic copy (400). According to the system (100) of any of the preceding request items, the second display (106) is an EPD display. According to the system (100) of any of the preceding request items, the first display (104) is a volatile display. The system (100) according to any of the preceding request items, the system (100) is a telecommunications device, the first display (104) and the second display (106) are positioned on the telecommunications device On the opposite side. A method of operating a system (100) including a first display (104) and a second display (106), the first display (104) is a volatile display and the second display (106) is bistable Mode display, the method includes:-executing a first application, execution of the first application generates a first graphical user interface GUI element (202) on the first display (104),-generating the first A graphical copy (400) of a GUI element (202)-displaying the graphical copy (400) on the second display (106). A computer program product comprising machine executable instructions for performing the method according to claim 14.

Images