GB2470658A

GB2470658A - Virtual image of a secondary display that has been disconnected.

Info

Publication number: GB2470658A
Application number: GB201008977A
Authority: GB
Inventors: Andrew John Fisher
Original assignee: DisplayLink UK Ltd
Current assignee: DisplayLink UK Ltd
Priority date: 2009-05-28
Filing date: 2010-05-28
Publication date: 2010-12-01
Also published as: WO2010136781A1; GB201008977D0; GB0909097D0

Abstract

Disclosed is a method and system for controlling the images shown on display screens when a secondary screen in disconnected. The display system has a display controller, a primary display and a secondary display. The method of operating the display system starts by displaying a first image on the primary display, and a second image on the secondary display. When it is detected that the secondary display has been disconnected, the method stores the second image as a virtual image. A user can then request a display switch, and the second image is displayed on the primary display. If it is detected that the secondary display has been reconnected, then the second image is re-displayed on the secondary display and the first image is re-displayed on the primary display. The system may allow a the user to select the primary display show both images at the same time, either the images being the same size or the secondary image being smaller.

Description

DESCRIPTION

DISPLAY SYSTEM AND METHOD

This invention relates to a method of operating a display system, and to the display system itself.

In desktop computing, it is now common to use more than one display device. Traditionally, a user would have a computer with a single display device attached, but now it is possible to have more than one display device attached to the computer, which increase the usable area for the worker. For example, International Patent Application Publication WO 2007/020408 discloses a display system which comprises a plurality of display devices, each displaying respectively an image, a data processing device connected to each display device and controlling the image displayed by each display device, and a user interface device connected to the data processing device. The system is arranged, following a defined user input at the user interface, for at least two of the display devices, to move at least a portion of the image displayed to a different display device. When carrying out the moving of the at least a portion of the image displayed to a different display device, the system is arranged to scroll the at least a portion of the image between the two display devices.

The additional display device may be provided by a laptop computer, which the user will connect to their main computer when they are at their desk.

When the user leaves their normal working desk, for example to go to a meeting room, or to travel to an external appointment, then they will disconnect the laptop, which is providing the additional display function, in order to take the laptop with them. Whilst being able to add and remove displays to and from a running system with ease is a very useful feature it does highlight some issues with current systems.

A significant issue is what to do when a display disappears following the disconnection of a secondary display device. Applications running on this display device can no longer be seen by the user. One way that the display software deals with this issue is by moving any windows that were on the removed display back onto another still visible display. An extension to this behaviour is to move applications back if the display is reattached. Thus when a user needs to take their laptop to a meeting, they simply unplug the connection and all applications are moved back onto the laptop's main display.

On returning from the meeting, the user plugs the connection back in and the displays automatically configure themselves as before and the windows can be moved back onto the extra displays, either manually or automatically.

One example of such a system is disclosed in United States of America Patent Application Publication US 2008/0231546 which discloses a multi-display system that comprises a host, a main display, multiple second displays connectable to the host and multiple display detection units. The display detection units are adapted for detecting the statuses of the second displays and transferring detection signals to the host in real time. The host is adapted to record the display mode and display content of the currently connected second displays in real time, and conducts display control when receiving the detection signals from the display detection units. The host stops recording, saves the display content, modifies the display mode and transfers the content displayed by the second displays to the main display when the detection signals from the display detection units indicate the second displays have been disconnected from the host. Further, the host recovers the saved display mode and automatically sets the content displayed by the second displays based on the records of the second displays when the detection signal indicates that the second displays have been connected to the host.

Whilst this solution is functional, it is still not ideal, as many compromises must be accepted. For example, when a user has multiple displays they tend to open up many more applications and windows than they would on a single display system. So when the additional display is unplugged and all these windows are moved back onto the primary screen it becomes very difficult to see the wood for the trees, as a large amount of data designed for a much larger display area is crammed onto a smaller display area. Often the user will choose to close many functioning windows to make things more manageable. The content of the secondary display is simply dumped onto the primary display, and this can result in a display situation that is far from ideal for the user.

A different solution is provided by an image display system and method disclosed in United States of America Patent Application Publication US 2008/0218432. This disclosure is directed to an image display system. In one example, the system comprises a plurality of display apparatuses each having a display area. The display apparatuses are configured to display images based on image information supplied from a processing unit. If it is determined that a second display apparatus is in a condition in which it is not connected to a computer, the image area information to be displayed on the second display apparatus is acquired. In one embodiment, the screen area information of the second display apparatus is displayed reduced in such a way as to be inset in a portion of the display screen on the first display apparatus. Alternatively, the screen displays of the first display apparatus and second display apparatus are merely switched and displayed.

This solution has a number of problems. When a secondary display is disconnected, either reducing the size of the image on this display and transferring the reduced image to the primary display or simply switching the secondary image to the primary display do not provide a sufficiently user-friendly experience. In the first solution offered, the addition of a new window onto the main display may be a problem, as the user may not wish any new material to be placed onto the primary display image. The second solution of simply switching the two images is likely to be highly undesirable in most practical working situations, as by definition, the primary display device is showing what is most important to the user.

It is therefore an object of the invention to improve upon the known art.

According to a first aspect of the present invention, there is provided a method of operating a display system, the display system comprising a display controller, a primary display and a secondary display, the method comprising the steps of displaying a first image on the primary display, displaying a second image on the secondary display, detecting that the secondary display has been disconnected, storing the second image as a virtual desktop, receiving a user input indicating a display switch, and displaying the second image on the primary display.

According to a second aspect of the present invention, there is provided a display system comprising a display controller, a primary display and a secondary display, wherein the primary display is arranged to display a first image, the secondary display is arranged to display a second image, the display controller is arranged to detect that the secondary display has been disconnected, to store the second image as a virtual desktop and receive a user input indicating a display switch, and the primary display is accordingly arranged to display the second image.

According to a third aspect of the present invention, there is provided a computer program product on a computer readable medium for operating a display system, the display system comprising a display controller, a primary display and a secondary display, the product comprising instructions for displaying a first image on the primary display, displaying a second image on the secondary display, detecting that the secondary display has been disconnected, storing the second image as a virtual desktop, receiving a user input indicating a display switch, and displaying the second image on the primary display.

Owing to the invention, it is possible to improve the user experience when adding and removing displays from a running system. This invention combines the concept of displays being hot pluggable and the concept of virtual desktops. The result of this is a practical solution to a very real issue being faced daily by users of multi-monitor displays, and thus which will aid productivity. It also has the distinct advantage of being easily understood by the user so the learning curve is easy and adoption should be widespread.

Additional display devices can be removed and added to a display system and the image being shown by the removed display device is handled in a manner that allows the user access to the material in that image in a simple and efficient way. The image on the removed display device is neither discarded nor placed onto the primary display device. It is maintained as a virtual desktop which the user can call onto their main display device if they so wish at any future time. The user can then interact with the virtual desktop when it is shown on the primary display device and the changes that result from the interaction will be maintained.

Advantageously, the method further comprises detecting that the secondary display has been reconnected, displaying the second image on the secondary display and displaying the first image on the primary display. The system can also operate to detect the reconnection of the secondary display and will switch the virtual desktop back to the secondary display, when that reconnection is recognised. Any changes to the virtual desktop that have taken place in the meantime will be reflected in the new image displayed on the secondary display device. In this way, the disconnection and reconnection of an auxiliary display device is handled in a way that improves the user's is experience of the process.

Preferably, the method further comprises receiving a second user input indicating an overall view, generating a third image from the first and second images and displaying the third image on the primary display. In one embodiment, the third image comprises the first and second images each of substantially the same size. In a second embodiment, the third image comprises the first and second images, one of substantially the full image size and the other of smaller size, overlaying the one. Once the user has disconnected the secondary display, then a virtual desktop is created which the user can switch to on their primary display. However, the user may still wish to see all of the information contained in both images, and therefore a third image can be created which the user can see as an alternative to the simple switching between the two images. This improves the user's interaction Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:-Figure 1 is a schematic diagram of a display device, Figure 2 is a schematic diagram of a display controller and multiple display devices, Figure 3 is a schematic diagram of the display controller and primary and secondary display devices, Figure 4 is a schematic diagram of the display controller and the primary display device, Figure 5 is a schematic diagram of internal components of the display controller, and Figure 6 is a flowchart of a method of operating a display system.

The use of a virtual desktop is shown in Figure 1, which shows a display device 10, which is showing a virtual desktop 12 comprised of three separate images 12a, 12b and 12c. A virtual desktop is a system that allows a user to split their work across several virtual monitors, shown as components 1 2a, 1 2b and 12c. The system generally provides a mechanism for moving windows 14 between the virtual desktops 12 and also mechanisms for changing which of these virtual desktops 12 is to be displayed on the physical monitor 10, as the part of the virtual desktop with which the user will currently interact. Recently, for example, Apple has introduced this concept into their OSX operating system under the name of spaces.

The spaces idea implements smooth scrolling animations when switching between virtual desktops 12 (spaces as Apple calls them) and a zoomed out overview option, which is shown in Figure 1, that allows for easy movement of windows 14 between virtual desktops 12 and selection of which of desktops 12 to display on the monitor 10. A hotkey on a keyboard can be used to bring up the overview as shown in Figure 1. Windows 14 can then be dragged between the spaces 12. A click on a given space/desktop 12 will bring that space/desktop 12 to be the current space/desktop 12 shown as a full screen on the display device 10.

The advantage for the user is that they can have a larger desktop area available than just the limited space provided by the current display device 10.

A much larger number of windows can be open at the same time than would normally be usable with a desktop that is limited in size to the parameters of the display area of the display device 10. The user can switch to the overview option shown in Figure 1, using a hotkey, to decide which part of the overall virtual desktop 12 they wish to access, if they wish to use windows that are not currently being displayed on the display device 10.

As illustrated in Figure 2, for some time now it has been possible to have more than one display device 10 attached to a computer 16 to be used to extend the user's physical working space. This is usually achieved using a multi headed graphics card and is supported by most current desktop io operating systems. Windows 14 can be freely dragged from one monitor 10 to the next and can even overlap multiple monitors 10. Currently, if a user has multiple display devices 10 these are regarded by the computer 16 as a single space (a virtual desktop).

Recently, companies such as DisplayLink (UK) Limited have made multiple monitor configurations much more accessible, by allowing users to add monitors 10 to their computer 16 by plugging them in to the main display controller as USB peripherals. This makes it possible to add display devices 10 to an existing system, without even opening the case, in fact extra display devices 10 can be added to and removed from a running system. Whilst being able to add and remove displays 10 to and from a running system with ease is a very useful feature, it does highlight some issues with current systems. A significant issue is what to do when a display 10 disappears. Applications 14 running on this display 10 can no longer be seen by the user.

The software running on the display controller 16 deals with this issue by moving any windows 14 that were on the removed monitor 10 back onto another still visible monitor 10. An extension to this behaviour is to move applications 14 back if the monitor 10 is reattached. Thus, when a user needs to take their laptop to a meeting they can simply unplug the USB connection which links that laptop to the display controller 16, and all the applications 14 are moved back onto the computer's main display 10. On returning from the meeting, the user plugs the USB connection back in, and the displays 10 automatically configure themselves as before and the windows 14 can be moved back onto the extra displays 10, either manually or automatically.

A problem with the above system is that when a user has multiple displays 10 they tend to open up many more applications and windows 14 than they would on a single monitor system. So when the displays 10 are unplugged and all these windows 14 are moved back onto the primary screen 10, it becomes very difficult to see the wood for the trees. Often the user will choose to close many windows 14 to make things more manageable.

Figure 3 illustrates the example of a display controller 16, which is a io conventional desktop computer 16 with keyboard 18, which is controlling a primary display lOa and a secondary display lOb. The secondary display lOb is the display of a laptop computer 20 and this is being controlled by the display controller 16 via an Ethernet connection 22. The user can plug in the laptop 20 and utilise the display area provided by the secondary display device lOb of the laptop computer 20. When the user wishes to move from their current desktop location, they can disconnect the laptop 20 and take the laptop with them.

The improvement of the current handling of the display disconnection, in order to make the system more usable, is to combine the ideas of virtual desktops and multiple transient displays. When a display device lOb disappears, instead of its windows 14 being moved onto another display device lOa, the removed display device lOb is replaced by a virtual desktop which the user can switch to in the same way that they would in a virtual desktop environment. When the user re-plugs the removed display device lOb the virtual desktop now turns back into a real display device lOb. This system combines two ideas with which users are already familiar in a completely new way which will aid productivity, but is easily understood and adopted into the user's day to day computer use.

In Figure 3, the primary display device lOa is displaying a first image A and the secondary display device lOb is displaying a second image B. The images A and B may be made up of multiple individual components, such as a background and individual windows that are fully or partially overlaid on top of other windows. The images A and B will be stored in a display memory of the display controller 16 and read out by a graphics chip that is generating the graphics data in a format that is understood by the actual physical display 10.

As the user performs their normal desktop functions such as moving an on-screen cursor with an input device such as a mouse, then the display controller 16 will perform the necessary functions with respect to the different images A and B being displayed by the different display devices.

Figure 4 shows what happens when a user disconnects the secondary display device lOb. The first image A being displayed by the primary display device lOa is maintained as if nothing has changed, but the second image B is converted to a virtual desktop 12. The nature of this virtual desktop 12 is not material to the functioning of the overall system, it may be stored as a simple pixel image, but more appropriately, will be stored as a collection of the components that made up the original image B, with size and position information. In this latter case, the virtual desktop 12 will be stored as a background with one or more windows of specific size and location.

Any applications that are open, as indicated by the windows present, within the virtual desktop 12, will be maintained in their current state, and updated as normal, for example in response to processor actions. For example, if an email client is open, and a new email is received, the window of the email client will be updated accordingly, and the virtual desktop 12 will be updated in the same manner. The virtual desktop 12 is also available to the user of the computer that comprises the display device 1 Oa, processor 16 and user interface 18. For example, a hotkey may be designated on the keyboard 18, and the user can press this hotkey to switch the image A being displayed by the display device I Oa with the virtual desktop 12.

The user can interact with the components that make up the virtual desktop 12 in the normal manner, once the user has switched the display lOa to display the virtual desktop 12. For example, if a word processing application is open on the laptop computer 20 this will be part of the virtual desktop 12 when the laptop 20 is disconnected from the main computer. Once the user has switched to the virtual desktop 12, the window for the word processing application will be visible, and the user can interact with the application and make changes to a document, for example, and this will be reflected in changes to the virtual desktop 12.

A schematic diagram of the internal components of the display controller 16 is shown in Figure 5. The display controller 16 is connected to the display 10 and also to a user interface component 18. The user interface component 18 may comprises more than one individual device, such as a mouse and keyboard. The display controller 16 may be a standard desktop computer or could be a "thin" device which is more like a dedicated display io management component and does not have other general processing functions. The user controls the functions of the display controller 16 via the user interface component 18. The display controller provides a graphics output that is then displayed by the display 10.

The display controller 16 comprises a processor 24 which is connected to a graphics chip 26 an external interface component 28 and a memory 30. In Figure 5 these components are shown as individual discrete components but the functions provided by the different components could be distributed or combined in one or more devices. For example, the memory 30 could be provided directly on-board the processor 24, and the graphics chip 26 could also be provided with its own memory. Indeed, all of the functionality provided by the different components 24 to 30, shown within the display controller 16, could be provided by a single chipset that includes processing, memory and connection functionality. This will particularly be the case if the display controller 16 is a thin client.

As discussed above, when the secondary display lOb is disconnected from the display controller 16, then the image B displayed by that secondary display lOb is converted into a virtual desktop 12. Once this occurs, the user has the ability to switch between the first image A currently being shown by the primary display lOa, and the virtual desktop 12 being stored in the memory 30.

The user makes a predefined user input at the user interface 18. When this occurs, the display controller 16 will display the second image (stored as the virtual desktop 12) on the primary display lOa. In this way the user can toggle between the first and second images, for example by pressing a hotkey on the keyboard, which has been designated for this function.

The method of operating the display system is summarised in the flowchart of Figure 6. The display system comprises the display controller 16, the primary display lOa and the secondary display lOb, which are both connected directly or indirectly to the display controller 16. The method comprises the steps of, firstly step Si, displaying the first image A on the primary display iOa and secondly step S2, displaying the second image B on the secondary display lOb. At step S3, the display controller 16 is arranged to detect that the secondary display lOb has been disconnected. In general, in most modern connection protocols such as USB, the detection of a device disconnection is easily achieved according to the functioning of the protocol.

Once the detection of the disconnection of the secondary display lOb has taken place, then the method continues at step S4, which comprises storing the second image B that was displayed by the secondary display lOb as a virtual desktop 12. The primary display device lOa continues to display the first image A, until at step S5 there is received a user input indicating a display switch, and the method continues with step S6 of displaying the second image B on the primary display lOa. The first image A is then likewise stored as a virtual desktop 12, and the user can toggle between the two images A and B. The method preferably also further comprises detecting that the secondary display lOb has been reconnected, at some future point, in which case the process continues by displaying the second image B back on the secondary display lOb and displaying the first image A back on the primary display lOa. This process of disconnecting and reconnecting the secondary display lOa can continue, with the latest image B being displayed on the secondary display lOa being stored as a virtual desktop 12 when the display lOb is disconnected and then restored to the display lOb when the display 1 Ob is reconnected. Any changes in the image B that have occurred while the display lOb is disconnected are maintained when that second image B is restored to the secondary display lOb.

Preferably, the method also further comprises receiving a second user input indicating an overall view, and thereby generating a third image from the first and second images and displaying this third image on the primary display lOa. In a first embodiment, this third image will comprise the first and second images A and B combined so that each is of substantially the same size. In a second embodiment, the third image will comprise the first and second images, though in this case, one image will be of substantially the full image size and the other image will be of a smaller size, overlaying the one. Once the user has disconnected the secondary display lOb, then a virtual desktop 12 is created which the user can switch to the primary display lOa. However, the user may still wish to see all of the information contained in both images A and B, and therefore a third image can be created which the user can see as an alternative to the simple switching between the two images A and B. Although the operation of the system has been described above with respect to a two display system, the system can be extended to include more than two displays. With a plurality of "secondary" displays, each time such a display 10 is disconnected, its current image is converted to a virtual desktop 12 and stored for access by the user. This can result in multiple virtual desktops 12 being stored at any one time by the display controller 16 if more than one "secondary" display has been disconnected. In this case, the predefined user input can be used to cycle through all of the current stored virtual desktops 12. When a display 10 is reconnected, then the correct virtual desktop 12 will be used as the new image to be displayed by the respective display 10.

Claims

CLAIMS1 A method of operating a display system, the display system comprising a display controller, a primary display and a secondary display, the method comprising the steps of: o displaying a first image on the primary display, o displaying a second image on the secondary display, o detecting that the secondary display has been disconnected, o storing the second image as a virtual desktop, a receiving a user input indicating a display switch, and o displaying the second image on the primary display.
2. A method according to claim 1, and further comprising detecting that the secondary display has been reconnected, displaying the second image on the secondary display and displaying the first image on the primary display.
3. A method according to claim 1 or 2, and further comprising receiving a second user input indicating an overall view, generating a third image from the first and second images and displaying the third image on the primary display.
4. A method according to claim 3, wherein the third image comprises the first and second images each of substantially the same size.
5. A method according to claim 3, wherein the third image comprises the first and second images, one of substantially the full image size and the other of smaller size, overlaying the one.
6. A display system comprising a display controller, a primary display and a secondary display, wherein a the primary display is arranged to display a first image, o the secondary display is arranged to display a second image, o the display controller is arranged to detect that the secondary display has been disconnected, to store the second image as a virtual desktop and receive a user input indicating a display switch, and o the primary display is accordingly arranged to display the second image.
7. A system according to claim 6, wherein the display controller is further arranged to detect that the secondary display has been reconnected, the secondary display is accordingly arranged to display the second image and the primary display is accordingly arranged to display the first image.
8. A system according to claim 6 or 7, wherein the display controller is further arranged to receive a second user input indicating an overall view and to generate a third image from the first and second images, the primary display accordingly arranged to display the third image.
9. A system according to claim 8, wherein the third image comprises the first and second images each of substantially the same size.
10. A system according to claim 8, wherein the third image comprises the first and second images, one of substantially the full image size and the other of smaller size, overlaying the one.
11. A computer program product on a computer readable medium for operating a display system, the display system comprising a display controller, a primary display and a secondary display, the product comprising instructions for: o displaying a first image on the primary display, o displaying a second image on the secondary display, o detecting that the secondary display has been disconnected, o storing the second image as a virtual desktop, o receiving a user input indicating a display switch, and o displaying the second image on the primary display.
12. A computer program product according to claim 11, and further comprising instructions for detecting that the secondary display has been reconnected, displaying the second image on the secondary display and displaying the first image on the primary display.
13. A computer program product according to claim 11 or 12, and further comprising instructions for receiving a second user input indicating an overall view, generating a third image from the first and second images and displaying the third image on the primary display.
14. A computer program product according to claim 13, wherein the third image comprises the first and second images each of substantially the same size.
15. A computer program product according to claim 13, wherein the third image comprises the first and second images, one of substantially the full image size and the other of smaller size, overlaying the one.