WO2014045079A1

WO2014045079A1 - A method of 3d user interface for mobile devices

Info

Publication number: WO2014045079A1
Application number: PCT/IB2012/054931
Authority: WO
Inventors: Sagara WIJETUNGA
Original assignee: Wijetunga Sagara
Priority date: 2012-09-18
Filing date: 2012-09-18
Publication date: 2014-03-27

Abstract

Invention disclosed is a novel way of display, browse, execute and rearrange operating system homepage short-cuts, applications and folders, of the file system hierarchy of a Mobile device, as three-dimensional ("3D") icon objects in 3D space. It presents operating system homepage short-cuts, applications and folders, as 3D icon objects on a 3D long run-time-extensible virtual icon objects box. If an icon object represents a folder is executed and that folder contains only operating system homepage short-cuts, applications and/or sub folders, the long 3D icon objects box is extended, and operating system homepage short-cuts, applications and/or sub folders of the executed folder, as 3D icon objects, are placed on the long 3D icon objects box as a separate icon objects set after the existing icon objects set. If an icon object represents a folder is executed and that folder contains only files other than operating system homepage short-cuts, applications and/or sub folders, a file browsing application is launched for those files. Invention disclosed here can generate visually appealing and useful three-dimensional ("3D") graphical user interfaces on Mobile devices, by processing data from various orientation sensors such as Accelerometer, Gyroscope, digital compass, etc. can offer a way to browse applications selections in 3D without touching the display; and by making pre-determined gestures at a distance without touching the display or by making pre- determined physical movements, one can select an application or folder item and launch it without touching the display at all. Further on Mobile devices it is possible to display an icon objects set in landscape mode or portrait mode based on the orientation of the Mobile device.

Description

A METHOD OF 3D USER INTERFACE FOR MOBILE DEVICES

DESCRIPTION

The invention described here, by way of example only, with reference to the accompanying drawings. In th regard, no attempt is made to show structural details of the invention in more detail than is necessary for fundamental understanding of the invention.

[Dl] FIELD AND BACKGROUND OF THE INVENTION

This invention is generally in the field of computer generated three-dimensional ("3D") graphical user interfaces.

Present method of graphical user interfaces in Mobile devices including Tablet Computers, Smart phones, etc. are mostly in two-dimensional ("2D"). FIG. 1 shows the present method. There are instances some have demonstrated 3D graphical user interfaces such as rotating boxes, multi-facet cylindrical structures where user can select a facet to launch an application.

Personal Computers including laptop and notebook computers already come with powerful multi-core Central Processing Units ("CPUs") and Graphical Processing Units ("GPUs"), many vendors offer a choice of CPU and GPU, therefore, sophisticated three-dimensional ("3D") graphical user interfaces are now feasible on Personal Computers including laptop and notebook computers.

Mobile devices are fast catching up with their processing capabilities with powerful yet energy efficient multi- core Central Processing Units ("CPUs"), Graphical Processing Units ("GPUs"), and even equip with Physics Processing Units ("PPUs"), and further, equip with various sensors such as Accelerometer, Gyroscope, Proximity Sensor Arrays, light sensors, digital compass, etc., therefore, much richer three-dimensional ("3D") graphical user interfaces mimicking real world physics and gravity, are now feasible on Mobile devices than Personal Computers including laptop and notebook computers.

[D2] SUMMARY OF THE INVENTION

Invention disclosed is a novel way of display, browse, execute and rearrange operating system homepage short-cuts, applications and folders, of the file system hierarchy of a Mobile device, as three-dimensional ("3D") icon objects (200) in 3D space (202). It presents operating system homepage short-cuts, applications and folders, as 3D icon objects (200) on a 3D long run-time-extensible virtual icon objects box (201). FIG. 2 shows this invention.

If an icon object (200) represents a folder is executed and that folder contains only operating system homepage short-cuts, applications and/or sub folders, the long 3D icon objects box (201) is extended, and operating system homepage short-cuts, applications and/or sub folders of the executed folder, as 3D icon objects (200), are placed on the long 3D icon objects box (201) as a separate icon objects set (300) after the existing icon objects set (300). FIG. 3 shows icon objects set (300) in the 3D long run-time-extensible virtual icon objects box (201).

If an icon object (200) represents a folder is executed and that folder contains only files other than operating system homepage short-cuts, applications and/or sub folders, a file browsing application is launched for those files.

Invention disclosed here can generate visually appealing and useful three-dimensional ("3D") graphical user interfaces on Mobile devices, by processing data from various orientation sensors can offer a way to browse applications selections in 3D without touching the display; and by making pre-determined gestures at a distance without touching the display or by making pre-determined physical movements, one can select an application or folder and launch it without touching the display at all.

Further on Mobile devices it is possible to display an icon objects set (300) in landscape mode or portrait mode based on the orientation of the Mobile device. FIG. 4 shows icon objects set (300) in landscape mode or portrait mode.

As per the innovation disclosed, thus enabling to browse traditional menu hierarchies in 3D space (202) as icon objects sets (300) and launch applications without touching the display at all is very useful in certain scenarios such as operating a Mobile device while stand and travel in a bus or train, cycling, sailing, etc., where one has to hold on to something for support, thereby, making only one hand available to hold the Mobile device.

[D3] BRIEF DESCRIPTION OF THE DRAWINGS

With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention, and are presented in the context of the most useful and readily understood description of the principles and conceptual aspects of the invention.

FIG. 1 is a diagrammatic representation of the present user interface, in general, of a mobile device. FIG. 1A shows a tablet computer, and FIG. IB shows a mobile phone. It is provided for the purpose of readily visually show the difference of the invention disclosed here compared to existing methods.

FIG. 2 is a diagrammatic representation of the innovation disclosed.

FIG. 3 is a diagrammatic representation of 3D icon objects sets (300) on the long 3D icon objects box (201).

FIG. 4 is a diagrammatic representation of icon objects set (300) in landscape mode or portrait mode.

FIG. 5 is a diagrammatic representation of the long 3D icon objects box (201) can be moved within the 3D space (202) in and out, by tilt the device or otherwise, on the X axis, to see 3D icon objects (200) nearer or bring not visible 3D icon objects (200) into visibility.

FIG. 6 is a diagrammatic representation of rearrange 3D icon objects (200). FIG. 6A shows rearrange 3D icon objects (200) by shake the mobile device. FIG. 6B shows rearrange 3D icon objects (200) by toss the mobile device.

[D4] DETAILED DESCRIPTION

[D4.1] Core Innovation

Invention disclosed is a novel way of display, browse, execute and rearrange operating system homepage short-cuts, applications and folders, of the file system hierarchy of a mobile device, as three-dimensional ("3D") icon objects (200) on a 3D long run-time-extensible virtual icon objects box (201) in 3D space (202). FIG. 2 shows this invention.

The 3D long run-time-extensible virtual icon objects box (201) comprises four walls, a floor and a ceiling. 3D icon objects (200) or 3D icon objects sets (300) are placed either on the floor or a distance above the floor of this long 3D icon objects box (201).

The long 3D icon objects box (201) can be moved within the 3D space (202) in and out , by tilt the device or otherwise, on the X axis, to see 3D icon objects (200) nearer or bring not visible 3D icon objects (200) into visibility (FIG. 5); shake (FIG. 6A) or toss (FIG. 6B) to rearrange 3D icon objects (200).

[D4.2] Method of display

Walls, floor and ceiling of the long 3D icon objects box (201) may display visible, not visible, partially visible or semi-transparent; Operating systems homepage short-cuts, applications and folders, are presented as 3D icon objects (200) in shapes such as small cubes, flat chips, coins, spherical, etc. or as a combination. (FIG. 2)

Further, icon objects (200) may arrange spatially apart as if similar magnet poles reject each other, so that icon objects (200) always keep equal distance to each other from all sides. Due to various gestures or physical movements, icon objects (200) may move, but icon objects (200) soon resettle to equal distance to each other from all sides and also keep the same distance to the floor and the ceiling.

Further, icon objects (200) may place a distance above the floor.

icon objects (200) are displayed as one icon object (200) (eg. First row center) is pre-selected (203), therefore, the pre-selected icon object (203) is visually highlighted by change of intensity of colour, shadow or animation.

The 3D Display environment can be 3D rendered into 2D to be displayed on the physical display of the mobile device or the 3D Display environment can be rendered into stereoscopic display as left and right views.

[D4.3] Method of browse

The whole 3D long run-time-extensible virtual icon objects box (201) is browsable in 3D space (202) by various means; Mobile devices, may use a stylus or one or more fingers (if the display is touch sensitive) or tilt the device or make other physical movements (if the device is equipped with various orientation sensors) to browse the long 3D icon objects box (201).

This icon object (200) pre-selection could be moved to left or right icon objects (200) by, for a example, a minor shake on the Mobile device counterclockwise or clockwise respectively; This icon object (200) selection could be moved up or down by another physical movement such as minor shake on the Mobile device on the X axis, top-side (eg. On a mobile phone, the speaker-side) down or bottom-side ((eg. On a mobile phone, the microphone-side) down.

If an icon object (200) of an icon objects set (300) on the long 3D icon objects box (201) is executed, icon objects sets (300) after the newly focused icon objects set (300), will be closed and the length of the long 3D icon objects box (201) will be reduced to fit the new length.

[D4.4] Method of execute

If an icon object (200) is clicked, touched, or executed by other means such as a pre-determined gesture at a distance without touching the display or pre-determined physical movement of the Mobile device, the action associated with the icon object (200) is executed, after a quick animation and/or light up the icon object (200) and also with or without playing a sound or vibration or both.

If an icon object (200) represents a folder is executed and that folder contains only operating system homepage short-cuts, and/or applications and/or sub folders, the long 3D icon objects box (201) is extended, and operating system homepage short-cuts, and/or applications and/or sub folders of the executed folder, as 3D icon objects (200), are placed on the long 3D icon objects box (201) as a separate icon objects set (300) after the existing icon objects set (300).

If a further sub folder icon object (200) is executed from the new icon objects set (300), and that sub folder contains only operating system homepage short-cuts, and/or applications and/or further sub folders, the long 3D icon objects box (201) is further extended and operating system homepage short-cuts, and/or applications and/or sub folders of the executed sub folder, as 3D icon objects (200), are placed on the long 3D icon objects box (201) as a separate set after the new icon objects set (300). Further hierarchical sub folder opening will similarly follow.

[D4.5] Method of rearrange

icon objects (200) in an icon objects set (300) may not be in any particular order. It can be reshuffled or rearranged by various means.

Mobile devices (if the device is equipped with various orientation sensors) may toss, shake or other predetermined physical action, to reshuffle or rearrange the icon objects (200) in the long 3D icon objects box (201).

Once the long 3D icon objects box (201) is shaken either by toss, arbitrary physical shake or as predetermined physical action, it is simulated that icon objects (200) throw off and knock on to walls, floor, ceiling of the long 3D icon objects box (201) and/or knock on to each other and bounce back and resettled as a different icon object (200) arrangement. FIG. 6A shows rearrangement of icon objects (200) by shaking the mobile device and FIG. 6B shows rearrangement of icon objects (200) by tossing the mobile device.

By physical movement of shaking or tossing the mobile device, thereby shaking or tossing the long 3D icon objects box (201), icon objects (200) get reshuffled or rearranged only within its icon objects set (300). Icon objects (200) may directly drag and drop to move to a new icon objects set (300); An icon object (200) may drag out of its icon objects set (300) and keep it out of all icon objects sets (300) and subsequently drop to an new icon objects set (300) as a way to move to a new icon objects set (300). Item 301 of FIG. 3 shows two icon objects (200) dragged out of its icon objects set (300) and keep it out of all icon objects sets (300).

[D5] DETAILS OF CARRY OUT THE INNOVATION

On Mobile devices, the 3D display environment could be displayed using one or more Graphics Processing Units ("GPUs"), a 3D graphics rendering engine, such as the Object-oriented Graphics Rendering Engine ("OGRE") and a Real-time Physics Engine.

One or more multi-core Central Processing Units ("CPUs") and hardware accelerated Real-time Physics Engine will render the 3D display environment smoothly and handles physics simulations smoothly.

[D5.1] Implementation on Mobile devices

Various sensors such as Accelerometer, Gyroscope, digital compass, Proximity Sensor Arrays, etc. are expected on board of a Mobile device to detect orientation, physical movements including shaking, tossing, gestures, etc.

Mobile devices, may use a stylus or one or more fingers (if the display is touch sensitive) or tilt the device or make other physical movements (if the device is equipped with various orientation sensors) to browse the long 3D icon objects box (201).

Method of display

The method of display is as per [D4.2] Method of display.

Method of browse

The Method of browse the long 3D icon objects box (201) is as per [D4.3] Method of browse.

Method of execute

The Method of execute of an icon object (200) is as per [D4.4] Method of execute.

Method of rearrange

The method of rearrange icon objects (200) as per [D4.5] Method of rearrange.

Operating system homepage short-cuts refers to links to applications.

Orientation sensors include, but not limited to: Accelerometer, Gyroscope and digital compass.

Mobile devices include, but not limited to: tablet computers, smart phones, personal digital assistants, network appliances, digital cameras, network base stations, media players, navigation devices, game consoles, remote controllers, radios, mobile televisions, or combinations.

File browsing application: A file browsing application receives files to be displayed. Files may be displayed either 2D or 3D with optional graphical effects. Once the user selects a file, either by click, touch, gesture or physical movement, the file browsing application displays/renders the file contents if the file browsing application capable of display/ render that file type or launch an other application to display or render the file contents if the file browsing application not capable of display/ render that file type.

The description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be carry out in practice.

Claims

A METHOD OF 3D USER INTERFACE FOR MOBILE DEVICES

1. A method of display, browse, execute and rearrange operating system homepage short-cuts,

applications and folders, of the file system hierarchy of a Mobile device, as three-dimensional ("3D") icon objects (200) on a 3D long run-time-extensible virtual icon objects box (201) in 3D space (202).

2. The method according to Claim 1, wherein, comprising four further methods: (1) A method of display, (2) method of browse, (3) method of execute and (4) method of rearrange.

3. A method of display according to Claim 2, wherein,

Operating system homepage short-cuts, applications and sub folders of a folder are presented as 3D icon objects (200) on a 3D long run-time-extensible virtual icon objects box (201) in 3D space (202); Icon objects (200) are moveable, roll-able and turn-able.

4. The method of Claim 3, further comprising,

The method of Claim 3, further comprising,

The long 3D icon objects box (201) is moveable within the 3D space (202) in and out, by various means including pre-determined physical movement of the mobile device or pre-determined gestu without touching the display.

6. The method of Claim 3, further comprising,

Spacial arrangement of icon objects (200) are simulated to automatically arrange spatially apart, as if similar magnet poles reject each other, so that icon objects (200) always keep equal distance to each other from all sides after any movement.

7. The method of Claim 3, further comprising,

icon objects (200) are displayed as one icon object (200) is pre-selected, therefore, the pre-selected icon object (203) is visually highlighted by change of size, intensity of colour, shadow or animation.

8. A method of browse according to Claim 2, wherein,

The long 3D icon objects box (201) is browsable by moving in and out in 3D space (202) by various means including pre-determined gestures at a distance without touching the display or predetermined physical movements of the Mobile device identified by processing data from various onboard orientation sensors.

9. The method of Claim 8, further comprising,

Icon object (200) pre-selection could be moved to left, right, up and down to select different icon objects (200) by various means including pre-determined gestures at a distance without touching the display or pre-determined physical movements of the Mobile device.

10. The method of Claim 8, further comprising,

11. A method of execute according to Claim 2, wherein,

If an icon object (200) is clicked, touched, or executed by other means such as a pre-determined gesture at a distance without touching the display or pre-determined physical movement of a Mobile device, the action associated with the icon object (200) is executed, after a quick animation and/or light up the icon object (200) and also with or without with playing a sound or vibration or both.

12. The method of Claim 11, further comprising,

If an icon object (200) represents a folder is executed and that folder contains only operating system homepage short-cuts, and/or applications and/or sub folders, the long 3D icon objects box (201) is extended, and operating system homepage short-cuts, and/or applications and/or sub folders of the executed folder, as 3D icon objects (200), are placed on the long 3D icon objects box (201) as a separate icon objects set (300) after the existing icon objects set (300); If a further sub folder icon object (200) is executed from the new icon objects set (300), and that sub folder contains only operating system homepage short-cuts, and/or applications and/or further sub folders, the long 3D icon objects box (201) is further extended and operating system homepage short-cuts, and/or applications and/or sub folders of the executed sub folder, as 3D icon objects (200), are placed on the long 3D icon objects box (201) as a separate set after the new icon objects set (300). Further hierarchical sub folder opening will similarly follow.

13. The method of Claim 11, further comprising,

If an icon object (200) represents a folder is executed and that folder contains only files other than operating system homepage short-cuts, applications and sub folders, a file browsing application is launched for those files.

14. A method of rearrange according to Claim 2, wherein,

Mobile devices may toss, shake or make other pre-determined physical actions, to reshuffle or rearrange the icon objects (200); Once the long 3D icon objects box (201) is shaken either by toss, arbitrary physical shake or as pre-determined physical action, it is simulated that icon objects (200) throw off and knock on to walls, floor, ceiling of the long 3D icon objects box (201) and/or knock on to each other and bounce back and resettled as a different icon object (200) arrangement; By shaking the long 3D icon objects box (201), icon objects (200) get reshuffled or rearranged only within its icon objects set (300).

15. The method of Claim 14, further comprising,

Icon objects (200) may directly drag and drop to move to a new location within the current icon objects set (300) or to a new icon objects set (300); An icon object (200) may drag out of its icon objects set (300) and keep it out of all icon objects sets (300) and subsequently drop to an new icon objects set (300) as a way to move to a new icon objects set (300).