Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
  • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Definitions

• the present invention is related to a user interface for displaying information on a computing device.
• FIG. 1 Various data is collected and processed during the operation of computing devices such as desktop computers, laptop computers, on-board telematics devices in cars, mobile devices (e.g., smartphones) and consoles.
• computing devices such as desktop computers, laptop computers, on-board telematics devices in cars, mobile devices (e.g., smartphones) and consoles.
• the information is presented to users in the form of windows that are displayed on a defined area of a display device.
• certain windows may be enlarged, reduced in size or moved to facilitate the users' operations.
• various data associated with the operation or control of the robot may be displayed on a display device.
• the various data displayed may include, for example, signals from sensors, angles of one or more joints, location of objects surrounding the robot, remaining computing or storage resources on the robot.
• Such data may be transmitted from the robot to a computing device located remotely from the robot where a user may view and take actions as needed.
• a single window allows a user to view certain information and perform predefined functions on the computing device.
• additional windows may need to be launched or activated on the computing device. For this and other reasons, users often launch multiple windows on display devices.
• the user may have a difficult time identifying and tracking information relevant to the user.
• the user may close or reduce the size of windows displaying less important information to focus on more windows that display more important information.
• the closing or reducing the size of window may involve user actions that are neither intuitive nor convenient.
• Embodiments relate to displaying data on a screen where a window is reduced in size by rotation in response to receiving user input to make space for other windows on the screen.
• Data processed at a computing device is displayed within an area of the screen defined by the window.
• the window is moved to a predefined region of the screen after receiving first user input.
• the window is rotated about an axis in response to receiving second user input after the window reaches the predefined region of the screen.
• the size of the window is reduced by the rotation of the window.
• the window is reduced into an icon in response to receiving third user input after the window is rotated to a predetermined angle.
• the first user input, the second user input and the third user input are caused by dragging a user input device in the same direction.
• the predefined region of the screen includes edge regions of the screen.
• the displayed data includes data associated with the operation of a robot.
• FIG. 1 is a schematic diagram of a robot and a remote computer communicating with the robot, according to one embodiment.
• FIG. 2 is a block diagram of the remote computer, according to one embodiment.
• FIG. 3 is a block diagram of software components stored in the memory of the remote computer, according to one embodiment.
• FIGS. 4A through 4C are diagrams illustrating transition of a fold-away window on a screen responsive to receiving a user input moving the window to the left edge of the screen, according to one embodiment.
• FIGS. 5A through 5C are diagrams illustrating transition of a fold-away window on a screen responsive to receiving a user input moving the window to the bottom edge of the screen, according to one embodiment.
• FIG. 6 is a flowchart illustrating a process of reducing the size of a window, according to one embodiment.
• Certain aspects of the embodiments include process steps and instructions described herein in the form of an algorithm. It should be noted that the process steps and instructions could be embodied in software, firmware or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by a variety of operating systems.
• Embodiments also relates to an apparatus for performing the operations herein.
• This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer.
• a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.
• the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.
• Embodiments relate to providing a user interface screen for displaying data associated with processing at a computing device where the user interface screen includes one or more windows that can be rotated and then minimized into an icon to render space for other windows.
• the window moves to an edge area of the screen.
• the window is rotated about an axis and then minimized into an icon. In this way, the windows presented on the screen can be intuitively reduced in size by a user.
• a "window" refers to a defined region on a screen for displaying images.
• the window is typically in the form of a rectangle that can be increased or decreased in size.
• a window may take up the entire region of the screen or part of the screen.
• embodiments are described below with reference to a computing device that controls or monitors the operation of a robot.
• the reference to the embodiments related to the operation of the robot is merely examples, and other embodiments may be used for other types of operations such as presenting other types of data not related to the operation of a robot.
• other embodiments may be related to presenting contact information or initializing communication or surfing the Internet using a mobile computing device (e.g., a smartphone).
• FIG. 1 is a schematic diagram of a robot 100 and a remote computer 150 communicating with the robot 100, according to one embodiment.
• the robot 100 may include, among other components, a plurality of body parts, actuators for causing relative movements between the body parts, a local computer 140, sensors and output devices (e.g., speaker).
• the plurality of body parts may include, for example, arms, hands, torso, head, legs and feet. The relative movements of these body parts are caused by actuators such as motors.
• the sensors may be attached to the body parts to sense the pose of the robot 100 as well as to capture visual images or acoustic signals.
• the local computer 140 is hardware, software, firmware or a combination thereof for processing sensor signals and other input commands, generating actuator signals, and communicating with other computing devices.
• the local computer 140 communicates with the remote computer 150 via a channel 152 to send data to or receive data from the remote computer 150.
• the channel 152 may be embodied using wired or wireless technology.
• the remote computer 150 is used by a user to gather information about operations of the robot 100 and/or provide instructions to the robot 100.
• the remote computer 150 may receive raw data or processed data from the robot 100 via the channel 152.
• the data transmitted over the channel 152 may include, among other data, stream of images captured by one or more cameras installed on the robot 100, sensor signals, coordinates and identities of objects detected around the robot 100, audio signals captured by microphones installed on the robot 100, and instructions to perform certain operations on the robot 100.
• FIG. 1 illustrates a humanoid form
• embodiments may be used in robots of various other configurations.
• the robot may be an industrial robot with a single arm configuration.
• FIG. 2 is a block diagram of the remote computer 150, according to one embodiment.
• the remote computer 150 may include, among other components, a processor 214, a display interface 218, a screen 220, an input interface 222, memory 230, a networking interface 234 and a bus 242 connecting these components.
• the remote computer 150 may include other components not illustrated in FIG. 2.
• the processor 214 is a hardware component that reads and executes instructions, and outputs processed data as a result of the execution of the instructions.
• the processor 214 may include more than one processing core to increase the capacity and speed of data processing.
• the display interface 218 is a hardware component for generating signals to display images on the screen 220 of the remote computer 150.
• the display interface 218 generates the signals according to instructions modules on the memory 230.
• the display interface 218 is a video card.
• the input interface 222 is a component that interfaces with user input devices such as mouse, keyboard and touchpad.
• the input interface 222 may be embodied as a combination of hardware, software and firmware for recognizing verbal commands issued by a user.
• the memory 230 is a computer-readable storage medium storing instruction modules and/or data for performing data processing operations at the processor 214. The details of instructions modules in the memory 230 are described below with reference to FIG. 3.
• the networking interface 234 establishes the channel 152 with the robot 100.
• the networking interface 234 may control transmission of data over the channel 152 using protocols such as IEEE 1394, Wi-Fi, Blootooth, and Universal Serial Bus (USB).
• protocols such as IEEE 1394, Wi-Fi, Blootooth, and Universal Serial Bus (USB).
• FIG. 3 is a block diagram illustrating software components of the remote computer 150, according to one embodiment.
• One or more software components illustrated in FIG. 3 may also be embodied as dedicated hardware components or firmware.
• the memory 230 may store, among other software components, an operating system 310, a middleware 320, and a plurality of application 330A through 330N (hereinafter collectively referred to as "the applications 330").
• the memory 230 may include multiple memory devices that can collectively store one or more software components illustrated in FIG. 3.
• the operating system 310 manages resources of the remote computer 150 and provides common services for the applications 330.
• the operating system 310 may include, among others, LINUX, UNIX, MICROSOFT WINDOWS, IOS, MAC OSX and ANDROID.
• the middleware 320 provides library and functions for some or all of the applications 330.
• the middleware 320 may include, among other instruction modules, a window manager 318 and an input handler 324.
• the window manager 318 manages one or more windows displayed on the screen 220.
• the window manager 318 provides library and functions that enable the applications 330 to create, move, modify or remove one or more windows on the screen 220. In one
• the window manager 318 enables the windows to be rotated and iconified in response to receiving user inputs, as described below in detail with reference to FIGS. 4 A through 4C.
• the input handler 324 receives user input from the user interface devices (e.g., mouse, keyboard and touchscreen) via the input interface 222, processes the user input and provides processed signals to the applications 330 and/or the window manager 318 for further operations based on the user input.
• the user interface devices e.g., mouse, keyboard and touchscreen
• Each of the applications 330 communicates data from the robot 100 via the channel 152 and some of these applications 330 render images for display on the screen 220 using the window manager 318.
• the applications 330 may also perform computing operations (e.g., trajectory planning) separate from or in conjunction with the local computer 140.
• the applications 330 may use the libraries and functions available from the middleware 320 such as the window manager 318 and the input handler 324 to perform their operations.
• Example applications 330 include the following: (i) a 3D scene geometry management application for loading geometric models and creating instances of geometric models based on events detected at the sensors of the robot 100, (ii) a videostream application for storing and/or displaying videostream from a camera mounted on the robot or stored in a file, (iii) a panoramic attention application for mapping objects to coordinates around the robot 100 and creating a panoramic display including the mapped objects, (iv) an instruction application for sending high level commands to the robot 100, (v) a plotting application for plotting streams of data associated with the operation of the robot 100 and (vi) a logger application that intercepts messages from the middleware 320 and logs the time at which an event associated with the messages occurred.
• the middleware 320 provides functions and libraries for reusable and extensible set of primitives that enable applications 330 to drawing images on the screen 220.
• various applications 330 can be programmed easily into a compact form.
• the primitives may also be used as a basis for extending functionality of the applications 330 through dynamic plug -ins.
• the use of dynamic plug-ins reduces the need to re-compile or modify existing applications 330.
• FIGS. 4A through 4C are diagrams illustrating the transition of a fold-away window 418A on a screen 410 responsive to receiving a user input moving the window 418A to the left edge of the screen 410, according to one embodiment.
• a user may decide to reduce the size of a window.
• the window manager 318 provides a way of rotating the window or iconifying the window in an intuitive manner.
• FIG. 4A illustrates the screen 410 where two windows 418 A and 414 are displayed in an overlapping manner.
• the user provides input (e.g., mouse input selecting the window 418A and dragging the mouse in the left direction) to move the window 418A in the left direction (shown by an arrow) to clear the screen 410 for display of the window 414, the window 418A moves toward the left edge of the screen 410 in a flat state.
• a flat state refers to the state of the window that is not rotated or iconified.
• the window 418B (corresponding to the window 418A) is rotated about an axis 420 as user input (e.g., dragging of the mouse in the left direction) in the direction of the arrow of FIG. 4B is received. That is, a virtual plane for projecting the window 418A is rotated about the axis 420, giving three-dimensional perception that the virtual plane and the window 418B is facing towards the right- front side of the screen 410.
• the window 414B takes up less space in the screen 410 and is less likely to obstruct the window 414.
• window 418B While the window 418B remains rotated with where the window 418B facing the right-front side of the screen 410, images may continue to be displayed on the window 418B . Moreover, other user interface elements (e.g., icons, controls and menus) of the window 418B remain operable in the slated position, allowing the user to take actions needed without having to expand the window 418B.
• other user interface elements e.g., icons, controls and menus
• an edge of the window 418B maintains its position while the window 418B is rotated.
• the left edge 411 of the window 418B remains in the same state while the right edge 413 of the window 418B moves progressively to the left, reducing the overall size of the window 418B with the rotation of the window 418B.
• the user may leave the window 418B in such a rotated position to view relevant data from the window 418B while focusing on images displayed on the window 414.
• the user may further reduce the window 418B into an icon 418C by continuing to provide the same user input (e.g., dragging the mouse in the left direction) after the window 418B is rotated along the axis 420 beyond a certain angle (e.g., 45 degrees).
• a certain angle e.g. 45 degrees.
• the angle at which the window 418B iconifies depends on the configuration of the user interface elements in the window 418B. If the size of the user interface elements in the window 418B is small, the window 418B may be iconified even when the window 418B is rotated for a small angle.
• the window 418B may be iconfied when the window 418B is rotated to a larger angle since the user may operate on the user interface elements at a large rotation angle. By iconifying the window, more space becomes available to display information from other windows or user interface elements.
• the icon 418C can be enlarged into the rotated window 418B or a flat window 418A by providing predetermined user input (e.g., double- clicking of the icon 418C).
• Two or more icons 418C can also be tiled on the screen 410 to facilitate the user to find and enlarge the relevant icons into windows.
• FIGS. 5A through 5C are diagrams illustrating the transition of a fold-away window 518 A on a screen responsive to receiving a user input moving the window to the bottom edge of the screen 510, according to one embodiment.
• the window 518A moves toward the bottom edge of the screen 510 in a flat state.
• the window 518B (corresponding to the window 518A) is rotated about an axis 520 as user input (e.g., dragging of the mouse in the bottom direction) is received from the user via the input handler 324.
• the window 518B may be reduced into an icon 518C by continuing to provide the same user input (e.g., dragging the mouse in the bottom direction) after the window 518B is rotated along the axis 520 beyond a certain angle (e.g., 45 degrees).
• a certain angle e.g. 45 degrees
• FIGS . 4 A through 5 C illustrate the rotation of the window 418B relative to a vertical axis 420 or a horizontal axis 520 after being moved to the left or bottom edge of the screen
• the same window may be rotated related to a vertical axis or a horizontal axis after the window 418B is moved to an upper edge or right edge of the screen.
• the user input causing the rotation of the window or iconification of the window may be different based on the type of input devices used to operate the remote computer 150.
• a pointing device such as a mouse
• clicking of the window followed by a translational (i.e., dragging) motion may cause the window to move to an edge of the screen followed by the rotation of the window and iconification of the window.
• a first double-clicking of the window may cause the window to rotate about the axis and a second double-clicking of the same window may cause iconification of the window.
• the scrolling action on the window may cause the window to move to an edge followed by rotation and iconification of the window.
• one or more of the windows may be semi-transparent in its flat state or in a rotated position.
• the semi-transparent windows may enable the users to view the images in other windows or screen that is obstructed by the semi- transparent window while also enabling the user to view the data displayed on the semi-transparent window.
• user input e.g., scrolling of a mouse wheel
• FIG. 6 is a flowchart illustrating a process of reducing the size of a window, according to one embodiment.
• the remote computer 150 receives 606 user input via a user input device to make translational movement of the window.
• the window moves 610 to an edge of the screen 410 (e.g., the left edge of the screen 410 as shown in FIG. 4B).
• the remote computer 150 continues 614 to receive the same user input after the window reaches the edge of the screen 410.
• the window is rotated 618 about an axis (e.g., a vertical axis 420). By rotating the window, other windows on the screen 410 may become unobstructed by the rotated window and may become visible to the user.
• the window is iconified 628.
• the iconified window takes up less space on the screen 410 and makes the remaining space available for other windows or user interface elements.
• fold- away windows may be used for displaying images associated with other applications such as web browsers, word processors and spreadsheets.

Landscapes

• Engineering & Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• User Interface Of Digital Computer (AREA)

Priority Applications (1)

Applications Claiming Priority (4)

Publications (1)

Family

ID=47292838

Family Applications (1)

Country Status (3)

Cited By (1)

Families Citing this family (1)

Citations (7)

Family Cites Families (24)

• 2012
  • 2012-05-01 US US13/461,598 patent/US20120314020A1/en not_active Abandoned
  • 2012-06-12 WO PCT/US2012/042096 patent/WO2012174016A1/en not_active Ceased
  • 2012-06-12 JP JP2014515923A patent/JP2014522541A/ja active Pending

Patent Citations (7)

Also Published As

Similar Documents

Legal Events