GB2354415A - Active graphics toolkit - Google Patents

Abstract

In known graphics toolkits graphical manipulations are restricted to design time. At deploy time, after the application is compiled, the graphical elements are fixed and cannot be manipulated with respect to position or in relation to one another. There is described a method of building a graphical user interface comprising: providing a graphic element class for handling a plurality of user interaction types performed on an graphic element and having a graphic 38,42A,42B,42C for a particular resource type and a subset of the user interaction types particular to the resource type; providing an active content class 40,44A,44B,44C for communicating with that particular resource type and having an interface with the superclass graphic element; creating an active instance of the subclass active content and associating it with a resource; establishing a communication link between the active instance and the associated resource; creating a graphic instance associated with the active instance; whereby once an active instance is created user interactions can be communicated from the graphic instance to the active instance as events and mapped by the active instance into resource commands for the resource. The display (34) represents the system in graphical form, like a map that can be edited. Editing (32) the map (34) changes the system, and changes to the system result in a change to the map. The user can 'see the system' at a glance, rapidly absorbing changes, and 'intuitively' manipulating the system by manipulating the map. An 'action' desired by the user is 'OK'd' by policy 36. The required code to carry out said action is then activated and an associated graphic is displayed.

Description

2354415 uj,.v 9 9 0 8 8 ACTIVE GRAPHICS TOOLKIT

FIELD OF INVENTION

This invention relates to an active graphics toolkit. In particular it relates to a toolkit for providing graphic control of resources in a system.

BACKGROUND OF INVENTION

Since graphical representations of complex systems are more readily grasped by the human mind it useful to describe a complex system with a diagram in which a set of graphical elements, each representing a part of the system, is displayed on a computer monitor. The relative positions and connectivity of the graphical elements forms a map of the system, compressing a rich description into a relatively small space.

Systems of graphic elements with interconnections may be drawn on a diagram editor such as Lotus Freelance Graphics. In such a diagram editor, however, the graphical elements are not linked to any programmable content. The diagram, while it may represent a system, is not actively linked to the system. If the system changes, the diagram must be manually edited to match. If a change in the system is required, then the diagram and the system must be changed simultaneously.

The general usability of programs improved significantly when operating systems, such as IBM OS2 and Microsoft windows, were given a graphical user interface to resources in products. Resources can now be represented by graphical icons for selection and control using a mouse controlled graphical cursor. Controlling the resources with representations of buttons and tick boxes allows further improvements in usability. For example a graphical representation may be a folder or a shortcut or a file or a resource. Each graphic controls a resource using an active, part which is configurable to an extent limited by the operating system but not easily changed. However such graphical controls are defined by the operating system. If a new type of graphic and corresponding active, part needed to be introduced it would be difficult to implement on an existing GUI operating system. Hence system resources linked to graphical elements are known from GUI operating systems.

Designer, graphical user interfaces may be constructed in visual program design toolkits. one such tool is IBM VisualAge comprising 'Bean, program components. A Bean may be a graphical representation of a resource and an 'active, program part for controlling the resource. An application may be built by manipulating a 'resource' Bean on a visual form, associating it with other Beans and changing its properties with Ui,A 9 0 8 8 2 the use of graphical buttons, drop lists, check boxes and custom controls. Such a visual program is then compiled to produce a runnable object code program which may control the resource using a graphical interface. The layout and appearance of the user interface is then fixed, at compile time. The user cannot add or remove (create or delete) controls.

Another toolkit, Micrografx media manager, is a GUI diagram tool which manipulates more complex graphical elements to create 'executable, flow charts. In both these toolkits, however, graphical manipulations are restricted to design time. At deploy time, after the application is compiled, the graphical elements are fixed and cannot be manipulated with respect to position or in relation to one another (see Figure 1). The user is presented at runtime with a fixed set of controls to maintain/monitor a complex system. The text in the controls may change as the user changes the layout and configuration but the layout of the tool does not change.

SUMMARY OF INVENTION

According to one aspect of the invention there is provided a method of building a graphical user interface comprising: providing a graphic element class for handling a plurality of user interaction types performed on an graphic element and having a graphic for a particular resource type and a subset of the user interaction types particular to the resource type; providing an active content class for communicating with that particular resource type and having an interface with the superclass graphic element; creating an active instance of the subclass active content and associating it with a resource; establishing a communication link between the active instance and the associated resource; creating a graphic instance associated with the active instance; whereby once an active instance is created user interactions can be communicated from the graphic instance to the active instance as events and mapped by the active instance into resource commands for the resource.

This method is embodied in an active graphics application; in contrast to GUI builders it allows the graphics to be manipulated at run time by the end user. The display represents the system in graphical form like a map that can be edited. Editing the map changes the system and changes to the system result in a change to the map. The user can see the system' at a glance, rapidly absorbing changes, and intuitively, manipulating the system by manipulating the map. The embodiment is customisable so that it could represent a system of printers and print controllers, a system of queues and queue managers or a web site, but could just as easily represent any other system.

Ux,j99 088 3 Once the active instance is created an initiate setup message is sent to the resource to establish a communication link. In the present embodiment the active content will alert the print driver or in another embodiment the active content will contain the printer driver and will alert the printer hardware. The active instance preferably requests the resource to send resource output to the active instance so that in the present case the printer will be sent the name of its associated active content and can send printer error messages to the active content Preferably the application class hierarchy comprises a superclass graphic element for handling the plurality of user interaction types performed on an graphic element wherein said graphic element class is the subclass and a superclass active content comprising an interface with the superclass graphic element wherein said active content class is the subclass.

Active, diagrams unlike other diagrams are programmatically linked to the resources that they describe so that when one changes the diagram changes are made to the resources. Furthermore a change in the resources is reflected in changes to the diagram and the diagram and system are automatically kept synchronised. This in turn makes management, maintenance, and monitoring of the system more intuitive and less prone to error. The structure of the superclasses facilitates the creation of the primitives for such active diagrams by providing prewritten code in an 'easy to reuse' form. The prewritten objects can be viewed as tools, and the prewritten application structure as a framework in which they can be used. The paradigm resolves to two parts:

a graphical representation and an underlying business object. The separation of these two elements into two class hierarchies simplifies development of specialised behaviour by separating the graphical behaviour from the business logic.

The root of the graphics hierarchy is an abstract 'Graphic Element'. the root of the business object hierarchy is an abstract 'Active Content'. These abstract root classes interact with each other through a well defined interface. Subclasses of both, but especially of the Graphic Element, are provided to simplify the development of specialised behaviour. Graphic manipulation (drawing, moving, sizing connecting etc.) is especially amenable to abstraction as the logic is well defined and common to a number of systems. Developers of new diagram types will specialise subclasses in one or both of these hierarchies, and associate the specialised graphic with the specialised business object.

By abstracting out (providing in reusable form) the common and well-known graphical manipulation algorithms, the toolkit user (diagram style designer) can concentrate on developing the business objects (active content), rapidly develop new types of active diagram.

UZI., 9 9 0 8 8 4 According to a second aspect of the invention there is provided a toolkit for a graphical user interface comprising: a graphic element class for handling a plurality of user interaction types performed on an graphic element and having a graphic for a particular resource type and a subset of the user interaction types particular to the resource type; an active content class for communicating with that particular resource type and having an interface with the superclass graphic element; means for creating an active instance of the subclass active content and associating it with a resource; means for establishing a communication link between the active instance and the associated resource; means for creating a graphic instance associated with the active instance; whereby once an active instance is created user interactions can be communicated from the graphic instance to the active instance as events and mapped by the active instance into resource commands for the resource.

Diagrams might be developed to represent networks resources, messaging systems, web sites and so forth. The important common factor being the usefulness of representing connectivity and relationships in a graphical fashion. The graphical manipulations would then front for creation and deletion of resource, connecting and associating resource, editing and monitoring resource.

BRIEF DESCRIPTION OF DRAWINGS

In order to promote a fuller understanding of this and other aspects of the present invention, an embodiment will now be described, by way of example only, with reference to the accompanying drawings in which:

Figure 1 is a schematic representation of the components of the embodiment; Figure 2 is a schematic representation of the application of the embodiment; Figure 3 is a schematic showing the relationships between active contents and graphics for a printer and controller example; Figure 4 is a schematic flow diagram for setting up a connection between two resources; and Figure 5 is a schematic flow diagram for changing a connection between two resources.

U"99088 5 DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

The present invention is embodied in a graphics application 10 which may be loaded into Microsoft windows 95 operating system 12 running on a pentium. computer platform 14 (see Figure 1). The platform is a standard computer system including a main processor 16, memory 18, graphics processor 20, input and output (1/0) adapter 22, keyboard 24, mouse 26, and monitor 28. On power up the operating system 12 and then the graphics application 10 are loaded into memory 18.

The graphics application 10 comprises components written in an object oriented language such as JAVA. The components include a java.awt component 30; a diagram editor 32; a diagram 34; a policy 36; an abstract graphic 38 and abstract active content component 40 (see Figure 2). The abstract graphic component 38 has in this embodiment specialised connector 42A, printer 42B and controller graphics 42C. The abstract active content component 40 has a specialised connector 44A, printer 44B and controller active component 44C to correspond with the specialised graphics. The application 10 uses an extensible class library of graphical primitives (abstract graphic 38) and a diagram editor 32 with the graphics application 10 to co-ordinate, manipulate, store and distribute diagram elements. The application 10 allows diagrams to be created, edited, and embedded into web pages, where the graphical elements have active content for controlling resources.

The java.awt 30 provides the platform independent graphical functions for the application 10. The diagram editor 32 creates and manipulates a diagram 34 displayed on the monitor 28 using the graphics processor 20 by converting user actions (mouse 26 movements, keys 24 pressed) into editing commands. Many graphical manipulation controls (e.g. drag to move, drag edge to resize) are familiar to users and such controls are provided by the diagram editor 32 via the java.awt 30. The diagram editor 34 allows for user generated events such as creation, deletion, (dis) connection, containment or graphics. The diagram 34 is displayed by a diagram editor 32 which allows the user to edit the state of the diagram 34. The editor 32 co-ordinates the activities of the diagram 34, including multi-element editing, saving, restoring. Each diagram 34 is a self-contained entity and can therefore be distributed and synchronised. The editor 32 can be extended to allow synchronous remote editing of a single drawing by multiple users or editing by one user for several remote users to view. Each graphical element can be manipulated and presents a standard interface to the diagram editor 32.

The policy 36 provides a control over the elements that may be created in diagram 34 and can be specialised by the implementation depending on the nature of the particular business model. For instance the rules for creating a graphic object would vary depending on the active content and U,9 9 0 88 6 the business applications. Furthermore each active content controls actions performed with respect to itself or another active content.

The abstract and specialised graphical hierarchy contains the graphical objects that can be presented and manipulated on the monitor 28. The active content hierarchy contains functional or business objects which are associated with one of the graphics. At the top of each hierarchy are the abstract classes that contain interface functions for general communication across a bridge, interface 46. A subclass of the graphic abstract inherits the behaviour of the graphic abstract allowing it to converse with a subclass of the active content abstract. A subclass of the abstract active content class inherits the behaviour of the abstract active, allowing it to converse with the associated graphic class across the bridge 46. The interface between the roots of the two hierarchies allows bi-directional communication between the graphical and business, worlds.. If all communication passes via the bridge, ANY Graphic Element subclass instance and ANY Active Content subclass instance will be able to form a communicating pair. This allows the predefined graphics to be used for new Active Content, or new graphics to be developed in parallel. (not sure if this is relevant, it is a consequence of bridge architecture). If specialised communication between Graphic Element and Active Content is required then this generality will need to be broken.

Specialised graphic classes can be primitives (lines, rectangles, ovals etc.), complex shapes (composed of several primitives), multiple segmented lines, containing areas, clip-art, bitmaps, etc. Graphical behaviour is a well known domain, and so most of this can be supplied pre-written by the application. In the present embodiment a printer graphic comprises a suitable printer shape, a control graphic comprising a controller shape and a connector graphic comprises a line. Each graphical element is associated with an instance of active content. Users of the application are likely to wish to specialise the graphic classes provided to display their own particular graphic representation and the graphical hierarchy is designed to facilitate this. This is the supertype for all the implementations of business objects/business logic within the application. The bridge 46 allows important graphical editing commands to be transmitted to the business object. Such commands include create, delete, edit, activate and select. The default implementation of this interface contains a reference to its associated graphic. The behaviour of the different kinds of graphical primitives so defined in a class hierarchy allows better reuse of behaviour when specialising graphics.

Active content objects 44 inherit the bridge interface 46, and supply their own implementations of the required commands. The active content objects 44 are autonomous objects and can perform actions in U,,99 088 7 business processes independently of the diagram 34. They can also reflect their changing state by changing their associated graphic object in the diagram. Events in the graphical world are generated as the user manipulates the diagram elements. These are transmitted to the 'active content'. Events in the business world are transmitted to the graphical world. Events in the business world can be reflected by changing the appearance of the associated graphic. Colour, flashing, shape and text could all be determined at runtime by the active object, according to the whims of the specialised active class designer.

The following example illustrates the creation of a printer, a controller and a connection between them (see Figure 3).

The user requests creation of print controller I graphic 54 using the diagram editor 32. The GUI creation event is intercepted by the diagram editor 32 and the policy 36 is queried as to the allowability of such a creation. A print controller object 50 is instantiated, in this case, and the object seeks out and sets up a connection with print controller I resource 52. once a connection has been set up a print controller graphic 54 is instantiated and added to the diagram 34.

The user requests creation of a printer using the diagram editor 32. The GUI event is intercepted by the diagram editor and the policy 36 queried as to such a creation. In this case a printer object 56 is instantiated which seeks out and sets up a connection with printer resource 58. Once a connection has been set up a print graphic class 60 is instantiated and added to the diagram 34.

Creation of a connection active content 62 and a connection graphic 64 is next described referencing the flow diagram of Figure 4. The user selects (step 1A) the printer icon 60 and the print controller icon 54 for connection and makes a request (step 2A) for such a connection (see the flow diagram in Figure 4). The policy 36 is queried (step 3A) as to allowability of the connection and responds by supplying (step 4A) the type of connection and the active content required for the connection. A connection active content class 62 is instantiated and queries (step 5A) the respective active content objects of the selected graphics about the connection and requests (step 6A) the connection as appropriate. once set up the connection object instantiates (step 7A) the appropriate connection graphic class 64.

The embodiment is advantageously put to use when a connection between a printer and a controller is to be changed. This example is described with reference to Figure 3 and 5. Controller II active content 66 is added which creates a connection with a controller II resource 68 and places a controller II graphic 70 on the diagram 34. In this example ul'_ 9 9 0 8 8 8 the user wishes to move the connection from controller I to controller II. The user selects the connection graphic 64 which hi-lights connection points on the printer graphic 60 and the controller I graphic 54. The user then selects (step 1B) the connection point on the controller I graphic 54 and drags (step 2B and the dotted arrow in Figure 3) it to the controller II graphic 70. This event on the connector triggers the connector active content to initiate changing the connection. The policy is queried to find out in general if such connection change is allowed.

Controller I active content is queried as to whether the connection can be deleted and controller II active content is queried (step 3B) to find out if the new connection is allowed. If so allowed the connection active content requests deletion (step 4B) of the old connection and set up (step 5B) of the new connection. Finally the connection graphic is changed (step 6B) to reflect the new connection.

In another embodiment the application is used to control connections between a queue manager and queues. A queue manager active content is created using a specialised queue managing application, connects to a queue manager resource and places a queue manager graphic on a diagram. A second queue manger active content and graphic is likewise created. Further a queue active content is created, connects to a queue resource and places a graphic on the display. The user uses the GUI to create a connection between the queue and the first queue manager.

A connector active content is created which sets up the connection between the relevant resources and puts a graphic connector in the appropriate place. The graphic connection can be moved from the first queue manager to the second queue manager and the connection active content processes the change in connection between the resources.

Another example of active content of graphical object is a URL.

Highlighted text can be both the name of a graphical element and an indication that the graphical element is associated with active content (a URL in this case). Clicking on a graphical element will invoke the active content. In this case the active content will cause a browser to open a page to that described in the URL.

Active content is not restricted to the resources mentioned above or URL references. Any object conforming to the standard active content interface can be activated. The active content can be informed of many graphical manipulations, for example on user-click, on being connected to, or disconnected from, another active content. The active content could be any object with behaviour defined externally. e.g. sound file, video clip, control process in manufacturing plant, another tool, or external control processes and information gathering services. The active object could change its own graphical representation in order to represent a change in state. The abstract graphic class is extensible to U" 9 9 088 9 allow custom graphical representations and the abstract active content is extensible to allow new kinds of active content.

Several modes of 'use, of the embodiment are envisaged. An end-user creates and manipulates an active diagram using the embodiment or may simply view or use the diagram without manipulation of the elements. An application designer can use the components of the application to create active drawings within a different application. An application designer can extend the graphical class library to create new graphical shapes, or extend the active content class library to handle new behaviour/ embedded objects.

The following code example illustrates how an active diagram may be embedded in an application, either for editing, or for viewing, and how customisation of the application can be achieved. The code example below embeds a new Editor into a Java application. The editor instance handles all of its own user input and diagram editing and could be replaced by a viewer.

tv = new TabletEditoro; setLayout (new BorderLayout 0 this.add("Center", tv); if (null!= filename) I loadFile(filename); 1 The following code example illustrates how to customise a graphical element of the application. This code example subclasses 'Line' class from the application and overrides the addActiveContento method to use a customised active content and is useful for customising end-user responses. The second method shows MyLine delegating to the active content the responsibility for allowing line connection. This kind of customisation is of most use in an editor.

class MyLine extends Line f public void setActiveContento activeContent = new MyActiveContent public boolean canConnect(Drawable start, Draw able end) if Hstart!= null) & (end!= null)) return activeContent.canConnect start. act iveContent, end.ActiveContent) return false; ut'j 9 9 0 8 8 10 The code fragment below shows a specialisation of the Activecontent class The specialisation handles a user double-click by moving to another URL and is end-user behaviour support. The specialisation also handles the connection decisions delegated from the graphical element (editor behaviour).

class MyActiveContent extends ActiveContent { public void doubleClicked (Applet app) { (MyApplet) app) moveURL (myData) public boolean canConnect (Act iveContent start, ActiveContent end) return (start. isOutputQueue () && end. isInputQueue IBM, AIX, OS2 are trademarks of IBM Corporation. Microsoft and windows 95 are trademarks of Microsoft Corporation. JAVA is a trademark of Sun Microsystems Inc. Micrographfx is a trademark of Micrographfx Corporation.

Now that the invention has been described by way of a preferred embodiment, various modifications and improvements will occur to those persons skilled in the art. Therefore it should be understood that the preferred embodiment has been provided as an example and not as a limitation. Although the embodiment is described in relation to a Wintel platform it is not so restricted and other operating systems with GUIs such as IBM AIX or OS2 may be used. The embodiment is not restricted to an operating system having GUI functionality if the embodiment itself has the GUI functionality necessary.

U.,J9 9 0 8 8 11

Claims (26)

1 A method of building a graphical user interface comprising:

providing a graphic element class for handling a plurality of user interaction types performed on an graphic element and having a graphic for a particular resource type and a subset of the user interaction types particular to the resource type; providing an active content class for communicating with that particular resource type and having an interface with the superclass graphic element; creating an active instance of the subclass active content and associating it with a resource; establishing a communication link between the active instance and the associated resource; creating a graphic instance associated with the active instance; whereby once an active instance is created user interactions can be communicated from the graphic instance to the active instance as events and mapped by the active instance into resource commands for the resource.

2. A method according to claim 1 whereby the active instance sends an initiate setup message to the resource.

3. A method according to claim 2 whereby the active instance requests the resource to send resource output to the active instance.

4. A method as according to claims 1, 2 or 3 f urther providing a superclass graphic element for handling the plurality of user interaction types performed on an graphic element wherein said graphic element class is the subclass.

5. A method according to any of claims 1 to 4 further providing a superclass active content comprising an interface with the superclass graphic element wherein said active content class is the subclass.

6. A method according to any of claims 1 to 5 comprising the further steps of:

providing a connector subclass graphic element and an associated connector subclass active content; Uk'V99088 12 creating a connector active instance for connecting two or more resources; establishing a communication link between said resources; and creating a connector graphic instance and graphically connecting it to said resources.

7. A method according to claim 6 wherein said connector active instance queries the respective active instances of the resources as to the allowability of the connection between the resources.

8. A method according to claim 6 or 7 wherein said connector active instance requests the respective active instances of the resources to set up the connection between the resources.

9. A method according to claim 6, 7, or 8 wherein, on manipulation of the connector graphic, the connector graphic instance sends an event to the connector active instance; the connector active instance interprets the event in terms of the connection between the resources and communicates with the relevant resource active instances.

10. A method according to claim 9 wherein, on a user moving the connector graphic from a first resource to a second resource, the connector graphic instance notifies the connector active instance of the connection change; the connector active instance queries the relevant resource active instances on the allowability of the connection change; and the connection active instance sets up the connection between the first and second resource.

11. A method according to claim 10 wherein the connection active instance requests cancellation of the connection to the first resource; and the connection active instance requests set up of a connection to the second resource.

12. A method according to claim 10 or 11 wherein connector graphic passes the names of the resource graphic objects involved in the connection change.

UI,J99 0 88

13 13. A method according to claim 10, 11 or 12 wherein the connection active instance requests the connection graphic instance to update itself according to the new connection.

14. A toolkit for a graphical user interface comprising:

a graphic element class for handling a plurality of user interaction types performed on an graphic element and having a graphic for a particular resource type and a subset of the user interaction types particular to the resource type; an active content class for communicating with that particular resource type and having an interface with the superclass graphic element; means for creating an active instance of the subclass active content and associating it with a resource; means for establishing a communication link between the active instance and the associated resource; means for creating a graphic instance associated with the active instance; whereby once an active instance is created user interactions can be communicated from the graphic instance to the active instance as events and mapped by the active instance into resource commands for the resource.

15. A toolkit according to claim 14 whereby the active instance comprises means for sending an initiate setup message to the resource.

16. A toolkit according to claim 15 whereby the active instance comprises means for requesting the resource to send resource output to the active instance.

17. A toolkit according to claims 14, 15 or 16 further providing a superclass graphic element for handling the plurality of user interaction types performed on an graphic element wherein said graphic element class is the subclass.

18. A toolkit according to any of claims 14 to 17 further providing a superclass active content comprising an interface with the superclass graphic element wherein said active content class is the subclass.

ux-,99088 14

19. A toolkit according to any of claims 14 to 18 comprising the further steps of:

providing a connector subclass graphic element and an associated connector subclass active content; creating a connector active instance for connecting two or more resources; establishing a communication link between said resources; and creating a connector graphic instance and graphically connecting it to said resources.

20. A toolkit according to claim 19 wherein said connector active instance has means for querying the respective active instances of the resources as to the allowability of the connection between the resources.

21. A toolkit according to claim 19 or 20 wherein said connector active 20 instance has means for requesting the respective active instances of the resources to set up the connection between the resources.

22. A toolkit according to claim 19, 20 or 21 wherein, the connector graphic instance comprises means for sending, for a particular manipulation of the connector graphic, an event to the connector active instance; the connector active instance comprises means for interpreting the event in terms of the connection between the resources and communicates with the relevant resource active instances.

23. A toolkit according to claim 22 wherein the connector graphic instance comprises means, on a user moving the connector graphic from a first resource to a second resource, for notifying the connector active instance of the connection change; and the connector active instance comprises means for querying the relevant resource active instances on the allowability of the connection change and setting up the connection between the first and second resource.

24. A toolkit according to claim 23 wherein the connection active instance comprises means for requesting cancellation of the connection to Ur'.; 9 9 0 8 8 15 the first resource and requesting set up of a connection to the second resource.

25. A toolkit according to claim 23 or 24 wherein the connector graphic comprises means for passing the names of the resource graphic objects involved in the connection change.

26. A toolkit according to claim 23, 24 or 25 wherein the connection active instance requests the connection graphic instance to update itself according to the new connection.