EP1105816A1 - Vrml event propagation with counting of incoming routes for every node - Google Patents

Abstract

A data processing apparatus comprising a processor coupled with one or more storage devices is operable to manipulate data stored therein. Where the data is in the form of a plurality of nodes (68, 70, 76), with interconnect relationships specified between selected node input and output fields, event handling in relation to the nodes is by propagation of an event cascade through the node interconnections. To avoid dependency problems where two or more nodes route to the input field of the same node, each node routed to or from has an associated notification structure (72) in a field (74) of which a count is maintained of the number of incoming routes for that node. The arrangement is particularly suited for configuration as a virtual environment data browser for VRML-compliant and similar environments where the nodes are interconnected data assets of the environment.

Description

VRML EVENT PROPAGATION WITH COUNTING OF INCOMING ROUTES FOR EVERY NODE

The present invention relates to data processing apparatuses arranged to process stored data arranged as a plurality of interconnected nodes by event cascade from one node to another. In particular, although not exclusively, the present invention relates to such apparatus configured as a means for accessing, from a remote source, data defining a graphical representation of a virtual environment, wherein objects within said environment and characteristics thereof are stored as interconnected nodes.

A description of a service providing a virtual environment (or cyberspace) accessible by remote users is given in European patent application EP-A-0 697 613 (Sony Corp.). The system described includes a server providing a virtual reality space, and user terminals connected to the server via a high-speed communications network (using optical fibres or the like). A notable example of standardisation in the field of data defining virtual environments is the so-called Virtual Reality Modelling Language (VRML) as described, for example, in the VRML standard, version 2.0, issued as ISO/IEC WD14772 on 4th August 1996. VRML is a file format for describing interactive three-dimensional objects and worlds to be experienced on the Internet/World Wide Web and it is generally analogous to the way HTML (HyperText Markup Language) is used to describe documents for transmission over the Internet. In order to simplify the specification of how objects appear within a virtual environment, it is a known technique to restrict such objects to constructions from indexed face sets of geometrical data including a limited set of "building blocks" such as rods, cones and cubes, which may be simply specified in relatively few parameters, with specification of both an objects appearance and their interaction being through use of a data structure comprised of interconnected nodes. In terms of interaction, in VRML content creators can specify the user interaction with the virtual world through so- called sensor nodes. These nodes generate events in response to user input or changes in the environment. The routing mechanism allows these events to be propagated further into the VRML world, allowing other nodes to generate more events or make changes in the scene graph. Once a sensor or script node has generated an initial event, the event is propagated from the output field of the node producing the event along any of the connecting routes to the respective input fields of one or more other nodes. These other nodes may respond by generating additional events, which are also propagated, with the process continuing until all routes have been honoured. This process is called an event cascade. All events generated during a given event cascade are assigned the same time stamp as the initial event, since all are considered to happen simultaneously. An example of an event cascade is given below with reference to Listing 1 and Figure 3 of the drawings. Since a single initial event can start several event cascades, and all these cascades have to be processed by the VRML engine as if they happened at exactly the same time, a fast event routing implementation is essential. One known routing implementation is described in "Wired for Speed: Efficient Routes in VRML 2.0" by D.J. Woods, A. Norton, and G. Bell, Proc. VRML 97, Second Symposium on the Virtual Reality Modelling Language, Feb. 1997, pp 135-140. In the described implementation, the routing is performed as a two stage process commencing with a notification stage during which all nodes on event cascades are visited and a link to the storage location of the node output field or fields having a routing link to the input field thereof is stored: this process is known as invalidation. In the second stage, update traversal, for each invalidated node, values from the identified output fields are pulled and used to evaluate that node.

According to the authors of "Wired for Speed", the update traversal is very suitable to be combined with the rendering stage, since all nodes would need to be visited at that stage anyway. A major advantage would be that for nodes outside the view of the user, no values are pulled. However, although this appears to allow a large part of the event handling to be clipped, it may yield wrong results. The decision not to pull values for a certain part of the scene graph is based on the fact that the location is outside the view of the user at the time before all event handling is completed. However, pulling the values may actually bring this part of the scene graph into the view of the user. Consequently, this potential input field dependency on multiple output fields of other nodes can lead to inaccuracies.

It is accordingly an object of the present invention to provide a data processing apparatus capable of handling data structures comprised of such interconnected nodes without problems of input field dependency.

It is a further object to provide such a data processing apparatus suitable for configuration as a browser for graphical virtual worlds conforming to VRML or similar standards.

In accordance with the present invention there is provided a data processing apparatus comprising a processor coupled with one or more storage devices and operable to manipulate data stored therein, said data including a plurality of nodes, each with a respective input field and output field, with interconnect relationships specified between selected input and output fields, wherein event handling in relation to the nodes is by propagation of an event cascade through the node interconnections, from the output field of one to the input field of another until all interconnected nodes have been reached, wherein the event cascade is a two stage process commencing with a notification stage during which the node input fields to be processed are identified and the respective memory locations for the node output fields connected thereto are stored, and an update traversal stage during which the values for the node output fields are obtained from the stored locations and used to evaluate nodes for which they have a link to the respective input field; characterised in that said one or more storage devices includes, for each node, a respective notification structure into a field of which a value for the number of node output fields having links to the respective node's input field is accumulated during the notification stage and all those output field values are pulled from their respective locations during traversal prior to evaluation of the node.

The notification structure for a node may further comprise fields updateable by the processor to record routes to the input field and from the output field of that node, and each node input field routed to by a node output field may suitably have a link to said output field, with the processor further arranged to obtain from said output field a link to the location of said output field value. Further features of the present invention are defined in the claims attached hereto to which reference should now be made and the disclosure of which is incorporated herein by reference.

Embodiments of the invention will now be described by way of example only, with reference to the accompanying program listings and drawings, in which:

Listing 1 defines a spinning cube specified in VRML;

Listing 2 defines a VRML TimeSensor node;

Listing 3 defines a notification structure associated with a node being routed to or from in VRML;

Figure 1 is a block schematic diagram of a data processing system suitable to embody the present invention;

Figure 2 is a schematic representation of components of the apparatus of Figure 1 configured as a browser for virtual environments; Figure 3 represents an event cascade in the rotation of the cube defined in Listing 1 ;

Figure 4 schematically represents the association of notification structures with nodes in an event cascade;

Figure 5 is a flowchart setting out pre-processing and notification stage actions of an apparatus embodying the present invention; and

Figure 6 is a flowchart setting out the procedural steps involved in the update traversal step from Figure 5. Figure 1 represents a data processing system, such as a personal computer, which acts as host for a software utility which configures the system as a browser for data defining a virtual environment. The system comprises a central processing unit (CPU) 10 coupled via an address and data bus 12 to random-access (RAM) and read-only (ROM) memory devices 14, 16. The capacity of these memory devices may be augmented by providing the system with means to read from additional memory devices, such as a CD-ROM.

Also coupled to the CPU 10 via bus 12 are first and second user input devices 18, 20 which may suitably comprise a keyboard and a cursor control and selection device such as a mouse or trackball. Audio output from the system is via one or more speakers 22 driven by an audio processing stage 24. Video output from the system is presented on display screen 26 driven by display driver stage 28 under control of the CPU 10. A further source of data for the system is via online link to remote sites, for example via the Internet, to which end the system is provided with a network interface 30 coupled to the bus 12. The precise construction of the interface is not an essential feature of the present invention, although it will be recognised that the interface configuration will depend on the type of data network to which the system is to be coupled: for example, where the system is for use by a private home user, the data link is likely to be a telephone connection to a local service provider. In such a case, the interface 30 will suitably incorporate a modem. For other types of data link, such as an ISDN connection, the interface will be configured accordingly. Turning now to Figure 2, the inter-relationship of a number of functions assembled to configure the hardware of Figure 1 as a browser is illustrated. The functions in the example illustrated are particularly suited for browsing virtual environments defined in accordance with the VRML standard, version 2.0, issued as ISO/IEC WD14772 on 4th August 1996, although it will be understood that the present invention is not restricted by, or limited to, conformance with this standard. Only those components and functions of the browser as are required for understanding of the present invention will be described in detail: a fuller description of this browser configuration and its operation is given in the commonly assigned International patent application

WO 98/54636.

The browser is based around a scene manager 40 coupled with respective stores for functions 42, assets 44, and a hierarchy of scene nodes

52. In terms of the hardware of Figure 1 , the function, asset and hierarchy stores will generally be held in RAM 14, although some of the assets (for example standardised texture maps) may be provided in fast local caches, as represented at 60 in Figure 2, or via offline storage such as CD-ROM. The browser implementation program is suitably held in ROM 16, optionally linked to the boot-up utility of the system. Also coupled with the scene manager are four interfaces (IF.1-IF.4) 46-49 and a clock 50. The first of the four interfaces 46 corresponds in part to interface 30 of Figure 1 in that it represents the source of data received via the Internet: as represented by dashed line 54, the received data may be passed on from the interface to other (not shown) destinations, such as a further browser for HTML content in the received data. The first interface is the general arrival point for the data defining the virtual environment and, as such, it may also receive data from more local sources such as storage on the system RAM (14; Fig .1 ) or a CD-ROM. From the first interface, the data passes to a parsing stage 56 which divides it into data defining or relating to assets (which data is then passed to store 44) and data defining or relating to the run-time node hierarchy (which data is then passed to store 52). In operation, it will generally be the case that only the data required (in terms of texture, co-ordinates etc) immediately or in the short term will actually be loaded up by the browser. Related information not immediately required is instead held in the form of an address for its current storage location: in the case of data to be called up via the Internet, the data held by the browser will suitably be a URL (uniform resource locator) so that the browser can simply call up the data, via the first interface IF.1 , when the need for it becomes imminent.

The second interface 47 is an external API (application programming interface) comprising a set of interfacing subroutines required to enable multiple-user extensions to the system, such as two or more users having respective viewpoints within a common environment.

The third interface 48 is for operational segments used to program behaviour in a scene: these segments are referred to in VRML as script nodes. The script nodes contain a program module which, in response to a change or user action within a scene, effect a change somewhere else in the scene depending on the contents of the program module. The program modules may suitably contain Java ® language segments (Java is the object- oriented platform-independent and general-purpose programming environment developed by Sun Microsystems Inc). The third interface 48 provides a path to a Java interpreter which converts the Java segments from their platform-independent form to a form recognised by the host system.

The fourth interface 49 handles the passing of signals from (and in some instances to) the user input devices. The functions (functional modules) held in store 42 include an audio manager, MPEG decoder, features for node run-time implementations (such as interpolators), collision detectors, three-dimensional pickers, viewpoint control and routing mechanisms.

The assets within store 44 are the basic building blocks and operational features from which the virtual environment is built up. These include:

Three-dimensional geometry: defining the configuration and relative location of the polygons to be assembled for the virtual environment. It should be noted that each node (or screen component) will generally be defined in terms of its own co-ordinate system, which co-ordinates are converted by matrix multiplication back through a hierarchy to convert all components to a common set of "world" co-ordinates.

Textures: surface detail (at variable levels of resolution where storage space permits) for mapping onto polygon surfaces during rendering.

Audio data: audio segments, sound effects etc. to accompany particular scenes or events within the virtual environment.

Java ® code: the segments of code within script nodes which are called to handle various processes or actions within the rendered scene. user interaction data: for example, viewpoint control modes, identification of different user input devices and so forth.

MPEG data; video image data with accompanying data identifying compression, stored motion vectors etc. As shown, within asset store 44 is a master list 62 which identifies the assets (or their URL's) within the store. With this feature, whenever an identifier such as a URL for an asset is encountered in a VRML file, a quick check of the identifiers listed in master list 62 will indicate whether or not it is necessary to call for the asset from a remote location. Where the assets are spread over separate storage devices, the grouping is suitably coincident with the asset types. Also, for each such further store, a local list of asset identifiers is provided, as shown for the texture cache 60.

The further store for run-time structures 52 contains the VRML scene graph together with routes to allow dynamic behaviour. The scene graph is a hierarchical file specifying the connections between the various geometrical components making up an object within the virtual environment.

Turning now to the issue of event handling, a small example of the use of event cascading will first be described by way of example and with reference to Listing 1 which describes a cube that spins around the vertical axis when the user clicks on it. The routing needed for this simple example is illustrated by Figure 3 in which four nodes with respective input and output fields (referred to herein as eventln and eventOuts respectively) are shown. The four nodes are respectively a so-called toucher 60 (a node that responds to user selection), a timer 62, an interpolator 64, and the box graphic itself 66. The first route statement at the bottom of the VRML world specified by

Listing 1 states that the time on which the user clicks on the box must be passed on to the startTime eventln field of the time sensor node 62. This will cause the time sensor, during the time span defined by the variable cyclelnterval to generate several fraction_changed events which are passed on to the interpolator 64 (the second route statement). The interpolator node 64 then computes the rotation based on the value received from the timer node 62, and passes this rotation to the box graphic node 66. Considering now the input field dependency problem, this arises in

VRML (but is not exclusive to VRML) as the execution model for event handling in the VRML 2.0 specification leaves room for a number of different implementations, possibly resulting in different outcomes for the same VRML world. In general, a node's eventOut is dependent on all the node's eventlns which have the same timestamp. As an example, the TimeSensor node defined in Listing 2 will be considered. At some stage during execution, an instantiation of the time sensor which is disabled (that is to say the exposedField "enabled" is set to FALSE) receives a TRUE event on "enabled" and a time event on "startTime", both with the same time stamp. Furthermore, the start time has already been set. In VRML 2.0, each type of sensor defines when an event is generated. The state of the scene graph after several sensors have generated events should be as if each event is processed separately, in order. If sensors generate events at the same time, the state of the scene graph will be undefined if the results depend on the ordering of events, and it is this lack of definition which leads to problems.

Hence, the two events mentioned above should be in cascades initiated by the same sensor (assuming that different sensors do not generate events with the same time stamp). In a straightforward implementation, which simply propagates the events along a cascade (without checking the dependencies on other eventlns with the same time stamp) until the cascade terminates, two scenarios are possible:

1. The event cascade resulting in a TRUE event on the TimeSensors "enabled" is treated first. This enables the sensor and since a startTime for the sensor had already been set, a fraction_changed eventOut based on the old startTime is sent. When the event cascade with the time event on startTime is treated, no additional event is sent because a node cannot send two events on the same eventOut with the same time stamp.

2. The cascade that brings the time event to the startTime is treated first. Since the TimeSensor is still disabled, no eventOut for fraction_changed is generated yet. When the cascade resulting in the enabled event is treated, the TimeSensor is enabled, and a fraction_changed eventOut based on the new startTime is sent.

Since the eventOut on the TimeSensor's "isActive" is clearly dependent on more than one eventln (with the same time stamp) scenario 2 gives the correct result. In other words, should a situation as described above occur during a single simulation loop (all event cascades are started by events with the same time stamp) the process should not continue the cascade past the

TimeSensor until all possible eventlns have arrived at the sensor.

In the above-mentioned paper to Woods et al, a message routing implementation was proposed in which message passing happens in two stages: a notification stage and an update traversal. During the notification stage, all the nodes that are on event cascades are visited. The routing tree is followed from the cascade initiator nodes all the way down to the nodes at the end of the cascade. During this traversal of the routing tree, no values are passed around. The notification algorithm invalidates the eventlns along the cascades, and for each invalidated eventln stores a link to the eventOut where the invalidation came from.

Loops in the routing structure, which are not strictly VRML, are broken by the notification stage. The notification algorithm will not pass through the same eventOut twice in the same cascade, as required by the specification.

In the first stage, all the nodes that are on an event cascade are visited and the appropriate eventlns are invalidated. No values for the eventlns are set yet. These values are pulled in the update traversal. In this stage, all the nodes are visited again, and if one or more of a nodes eventlns are invalidated, the value for that eventln is pulled, and then the node is evaluated.

In order to prevent the eventln dependencies problem described above, the present invention uses a modified two stage event handler with the main goal of reducing the time needed at run time to process an event cascade, possibly at the expense of pre-processing time or memory. As illustrated schematically in Figure 4, in a pre-processing step, prior to handling of the event cascade from a first node 68 through a plurality of further nodes 70, a respective notification (vbNotify) structure 72 is associated with each node 70 that is routed to or from (or both). This notification structure 72, an example of which is defined in Listing 3, contains all the routing information for the owner, such as a list of eventlns and eventOuts and, as will be described in greater detail below, a field 74 for the number of routes arriving at that nodes eventln.

In the example shown, no notification structure is provided for that node 76 which is not routed to from any other node.

The general notification procedure is illustrated by the flowchart of

Figure 5. From the beginning 80 of each simulation loop for the VRML world, a couple of pre-processing steps are performed, commencing at 82 with the association of the notification structure with each node that is routed to or from (or both). This structure contains all routing information for the owner, such as a list of eventlns and eventOuts.

The second pre-processing step 84 is to create (and, for subsequent loops, update) a list of those nodes that can generate events. Initially this list contains all the sensor nodes, which nodes can start event cascades depending on user input or changes in the environment. Every loop this list is examined together with the user input and all appropriate cascades are initiated. The list is not static: for example a disabled TimeSensor that receives an enable event should be placed on this list, and if it is disabled again it should be removed.

The notification proper starts at 86 with the browser getting the first node from the list for evaluation and starting an event cascade from its eventOut, routing to the next eventln or eventlns 88. When the cascade arrives at a certain eventln, the eventln is invalidated 90 and the no_invalid_eventln field in the associated notification structure is increased 92 to reflect the number of invalidated eventlns the node has. Furthermore, the owner node is put on a list 94 of those nodes having invalidated eventlns, which list will be processed at the update traversal stage. A node evaluation function determines 96 if and through which eventOuts the cascade should be continued, and with these eventOuts stores a reference 98 to the location where the numerical value is stored which reference is also needed during update traversal. Following checks to determine whether there are any further routes to follow in the present event cascade 100 or any further cascades to initiate 102, update traversal 104 is performed before the process reverts to step 80 for the start of the next simulation loop.

The update traversal 80 is shown in greater detail in Figure 6 and involves visiting, in no specific order, all those nodes with invalidated eventlns listed at step 94 of the notification process. The traversal starts with selection of a first node from the list 106 and requesting of the value for the respective eventln 108. The eventlns have a link to the eventOut that caused the invalidation, which link is traced at 110, and stored with the eventOut is a reference to where the value can be obtained 112. After all the values for invalid eventlns have been pulled and updated 114, the node is evaluated 116. This evaluation also updates the reference 118 to the location of the value for the nodes eventOuts. When all nodes on the list have been evaluated, as determined at 120, the update traversal has been completed.

By use of the notification structures identifying how many eventlns of a node have been invalidated, the dependencies problem of nodes being evaluated in the wrong order is avoided. Furthermore, by storing data location references for eventOuts, it is not necessary to copy large volumes of data from node to node during the evaluation of long or complex event cascades.

Although defined principally in terms of a software browser implementation, the skilled reader will be well aware than many of the above- described functional features could equally well be implemented in hardware or a combination of hardware and software.

From reading the present disclosure, other modifications will be apparent to persons skilled in the art. Such modifications may involve other features which are already known in the design, manufacture and use of image processing and/or data network access apparatus and devices and component parts thereof and which may be used instead of or in addition to features already described herein.

Claims

1. A data processing apparatus comprising a processor coupled with one or more storage devices and operable to manipulate data stored therein, said data including a plurality of nodes, each with a respective input field and output fields, with interconnect relationships specified between selected input and output field, wherein event handling in relation to the nodes is by propagation of an event cascade through the node interconnections, from the output field of one to the input field of another until all interconnected nodes have been reached, wherein the event cascade is a two stage process commencing with a notification stage during which the node input fields to be processed are identified and the respective memory locations for the node output fields connected thereto are stored, and an update traversal stage during which the values for the node output fields are obtained from the stored locations and used to evaluate nodes for which they have a link to the respective input field; characterised in that said one or more storage devices includes, for each node, a respective notification structure into a field of which a value for the number of node output fields having links to the respective node's input field is accumulated during the notification stage and all those output field values are pulled from their respective locations during traversal prior to evaluation of the node.

2. Apparatus as claimed in Claim 1 , wherein the notification structure for a node further comprises fields updateable by the processor to record routes to the input field and from the output field of that node.

3. Apparatus as claimed in Claim 1 , wherein each node input field routed to by a node output field has a link to said output field, with the processor further arranged to obtain from said output field a link to the location of said output field value.

4. Apparatus as claimed in any of Claims 1 to 3 and configured as a virtual environment data browser, the processor and at least one storage device being coupled with data network interface means capable of receiving data defining a virtual environment and objects therein when coupled to a data network including a source of such data, the apparatus further comprising user input means and being configured to periodically generate an image based on said data defining the virtual environment and objects and from a viewpoint at a location and with an orientation in said virtual environment determined at least partially by input from said user input means.

5. Apparatus as claimed in Claim 4, wherein the apparatus is arranged to maintain a record in the form of a master list in the at least one storage device identifying data assets received via said network and stored in the storage device, to scan the master list to determine whether a particular asset is already stored on encountering a subsequent apparatus requirement for that asset and, if not, to formulate and send to the remote source of data via the data network a request for supply of said asset.

6. Apparatus as claimed in Claim 4, wherein the network is the World Wide Web and assets obtained from data sources coupled thereto are identified in the master list by their respective Uniform Resource Locators.