WO1997026608A1 - Systeme de mediatisation et de publication pour des applications ordinateur multimedias interactives - Google Patents

Systeme de mediatisation et de publication pour des applications ordinateur multimedias interactives Download PDF

Info

Publication number
WO1997026608A1
WO1997026608A1 PCT/CA1997/000039 CA9700039W WO9726608A1 WO 1997026608 A1 WO1997026608 A1 WO 1997026608A1 CA 9700039 W CA9700039 W CA 9700039W WO 9726608 A1 WO9726608 A1 WO 9726608A1
Authority
WO
WIPO (PCT)
Prior art keywords
elements
database
apphcation
media
computer
Prior art date
Application number
PCT/CA1997/000039
Other languages
English (en)
Inventor
Michael A. Lightheart
Scott R. Henderson
James Donald Durnford
Johannes Heupel
Praveen Reddy
Original Assignee
Vicom Multimedia Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vicom Multimedia Inc. filed Critical Vicom Multimedia Inc.
Priority to AU12984/97A priority Critical patent/AU1298497A/en
Publication of WO1997026608A1 publication Critical patent/WO1997026608A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data

Definitions

  • This application relates to a system, including apparatus and computer implemented methods, for authoring, publishing and running computer applications.
  • the invention has particular application to authoring, pubhshing and running interactive multimedia works, such as interactive tutorials.
  • the system includes a database for storing elements of computer applications and preferably has an integrated project management system which controls and tracks the operation of media input, database management, authoring, and output subsystems.
  • Multimedia works are becoming standard tools for educational, entertHinment and business purposes.
  • paper-based technical service and training manuals are being replaced by interactive CD-ROM based products.
  • Electronic manuals enable users to access tedrnical information more quickly and ef iciently than paper-based materials, while providing enhanced facilities for user-driven trairiing and skills development.
  • interactive manuals and other CD-ROM based multimedia works offer obvious benefits, producing an interactive multimedia work is time intensive. This makes the cost of production high as compared to paper-based resources.
  • the next step is to collect the various physical media elements which will comprise the content of the application.
  • the media elements may include text, graphic images, sound clips, film clips and the like. Some media elements may have to be created specifically for the application. The proposed interactions between the various media elements must also be plotted.
  • media elements Management of media elements is often a severe bottleneck in multimedia production. Before media elements can be combined into a multimedia work they must be made accessible, in computer readable form, to a computer system where they can be combined. Some graphic, text and animation elements may already be available in digital form. Other media elements must be converted to compatible digital files, which must be identified, stored, and then brought individually to the authoring process.
  • Computer readable media elements are typically stored in "file-based" systems which require an operator to assign a file name to each media element.
  • file names are limited to short, cryptic, combinations of letters and/ or numbers.
  • a user can only access a media element in such methods by recalling and using the assigned file name.
  • the structures and contents of the media elements are often not clear from the file name or from the media elements themselves. This information must be tracked manually. Usually the collection, inventorying, input and digitization of media elements is performed by skilled technicians which inflates production costs.
  • the final step in the production process is to create a run-time 5 application of the work for use by end-users.
  • the program may be stored on a
  • CD-ROM or loaded directly onto a file server hard drive or some other computer- readable medium, depending upon the end-user's requirements.
  • United States Patent No. 5,307,456, MacKay, issued 26 April, 1994 relates to an integrated studio for o producing multimedia works such as films or studio for producing videos.
  • the studio includes various multimedia production resources coupled to real-time local area networks.
  • the system provides multiple users with control of a plurality of dynamically allocated shared resources using a common user interface.
  • the real-time nature of the production system makes it particularly well suited for use in the media production of "live" events such as news events, concerts and sporting events.
  • the user interface enables multimedia elements to be created, edited, bundled, integrated and rearranged by multiple authors.
  • the system includes databases to store some media elements. Data elements may be stored and managed by an object-oriented database which also stores the current state of production resources and routing information.
  • the MacKay et al system is not tailored for producing interactive multimedia computer applications. It does not provide facilities for assembling or managing computer software. Further, it is not adapted for creating interactive technical manuals and the like. It lacks a scheduling system which is capable of prioritizing different projects.
  • Objects may include embedded references to other objects which are accessed via the object manager.
  • the Agrawal et al. system is designed for accessing and displaying media elements from a central database according to a predetermined format which is determined by the design of the database.
  • Agrawal et al do not include an authoring system for creating multimedia works. Users cannot establish an unrestricted number of links or associations between selected media objects.
  • United States patent No. 5,349,648, Handley, issued 20 September, 1994 relates to an automatic high speed publishing system for creating printed works comprising both text and graphics.
  • the Handley system allows textual data to be entered by unskilled personnel in "assembly-line" fashion. The data entry personnel do not need to concern themselves with text formatting but need only specify the general type of information being entered. The way that different elements in the finished work will be formatted is handled later by a skilled operator.
  • the Handley system is primarily file-based and requires the operator to specify text to be included in a publication by providing computer path names.
  • the Handley system is not well suited to importing diverse media types and is not able to create interactive, media dense, multimedia applications.
  • An object of this invention is to provide methods and apparatus for creating computer applications which overcome some of the deficiencies of prior art systems.
  • Another object of this invention is to provide methods and apparatus for authoring interactive multimedia works which overcome some of the deficiencies of prior art systems.
  • Another object of the invention is to provide a multimedia publishing system having a centralized database for storing media elements and integrated media input, object management, authoring, project management, and output subsystems associated with the database for authoring multimedia works in a production-line manner
  • Figure 1 is a data-flow diagram of a multimedia pubushing system according to the invention.
  • Figure 2 is a schematic diagram showing a computer network which may be used in the invention.
  • Figure 3 is a flow chart showing the overall sequence of steps in authoring a multimedia work according to the invention.
  • Figure 4 is a screen print showing a partial list of tasks required to create a typical multimedia work;
  • FIG. 5 is a block diagram of an input subsystem according to the invention.
  • Figure 6 is a diagram showing the structure of a media element
  • Figures 7A through 7D illustrate the components of four types of program elements which are used to provide program logic in applications created according to this invention
  • Figure 8 is a block diagram iUustrating the interrelationship between the classes of elements according to a preferred embodiment of the invention.
  • Figure 9 is a block diagram showing links between elements in a simple exemplary application;
  • Figure 10 is a view of a computer screen illustrating the appearance of the application of Figure 9 to an end user;
  • Figure 11A is a schematic illustration showing the relationship of 20 run-time interpreter software to a workstation and Figure 11B is a schematic illustration showing a possible architecture for a run-rime interpreter for use in the invention;
  • Figure 12 is a flow diagram illustrating a sequence of steps performed by a run-time interpreter in running an application
  • Figure 13 is a flow diagram illustrating a sequence of steps performed by a run-time interpreter in interpreting process elements in an appUcation
  • Figure 14 is a schematic diagram illustrating the relationship of an authoring program to apphcation elements stored in a central database; 3 o
  • Figure 15 is a simplified view of a screen illustrating a user interface in the system of the invention;
  • Figure 16 is a view of a screen showing the appearance of the user interface of the invention for a newly started apphcation;
  • Figures 17A through 17D are flow charts illustrating the sequence of steps involved in authoring a simple example apphcation according to the invention.
  • Figure 18 is a view showing an exemplary properties window for a control element of type button;
  • Figure 19 is a schematic illustration showing the organization of information in a database containing apphcation elements for use in the invention
  • Figures 20 A and 20B are views showing screens of a user interface of an exemplary apphcation.
  • Computer systems may be implemented using many different combinations of computer hardware and software. It will be apparent to those skilled in the art that the detailed implementation of the invention will vary depending upon the hardware and software tools used to practise the invention. To avoid confusion and to enable those skilled in the art to practise this invention, the invention is described using illustrative flow charts, screen displays, and algorithmic descriptions. Things which are well known to those skilled in the art, such as programming techniques and details of computer hardware and software are generally not described in detail in the following description to avoid obscuring the invention.
  • Application - an apphcation is a set of computer instructions that causes a computer to behave in a desired way.
  • Apphcation elements include media elements and program elements. All apphcation elements include a field containing a unique descriptor.
  • App Set - an "App Set” is a set of apphcation elements that are stored together so that they may be retrieved and reused in the authoring of apphcations.
  • Author - an author is a person who uses the methods and apparatus of the invention to create new apphcations.
  • Media data - media data is a representation in computer readable form, of information which can be perceived by a person playing back a multimedia apphcation.
  • media data could be: a body of data representing an image in a format, such as TIFF, GIF, JPEG etc.; alphanumeric data; a document in a particular word processing format; an animation file; etc.
  • Media data is not mear ⁇ gful until one understands its content and structure. For example, a user could consider a file containing the data representing a 24-bit scan of a painting to be a meaningless string of ones and zeroes. If a user attempted to play such data as an audio clip then the result would likely be rneaningless white noise.
  • Media element - A media element is media data combined with information which identifies the content and structure of the media data and/ or gives other information about the media data or its relationships to other information.
  • Physical Media - Physical media are any physical items which contain 0 information which can be perceived by a person which can be captured and stored in a computer storage device as media data.
  • physical media include photographs, negatives, sound recordings, video recordings, text documents etc.
  • Physical media can be anything which is not already stored in digital form. 5
  • Program Element those elements of an apphcation that are not media elements.
  • Program elements typically include one or more fields containing instructions a computer to do something in a desired way, information describing the relationship of the program element to other elements in an apphcation and o fields which contain information about the program element.
  • Subroutine - a set of one or more linked appUcation elements which are invoked by a gosub process element and which, when completed, chain to the element pointed to by the gosub process element that invoked the subroutine. 5
  • User - a user is a person who runs apphcations according to the invention.
  • an exemplary system 120 5 includes: a media input system 122 for capturing physical media in computer readable form and cooperating with other systems for associating information with the resulting media data to make media elements; a database management system 124 for cataloguing the elements of multimedia works; an object-oriented database 126 for storing media elements and other elements of multimedia 0 works (as is described in detail below); an authoring system 128, for liiilang elements together to create multimedia works; a project management system 130 to control, track and coordinate the operation of other parts of system 120; and, an output system 132 for outputting finished multimedia works.
  • System 120 facilitates rapid efficient development of highly sophisticated multimedia works. 5
  • system 120 typicaUy includes a number of computer workstations connected to a computer network 270 such as a local area network (LAN) or a Wide Area Network (WAN).
  • Network 270 includes one or more input stations 255 having displays 257, one or more server computers, 260, o 262 and one or more workstation computers, 272 aU connected by network 270.
  • TypicaUy database management system 124 and its associated database 126 reside in a database server 262.
  • Project management system 130 may comprise a process rurtning on a separate server 260.
  • Network 270 may be a peer-to-peer network in which case the functions of servers 260, 262 are carried out by one or 5 more of workstations 255.
  • the design phase includes selecting the subject matter of the work, planning the way in which the subject matter wiU be presented and choosing media content that wiU be included in the work. Physical media or information which is aheady in computer readable form and contains the desired media content must then be located and coUected. For example, if the work is about travel in Canada then the designer may want to include as media content maps of various Canadian cities, interesting photographs of Canadian scenes, sound chps of Canadian nightlife, text describing places to go and things to do in Canada and video chps. These materials must aU be procured. As another example, the project might be to create a multimedia technical service manual for a piece of electronic equipment. The proposed service manual wiU include a wide variety of media elements including text, technical drawings, video chps, audio and electronic measurements.
  • step 340 additional physical media that need to be digitized and other tasks that need to be completed to finish the project are entered into project management system 130.
  • the information includes detaUs of the steps which need to be completed to finish the multimedia work being created, and the deadlines for completing those steps. For example, if the project wiU include two video segments from a videotape, then the start and stop points of each segment are entered into the system.
  • the physical materials such 5 as paper drawings, sound recordings, videotapes etc. which will be digitized to obtain the media elements for a project are individuaUy identified by marking them with bar codes or other machine-readable tags.
  • Figure 4 shows a computer screen display 402 showing a partial hst 0 of tasks to be completed for a typical project. Many of the tasks involving digitizing physical media to create media data for inclusion in media elements.
  • Project management system 130 may also be used to request that additional media elements be input through input system 122 after a project has been started. 5
  • input system 122 is used to capture the media elements required for the project from the physical media which have been coUected and labeUed.
  • the captured media elements are stored in database 126 (step 342).
  • Project management system 130 coordinates data acquisition step 342 and o ensures that the resources required to complete the work are made avaUable in time to meet the project's deadlines.
  • data storage step 344 the digitized data acquired in step 342 is stored in database 126, together with additional information from project management system 130, in the form of media elements 680.
  • Each media element 680 consists of digital data 682 together with a header 5 684 containing identification and indexing information.
  • an author Links stored media elements together with program elements to make a multimedia work.
  • the author may be a subject matter expert, or may be a subject matter expert assisted by a technician.
  • system 120 preferably incorporates an authoring system which has a user interface which simplifies 0 creation of multimedia apphcations. New elements generated by the author are stored in database 126 in step 348.
  • each program element contains one or more fields containing information which can serve as computer instructions and a header containing identification and indexing information. 5
  • Output system 132 may optimize the arrangement of the work to suit the medium in which the finished work wiU be output. 0
  • Input subsystem 122 shown in detail in Figure 5, is designed to 5 facilitate the automation of media input tasks so that they may be undertaken by non-skLUed technicians.
  • Input subsystem 122 comprises a plurahty of computer workstations 255 connected to computer network 270. Each workstation has a display 257 and a bar code reader 558.
  • Each input workstation 255 is also connected to one or more input devices such as a scanner 560, an audio digitizer 561, a digital oscilloscope 562, a videotape player 563, digital camera 564.
  • the input devices capture the information content of physical items, such as a videotape, audiotape, or a sheet of paper as media data for storage in database 126.
  • Project management system 130 prioritizes pending information capture tasks, identifies which one(s) of workstations 255 are equipped with the input devices needed for each input task and directs each input task to a specific workstation. Input tasks may be routed depending upon what workstations and devices are aheady in use. Project management system 130 may also aUocate certain input tasks to specific human technicians. For example, some technicians accessing the system may be authorized to perform only simple input tasks, but not other more comphcated tasks.
  • project management system 130 When an operator logs into system 120 at one of workstations 255, project management system 130 prompts the operator to commence the next input task scheduled for that workstation (and that operator). The operator locates the appropriate physical medium and scans its bar code with bar code reader 558. System 120 verifies from the scanned bar code that the operator has selected the correct physical medium. The operator then inserts the physical medium into the appropriate input device and the system captures the required information in digital form.
  • technical detaUs such as the desired output file format, scanning resolution, scanning colour depth, audio sampling rates etc. are set by software under the control of project management system
  • Input subsystem 122 forwards the captured information to database 126 where it is catalogued using information about the structure and content of the media data from project management system 130 and stored as a media element 680.
  • System 120 may generate images for display on icons representing media elements 680.
  • the images may be standard icons representing various types of media element but are preferably low resolution "thumbnaU" views of the image represented by media data in each media element.
  • Data representing the "thumbnaU" view of a media element may be stored in the record for the media element in database 126.
  • System 120 includes a database management subsystem 124 for retrieving, displaying and categorizing elements stored in database 126.
  • Figure 6 is a block diagram showing the various components of a media element. Each media element 680 includes media data 682, and a header containing several information fields 684. The header 684 contain information about media element 682. Header fields 684 preferably include, at least, the fields shown in Table I.
  • Descriptor identifies this element and can be used as a pointer
  • the element descriptor is preferably automaticaUy aUocated by database management system 124.
  • An element can be retrieved from database 126 by requesting the element by its descriptor.
  • Media Type indicates what kind of media data 682 represents 686 and the format in which the data is stored
  • Project indicates the name of the project 687
  • Index a group of fields that identify the subject matter 688 represented by media data 682, the author etc.
  • Media a group of fields that identify technical detaUs 689 Description about data 682. For example, for a media element which represents an image, these fields specify things such as the resolution, colour depth, fUe format, size etc.
  • Database management subsystem 124 manages the contents of database 126 which includes both media elements 680 and program elements 700 for the creation and playback of multimedia works.
  • Media elements 680 are initiaUy registered in database 126 when they are captured by input subsystem 122 as described above.
  • previously digitized information may be directly imported into database 126.
  • database 126 can accept a previously digitized bitmap image and store it as a media element 680 for use in authoring.
  • database management system 124 requires the user to input at least a rruhirnum amount of information for information fields 684 so that the information can be stored as a media element 680.
  • System 120 can determine some information about a fUe containing previously digitized media data, such as the fUe format, the size of a digitized image etc. by studying the contents of the fUe.
  • Database management system 124 ensures that required fields are fUled in before an element can be stored in database 126. Additional elements may be created and automaticaUy stored in database 126 during the authoring process, which is described further below.
  • database 126 relieves users from manuaUy tracking media elements.
  • Database 126 and database management system 124 make it completely unnecessary for a user to remember cryptic path names to retrieve, play or create links to media elements.
  • a user can locate and retrieve media elements 680 by querying database 126.
  • Further advantages of storing media elements in database 126 are that the media elements are avaUable to aU users on network 270 and that multimedia apphcations which use the media elements do not need instructions from a user about the content and structure of the media elements because the necessary information is contained in each media element.
  • Database 126 may be any suitable existing avaUable database.
  • Database 126 may be a relational database or may be buUt on some other data model.
  • database 126 could be an object oriented database management systems (OODBMS) database, a flat fUe database, or an indexed sequential access method (ISAM) database.
  • OODBMS object oriented database management systems
  • IAM indexed sequential access method
  • database 126 is managed by a relational database management system.
  • database management system 124 may comprise OracleTM or SybaseTM database software.
  • Database 126 may reside entirely on one storage device, or an array of storage devices in one server computer, or may even be distributed across a computer network. Wh e it is not o preferable, database 126 could comprise two or more separate databases, each separate database containing different types of elements.
  • database 126 contains hbrary elements 1930 as weU as the elements of individual apphcations.
  • Library elements are elements which may 5 be reused across several apphcations. Combinations of program elements (App
  • Another advantage of storing the elements of apphcations in a 0 database 126 is that it is very easy to patch apphcations which are found to include errors. This can be done by simply replacing any defective elements in database 126 For example, a large apphcation distributed on CD-ROM may contain errors in one or two program elements. The problem can be corrected by supplying users with corrected copies of the defective elements together with a 5 smaU utility for replacing the defective elements in database 126 with the corrected elements. In most cases the corrected replacement elements will fit on a cheap low capacity storage medium, such as a floppy diskette. Program elements do not need to be stored in any particular order in database 126.
  • the structure of an apphcation is determined by links between the elements of the o apphcation and not by the order in which they are stored. This is very different form conventional computer programs in which the order in which individual computer instructions are stored can determine the order in which the instructions are processed. Patching an apphcation, as described above, is much easier and much more inexpensive than replacing the entire apphcation by creating a new corrected CD-ROM as might be required with a traditional monolithic computer apphcation.
  • a multimedia apphcation is built by linking together media elements 680 with program elements 700 which control how an end user will be able to traverse the media elements.
  • Program elements 700 are stored in database
  • a completed apphcation consists of a series of media elements 680 linked together by one or more program elements 700. Each element in an apphcation points to, or is pointed to by, one or more of the other elements of the apphcation.
  • a multimedia apphcation is created by combining five different types of elements, media elements 680, and four different types of program elements 700, namely, control elements 702, process elements 704, action elements 706 and hyperlink elements 708.
  • the elements of an apphcation are retrieved, as they are needed, and interpreted by a run-time interpreter 1130 comprising software running on a user's computer.
  • Program elements 700 define the flow of a finished apphcation and define the ways that an end user can interact with the completed apphcation.
  • a process element 704 is an element that includes one or more computer instructions that provide procedural logic for apphcations. For example, a process element 104 could start an apphcation running, create a control element, delete a control element, change the properties of a control element, caU a subroutine, create a loop or a branch, etc.
  • a control element 702 specifies the characteristics for a control, which is something that provides interaction with a user, either by providing a way for a user to provide input to the apphcation and/ or by providing a way for the apphcation to provide information to a user.
  • a control element 702 can act as a trigger for an event to occur in an apphcation.
  • control elements 702 of different types may be provided to specify a button, a panel, a hot spot, a certain frame of a video clip etc.
  • An action element 706 defines an action which can be performed when a control element is triggered.
  • a single control element may provide several different triggers.
  • a control element of type button may be associated, or "linked”, with: a “Click” action element, which responds when a user uses a pointing device, such as a mouse, to place a cursor on the button and then presses a switch on the pointing device; a "DoubleClick” action element, which responds when a user places a cursor on the button and presses twice in quick succession on a switch on the pointing device; a “MouseEnter” action element which responds whenever the cursor is moved into the area occupied by the button control in question; etc.
  • hyperlink elements 708 relate other elements to each other as is described in more detaU below.
  • a hyperlink element 708 of type NEXTPREVIOUSPAGE could be defined to relate sequentiaUy ordered pages to each other, as described below.
  • IUustrative sample of specific types of process elements 704, action elements 706, and control elements 702 are given in Tables II, III and IV below.
  • Program elements 700 each include a DESCRIPTOR field containing a descriptor 685 which is unique to that element.
  • Each program element 700 also includes pointer fields that identify other elements to which the program element is linked.
  • a process element 704 typicaUy comprises a TYPE field 710, which contains the type of process that the process element relates to, a CHILDJD field 712 which contains a pointer to the next process element in an apphcation, a group of fields 714 which contain operands, such as pointers to media elements, which the process element wUl affect, a CONTROL_ID field 716 which indicates the control element that the process element affects (if any), and a label 718 which contains a name for the process.
  • a control element 702 (Fig 7B) typicaUy includes a LABEL field 720 which names the control to which it relates, a TYPE field 722 which specifies the type of control to which control element 102 relates, a group of fields 724 which contain operands which determine various properties associated with the control (For example, if a control element 702 is of type "button" then the properties specified in fields 724 may specify the button's size, position, caption, color, an image to display on the button, and so on).
  • a control element may also have a PARENT ID field 726 which contains the descriptor 685 for another control on which the control in question is located; an APPVEW_ID field 728 which identifies a record in an external database with which the control is associated, and a NEXT ID field 730 which can be used to create multiple controls in a single operation, as described below.
  • An action element 706 (Fig 7C) typicaUy includes a field 740 containing a pointer to a control element to which the action element relates, a TYPE field 742 which specifies the action that wiU trigger the action element 706, an operands field 744 containing parameters that affect the operation of the action element, such as a parameter which defines specific frames of a video chp which, when played, wiU trigger the associated action, and a field 746 containing a pointer to a process element which wUl be executed when action element 104 is triggered.
  • a hyperlink element 708 (Fig 7D) includes a TYPE field 750 that contains information that identifies what the hyperlink does, and two or more, and preferably three or more, pointer fields 752, 754, and 756 which include the unique descriptor for elements (of any type) being linked.
  • First pointer field 752, IDl contains the information used to locate the desired hyperUnk element.
  • a field 758 may be provided to keep additional information about the hyperlink element 708.
  • Each type of hyperlink element 708 has an associated type of process element that is designed with knowledge of the structure and organization of the hyperlink element.
  • the various elements of a multimedia apphcation are related to one another by means of a model (the "MAJJIS model") which governs what classes of elements an element in any particular class can point to.
  • Figure 8 illustrates the MAJJIS model.
  • a process element 704 can point to another process element 704 (link 810) by means of a CHILDJD pointer, a media element 680 (link 811) by means of a MEDIA_ID pointer and/ or a control element 702 (link 812) by means of a CONTROL_ID pointer.
  • a control element 702 can point to another control element 702 (link 814), or an action element 706 (link 815), An action element 706 can only point to a process element 704 (link 817) and a hyperlink element 708 can point to two or more other elements of any type.
  • Figure 9 shows the elements which make up a short multimedia apphcation 900.
  • apphcation 900 When apphcation 900 is run it displays a form 1000 ( Figure 10) bearing an image control 1002 and a button control 1004. A portion of the image of image control 1002 is defined as a hot spot control 1006.
  • an audio clip is played by an audio control 1008 (which does not necessarily have any visible manifestation on the user's screen) and the image displayed in image control 1002 changes to a different image.
  • button 1004 a subroutine is run which plays a video clip and then terminates apphcation 900.
  • the first element 920 in apphcation 900 is a start process element 901.
  • a start process element 901 begins every apphcation.
  • the record for a start process element 901 may contain fields which contain information such as the name of the apphcation, the project that the apphcation belongs to, the name(s) of the author(s) of the apphcation, and so on.
  • the primary function of a start process element 901 is to point to the first program element in an apphcation.
  • a list of the apphcations in database 126 can be read y obtained by querying database management system 124 for a list of aU start process elements 901 in database 126.
  • Every apphcation has a "start" process chain 970 which consists of a start process element 901 and a series of other process elements which are linked to start process element 901 either directly, or through other process elements 704.
  • Each process element has a pointer (CHILDJD) 712 which contains the descriptor 685 for the process element which wiU be executed next.
  • CHILDJD pointer
  • FIG. 9 the flow of execution of process chains is indicated by sohd lines.
  • Other links between elements of apphcation 900 are indicated by dashed lines.
  • Each process element in a process chain has a CHILD_ID pointer 712 that points to one subsequent process element (unless it is the last process element in a process chain in which case the CHILD_ID pointer 712 contains a value which indicates that there is no subsequent process element).
  • Other process chains are initiated by action elements. For example, the sequence of elements which causes sound clip 954 to play when hot spot control 936 is clicked is a process chain 972 associated with the CLICK action 950 associated with hot spot control 1006.
  • the first element 920 is a start process element 901 which simply points to another process element 922 of a type which creates a form.
  • Create control process element 922 contains the instruction CREATE_CONTROL which causes the computer on which apphcation 900 is running to create form 1000 within which other controls for apphcation 900 wiU be displayed.
  • Process element 922 creates a form control, rather than some other kind of control, because it includes a pointer to control element 924 which is of type "form". Properties of form 1000, such as its size and color are stored in form control element 924. It can be appreciated that form control element 924 modifies the effect of the CREATE_CONTROL instruction which is stored in create control process element 922.
  • Process element 922 contains a CHILDJD pointer 712 to another process element 926.
  • Element 926 is also of the create control type.
  • Element 926 has a CHILD_ID pointer to image control element 928.
  • image control 1002 is drawn on form 1000. Properties of image control 1002, such as its size, position, whether or nor it initiaUy displays an image and, if so, what image are contained in image control element 928.
  • Process element 926 points to another process element 930.
  • Process element 930 is of a type which sets properties for an aheady existing control.
  • Process element 930 has a pointer 931 to image control 928 (which indicates that it is the properties of image control 1002 as set by image control element 928 that process element 930 wiU affect) and a pointer to media element 932.
  • image control 928 which indicates that it is the properties of image control 1002 as set by image control element 928 that process element 930 wiU affect
  • process element 930 sets image control element 928 so that the image of media element 932 will be displayed in image control 1002.
  • a process element of type "set property" such as process element 930 could change any properties of image control element 928.
  • Process element 930 points to a process element 934 of type "create control" which points to hot spot control element 936 and therefore creates hot spot control 1006.
  • Process elements 938 and 942 are chained, in turn, to process element 934.
  • Process element 938 points to audio control element 940 and therefore creates audio control 1008, which permits playback to a user of sound clips.
  • Process element 942 points to button control element 944 and therefore creates button control 1004.
  • apphcation 900 When apphcation 900 is run on a computer, as described below, the computer instructions in each of the process elements of start process chain 970 are executed in sequence, with the result that form 1000 and controls 1002, 1004, and 1006, as shown in Figure 10, are displayed on a user's computer screen and audio control 1008 is created to permit sound to be played to a user. Apphcation
  • Hot spot control element 936 which creates hot spot control 1006 contains a pointer to an action element 950 of a type which responds to clicks. If a user chcks on hot spot control 1006 then process chain 972 is triggered.
  • Process chain 972 begins with a process element 952 of type "set property" which contains a pointer 953 to audio control element 940.
  • Process element 952 also contains a pointer to media element 954 which contains as media data 682 a sound fUe that can be played back by audio control 1008.
  • Process chain 972 continues with a second set property process which changes the properties of image control 1002 (as indicated by pointer 957 to image control element 928) to display the image represented by the media data of media element 958 instead of the image of media element 932.
  • process chain 974 is performed.
  • the first process element 962 in process chain 974 caUs a subroutine
  • Process element 962 contains a pointer to process element 964 which is of a type that starts a subroutine 966.
  • Subroutine start process element 964 contains a pointer to further process elements (not shown) which make up subroutine 966.
  • execution of process chain 974 continues with process element 968 which terminates apphcation 900.
  • apphcation 900 links together four media elements 932, 954, 958 and the video media element of subroutine 966 by means of a number of process elements.
  • pointers which constitute the links between elements in an apphcation are the same as the unique descriptors
  • each process element has a CHILD_ID field 712 which contains the descriptor 685 for the next process element in the process chain.
  • Each process element which interacts with a control has a CONTROL_ID field which contains the descriptor 685 for the control element which defines the properties of the control etc.
  • This aUows the run-time interpreter software 1130 on a workstation 272 which plays back a multimedia apphcation to request the next process element in a multimedia apphcation, (or an element which contains information which is needed by a process element) by querying database 126 using the unique descriptor 685 for the requested element.
  • Storing apphcations in a database 126 provides concurrency, security and integrity within a set of elements and aUows simultaneous authoring and playback of apphcations.
  • the system described herein also can ⁇ uhirnize traffic across a computer network.
  • elements need only be loaded from database 126 as they are required. Only those parts of an apphcation which a user traverses need to be sent across network 270. It is possible, but not usually necessary on a fast network 270, to pre-f etch additional elements from database 126 and to store the additional elements in a buffer untU they are needed. Buffering elements of an apphcation may be desirable to increase the performance of the apphcation if network 270 is a slow network.
  • Network 270 may be, for example, the Internet, in which case it is preferable to buffer the elements of apphcations before playback.
  • the information contained in program elements is preferably in the form of short higher level commands, rather than relatively very large blocks of compUed computer code.
  • apphcations created by authoring system 128 are run by a run-time interpreter 1130.
  • the run-time interpreter 1130 preferably comprises software running on a workstation 272.
  • Run-time interpreter 1130 has access to database 126.
  • Figure 12 shows the general sequence of steps that take place when run-time interpreter 1130 runs an apphcation.
  • Figure 13 illustrates the general sequence of steps associated with performing a process (step 1220 from Fig 12). It must be emphasized that these diagrams are not meant to definitively iUustrate the actual flow of control in run-time interpreter 1130 Those skUled in the art will realize that run-time interpreter 1130 may be implemented in many possible ways in many different programming styles and languages.
  • Figures 12 and 13 are included to aid in understanding the steps involved in running an apphcation according to the invention but should not be construed so as to limit the scope of the invention.
  • Run-time interpreter 1130 aUows a user to select an apphcation to run (step 1212).
  • Selection step 1212 typicaUy involves querying database 126 to identify start process elements 901 in database 126 which start apphcations, to which the user has access rights, and which meet other criteria selected by the user. The user can select the desired apphcation by identifying it from the results returned by the query. If the workstation is ruivning the Microsoft WindowsTM operating system this is typicaUy done by double-clicking on an icon representing the desired apphcation. Run-time interpreter 1130 then retrieves the process element 901 that starts the selected apphcation (step 1214).
  • run-time interpreter 1130 retrieves from the CHILD_ID pointer field 712 of starting process element 901 the descriptor 685 for the next process element in the apphcation (step 1216). Run-time interpreter 1130 then retrieves the next process element in the apphcation by querying database 126 using descriptor 685 (step 1218), and so-on, until the end of the start process chain for the apphcation has been reached (the last element in the starting process chain is identified by the fact that it does not include a pointer to any subsequent process element).
  • Run-time interpreter 1130 may perform the actions specified by process elements as the process elements are retrieved from database 126 (as shown in Fig 12). In the alternative, run-time interpreter 1130 may retrieve aU of the process elements in a process chain first and perform the actions specified by process elements after the entire chain has been retrieved. Where the latter approach is taken then run-time interpreter 1130 may optionaUy perform pre ⁇ processing steps before the process chain is run. For example, pre-processing steps may include loading media elements into the memory of workstation 72 ready for playback or doing mathematical calculations etc.
  • run-time interpreter 1130 reads the TYPE field 710, which indicates what type of process element has been retrieved, and reads the element's properties and computer commands from the appropriate fields.
  • the computer commands are preferably subsumed in the element type (i.e. the actions performed by run-time interpreter
  • Run-time interpreter 1130 are determined by the element type. A separate field for computer commands is unnecessary). Run-time interpreter 1130 then causes the computer on which it is rurining (e.g. workstation 272) to perform the actions indicated by the process element being interpreted.
  • the TYPE field of a process element can be very short.
  • a TYPE field may be 50 bytes or less in size and can typicaUy be about 10 bytes in size.
  • run-type interpreter 1130 causes the computer on which it is running to perform many operations in response to each instruction (i.e. the TYPE fields in the process elements 704 serve as high level instructions akin to instructions in a scripting language). This provides advantages over systems in which large volumes of low level computer code must be transferred over a network.
  • Figure 11 shows one possible architecture for run-time interpreter 1130.
  • Run time interpreter 1130 includes a process handler 1131 which coordinates running apphcations and retrieving process elements from database
  • Process handler 1131 reads type information from each process element which is retrieved from database 126 and then caUs a process function 1132 to perform the actions specified by the process element.
  • a separate process function is provided for each type of process element.
  • Some types of process element may instruct the computer to perform different actions depending upon the type of media element that is being pointed to. For example, a create control process behaves very differently depending upon whether it is creating an audio control, an image control, or some other control type.
  • the process function 1132 associated with the "create control" process deterrnines what kind of control is to be created (by parsing the TYPE field 722 of the control element 702 pointed to by the CONTROL_ID pointer of the process element 704 being processed) and calls the appropriate one of a number of control functions 1133 to create a control of the type required.
  • run-time interpreter 1130 buUds a data structure such as control table 1140 (Fig. 11A) in the computer's memory.
  • Control table 1140 contains a record 1142 for each control which has been created in running an apphcation.
  • record 1142 contains a list of the control's current properties.
  • handler functions 1150 necessary to implement the control (step 1316). Functions 1150 may be part of run-time interpreter 1130 or may be provided, in part, by the operating system 1120 of the computer on which run-time interpreter 1130 is running.
  • Run-time interpreter 1130 may be constructed so that a single create control process element can create multiple controls. This may be done by means of NEXTJD fields 730.
  • an apphcation to create three controls having properties defined by control elements CONTROL1, CONTROL2 and CONTROL3 contains a "CreateControl" process element PROCESSl having a CONTROLJD field containing a pointer to CONTROL1.
  • the NEXTJD field of CONTROLl is set to point to CONTROL2 and the NEXTJD field of CONTROL2 is set to point to CONTROL3.
  • the NEXTJD field of CONTROL3 is blank.
  • run-time interpreter 1130 retrieves the contents of the NEXTJD field from CONTROLl and uses those contents as a pointer to retrieve CONTROL2. Run ⁇ time interpreter 1130 then creates the control defined by CONTROL2. Next, run ⁇ time interpreter 1130 retrieves the contents of the NEXTJD field from CONTROL2 and uses those contents as a pointer to retrieve CONTROL3. FinaUy, run-time interpreter 1130 creates the control defined by CONTROL3. As the NEXTJD field of CONTROL3 does not contain a pointer to any other control element then run-time interpreter 1130 continues by processing the process element foUowing PROCESSl.
  • run-time interpreter 1130 causes run-time interpreter 1130 to change the record 1142 in table 1140 which relates to the control being modified.
  • run-time interpreter 1130 runs the control handler function 1150 associated with that control so that the behaviour of the control is updated. For example, if the "play state" property of the record for a video control in table 1140 is changed from "STOP" to "PLAY" then run-time interpreter 1130 causes the handler function 1150 for that video control to begin playing the video data associated with the media element currently associated with the video control.
  • the computer commands in a process element are symbols or tokens which are interpreted by run-time interpreter 1130 in hght of the element's properties and the properties of other program elements to which the process element points. For example, if a process element is a process element of type "create control" then run-time interpreter 1130 retrieves the control element which is pointed to by the process element. If the control element is of type "button” then run-time interpreter proceeds to create a button control by causing a button to be drawn on the computer's screen and by setting up a handler to detect and pass on events caused by a user interacting with the control.
  • Run-time interpreter 1130 preferably includes within itself, or can access by making caUs to functions of the operating system on the computer, the computer code necessary to do this.
  • run-time interpreter 1130 preferably has the capacity to draw a button on the screen of workstation 272 and to animate the button as a user "chcks" the button using a mouse or other pointing device.
  • Run-time interpreter 1130 preferably does not need to retrieve from database 126 the specific computer code needed to implement these features.
  • the size, shape, position, colour, title, and other aspects of the button are determined by the properties fields of the control element which is pointed to by the process element and which is retrieved from database 126 and read by run-time interpreter 1130.
  • run time interpreter 1130 begins by sequentiaUy retrieving the process elements in start process chain 970 from database 126. Run-time interpreter 1130 then interprets the process elements in process chain 970 begiruiing with create control process 922. Run-time interpreter 1130 first determines that process element 922 is a create control process. Run-time 5 interpreter 1130 then retrieves form control element 924 and determines that element 924 specifies a form control. Run-time interpreter 1130 then writes the properties from element 924 in a record 1142 in table 1140 and creates and runs a handler function 1150 for the form control.
  • the handler function 1150 displays a form on the screen of the user's computer having the properties defined by 0 form control element 924.
  • a handler for a form control may be created, for example by calling the CreateWindow ⁇ system function.
  • Run-time interpreter 1130 then proceeds to interpret process elements 926, 930, 934, 938, and 942.
  • table 1140 contains 5 records, one record corresponding to each of form 1000, 5 image control 1002, button control 1004, and hot spot control 1006.
  • the fifth record corresponds to audio control 1008.
  • the user's screen appears as shown in Figure 10.
  • run-time interpreter 1130 After run-time interpreter 1130 has interpreted element 942 then it o simply waits because the CHILD JD field 712 of element 942 does not point to any other process element. If a user then chcks on hot spot control 936 run time interpreter loads and interprets process chain 972. When run-time interpreter 1130 receives set property process element 952 from database 126 it first determines that it has received a process of type "set property" and then 5 deterrnines, by reading the CONTROLJD field of element 952 (indicated by link
  • That process element 952 sets properties of the audio 1008 control associated with audio control element 940.
  • Process element 952 causes run-time interpreter 1130 to set two o properties of the control. First it sets the control to point to media element 954, which contains media data representing a sound clip to be played. Process element 952 also sets the "playstate" property of the control to "PLAY". It does this by changing the properties for the control in the appropriate record in table 1140. When run-time interpreter 1130 changes table 1140 it automaticaUy runs the handlers 1150 for any controls affected by the changes so that the changes manifest themselves. Process element 956 causes run-time interpreter 1130 to change the image displayed in image control 1002 in a si ⁇ lar fashion.
  • process chains 972 and 974 do not need to be retrieved from database 126 until a user has initiated execution of those process chains by chcking on hot spot control 1006 or button control 1004 respectively. If a user never initiates execution of a process chain then the elements of that process chain do not need to be retrieved from database 126, sent across network 270 or processed by run-time interpreter 272 at the user's workstation 272.
  • a subroutine may be handled, for example, by preserving the state of run-tirne interpreter 1130 and then calling another instance of nrn-time interpreter 1130 to process the subroutine. After the subroutine ends the original instance of run-time interpreter 1130 continues where it left off.
  • An advantage of the arrangement described herein is that a single apphcation can run on any computer as long as there is a suitable run-time interpreter 1130 rurming on the computer.
  • some workstations 272 might be PC computers ruir ing Microsoft Windows95.
  • Other ones of workstations 272 might be Apple Macintosh computers running the Apple System 7.0 operating system.
  • StiU other ones of workstations 272 might be UNIX workstations.
  • the same apphcation could be run on any of workstations 272 as 5 long as each workstation is running a version of run-time interpreter 1130 which is compatible with the operating system and hardware of the workstation.
  • the run-time interpreter software can reachly be customized so that elements agree with the customs of the user interface for the operating system in which run-time interpreter 1130 is running.
  • an Apple Macintosh run-time l o interpreter 1130 could display button control elements so that they look like the buttons commonly used in Apple Macintosh apphcations.
  • a run-time interpreter 1130 for the Windows NT operating system could display button control elements so that they look like the buttons commonly used in the Windows NT user interface, and so on.
  • a further advantage of this arrangement is that the amount of network traffic is jrun-Lrtized. Only those elements of an apphcation which the user plays back are sent across network 270. The elements are very small because they can contain short higher level instructions instead of blocks of object code
  • run-time interpreter 1130 does not require aU of the fields in the elements of an apphcation to play back the apphcation.
  • Traffic across network 270 can be further limited by sending to run-tixne interpreter 1130 only the contents of those fields from the elements of an apphcation which run-time interpreter 1130 requires to play back the apphcation.
  • 25 Traffic across network 270 can be reduced further stiU by providing some processing in database management system 124.
  • database management system 124 is a chent-server database management system
  • run-time interpreter 1130 can send a single query (in a suitable query language such as SQL in a manner which is weU understood to those skiUed in the art) 0 which wUl result in database management system 124 returning the essential fields from aU of the elements in a process chain of an apphcation.
  • a suitable query language such as SQL in a manner which is weU understood to those skiUed in the art
  • the essential information form each element of an apphcation is smaller than a network packet.
  • information sent over the Internet using the TCP/IP protocol is typicaUy broken into packets of 1024 bytes.
  • a single TCP/IP packet could, carry four process elements of record size 256 bytes.
  • Other systems which transmit computer code across a network typicaUy require large blocks of code to be transmitted.
  • Hyperlink elements provide a means for substituting one set of media elements for another set of media elements during execution of an apphcation. Hyperlink elements also provide a means to control the flow of execution of an apphcation and/ or to alter the media elements that are played back to a user of an apphcation on the basis of the state of the apphcation. Hyperlink elements can best be explained by means of a simple example.
  • the apphcation wiU provide two button controls on each page. One button control wiU change the display to show the next page of the manual. The other button control will change the display to show the previous page of the manual. Such an apphcation could readUy be written using the methods described above.
  • NEXTPREVIOUS PAGE type of hyperhnk element 708 having the structure shown in Figure 7D can be defined.
  • a NEXTPAGE type of process element and a PREVIOUSPAGE type of process element are also created (typicaUy the creation of a new type of process element involves defining a new process function for incorporation into run-time interpreter 1130 and would not be done by an author).
  • Each NEXTPREVIOUSPAGE hyperlink element has a field containing a pointer (e.g. the unique descriptor 685) for the media element containing the previous page, a pointer to the media element contaiiung the current page, and a pointer to the media element containing the next page.
  • a pointer e.g. the unique descriptor 685
  • These hyperlink elements are stored in database 126.
  • the preparation of these hyperhnk elements can generaUy be automated as the media elements which contain media data representing the pages of the technical manual generaUy include an index field containing the page number.
  • the NEXTPAGE process element operates as foUows (the PREVIOUSPAGE process element functions in an analogous manner).
  • the NEXTPAGE process element contains a pointer to a control element. In this case the pointer is to the image control which displays the page images for a user.
  • run-time interpreter 1130 determines from table 1140 what is the MEDIAJD property for the image control which is displaying the current page on a user's screen.
  • the MEDIAJD property points to the media element which is currently being displayed.
  • Run ⁇ time interpreter 1130 queries database 126 for a hyperlink element for which the "current page" pointer is the same as the retrieved MEDIAJD property. Database 126 returns the appropriate hyperlink element from which the run-time interpreter 1130 extracts the "next page” pointer.
  • Run-time interpreter 1130 sets the MEDIAJD property of the image control in table
  • run-time interpreter 1130 updates the image displayed by the image control so that the user sees the next page displayed.
  • hyperlink elements are generaUy retrieved by conducting a query against database 126 wherein the form of the query is determined by the state of the apphcation being run.
  • the query preferably searches for hyperlinks which are of a desired type and which contain a desired value in IDl field 750.
  • the searched for value in the IDl field could, for example, be the current property of a control or the descriptor 685 for a control. If hyperlink elements 708 are always searched for by searching the same fields, e.g. their TYPE and IDl fields, then a generic query can be defined for locating any hyperlink 708.
  • Other types of program elements are retrieved from database 126 by requesting them according to their unique descriptors 685.
  • hyperlink elements 708 may be used in many contexts.
  • an apphcation may be a multimedia technical manual.
  • the apphcation may, for example, be a service manual for a piece of electronic equipment which displays a schematic diagram 2030 and/ or an image of a circuit board 2014.
  • the application aUows a user to chck on any component 2018 shown on the circuit board 2014 or schematic diagram 2030 to produce a pop up menu which aUows the user to select various types of information about the selected component 2018.
  • the apphcation may, for example have the options: "What is?" - which opens a window 2050 ( Figure 20B) containing information about selected component 2018; "Where is?" - which highlights the selected component on schematic diagram 2030 or circuit board image 2014; "Measure” - which opens a window displaying information about measurements that can be made to that component; and so on.
  • the apphcation may create an image control containing schematic diagram 2030 and create hot spot controls 2036 corresponding to components 2018 on the schematic diagram.
  • Each of hot spot controls 2036 is linked to a "Chck" action element which link to a chain of process elements.
  • the 5 process cliains include a process element which produces a pop-up window for receiving the user's selection and caUs a subroutine which depends upon the users choice.
  • a l o subroutine is caUed which includes a process element which queries database 126 for hyperhnk elements of type "Whatls" which contain the descriptor for the selected hot spot control.
  • Database management system 124 searches database 126 and returns the requested hyperlink element which also contains a pointer to a process element which begins a process chain for displaying the information 15 that the user requested.
  • a subroutine is caUed which includes a process element which queries database 126 for hyperhnk elements of type "Wherels" which contain a field containing the
  • Database management system 124 queries database 126 for hyperlink elements of type "Wherels" which contain the desired component name. Database management system 124 then returns a hyperhnk element containing a pointer to a process element chain which highlights the component
  • Hyper nk elements may also be used to create apphcations which can be played back in any one of several languages. For example, an apphcation
  • control captions and media elements may be presented in either the English language or the German language. This can be done by referencing all language-specific captions and media through hyperlinks.
  • Two types of hyperlink would be provided, the a hyperhnk of type ENGLISH_LANGUAGE would contain an identifier in its IDl field and the unique descriptor 685 for a media element containing the desired Enghsh language text (or audio or video) in a second pointer field.
  • a hyperhnk of type GERMAN_LANGUAGE would contain the same identifier in its IDl field and the unique descriptor 685 for a media element containing the equivalent desired German language text (or audio or video) in a second pointer field.
  • a pair of such hyperlink elements would be included for each different language-specific caption and each language specific media element.
  • the identifiers would be different for hyperlink elements specifying different language-specific captions or language specific media elements.
  • AU language-specific captions and media would be included through the use of a "SELECT_LANGUAGE" set properties process element which operates as foUows.
  • the SELECT LANGUAGE process element comprises a field containing the identifier contained in the IDl fields of the pair of hyperlink elements containing references to the appropriate language-specific media elements.
  • rim-rime interpreter 1130 Prior to ruruiing the apphcation rim-rime interpreter 1130 would receive language selection info ⁇ nation from a user.
  • run-time interpreter 1130 would cause run-time interpreter 1130 to either always retrieve the ENGLISH_LANGUAGE type hyperlink element or to always retrieve the GERMAN JANGUAGE type of hyperlink element depending upon whether the language selection information specified that the user wished to view the apphcation in Enghsh or German.
  • run-time interpreter 1130 encounters a SELECT_LANGUAGE process element then it queries database management system 124 for the appropriate type of hyperlink element (i.e. ENGLISH JANGUAGE or GERMANJANGUAGE) for which the IDl field contains the identifier stored in the SELECT_LANGUAGE process element.
  • Run-time interpreter 1130 then extracts a pointer to the media element (in the desired language) and sets the MEDIAJD for the control in question to point to that media element. If a control caption is being set then the text for the caption may optionaUy be stored in the hyperlink element itself. This method can be used to play back apphcations in any of many languages. 0
  • System 120 has a graphical user interface which aUows users to author even sophisticated multimedia works without typing scripts or lines of 5 computer code a hyperlink of type ENGLISH_LANGUAGE would contain the descriptor for a control in its IDl field and the descriptor for a media element containing the desired Enghsh language text (or audio or video) in a second pointer field, or doing other tasks which are traditionaUy associated with computer prograrnrrdng and which require substantial knowledge about o computer prograrrL ing.
  • the interface aUows users to author apphcations by graphically manipulating icons which represent media elements 680 and program elements 700.
  • Multimedia apphcations are written in a manner somewhat similar 5 to programming in MicrosoftTM Visual Basic.
  • the elements of a multimedia apphcation are created by creating new program elements 700 by copying prototype program elements, setting properties of the copies of the prototype program elements, storing the resulting program elements 700 in database 126 and linking the resulting program elements 700 to each other and to media o elements 680 from database 126.
  • Elements are selected by the author(s) from tool boxes or other displays and then are combined in the graphical user interface.
  • high-level multimedia works can be intuitively created using a visuaUy- driven "point and chck" approach.
  • the authoring subsystem is simultaneously accessible by multiple authors located at multiple workstations.
  • authoring system 128 can identify the kind of media stored in a media element, the authoring system can prevent authors from mistakenly hnking the wrong type of media element to a process element. For example, the authoring system can prevent an author from trying to display an image in TIFF format using an audio playback process element.
  • an apphcation when an apphcation is being authored on system 120, a user at one of workstations 272 runs authoring software 1400 on workstation 272.
  • Authoring software 1400 interfaces with database management system 124 through network 270.
  • Authoring software 1400 aUows a user to record, catalogue and play back new apphcations.
  • Preferably authoring software 1400 plays back apphcations by means of run-time interpreter 1130.
  • AU of the elements of the apphcations are automaticaUy stored in database 126 by authoring software 1400 as the apphcation is being authored. These elements can therefore be made avaUable to other users who have access to database 126.
  • Figure 15 shows a screen 1500 from an exemplary user interface for an authoring system for use in the invention.
  • Figure 15 is somewhat simplified for clarity.
  • the icons, which are represented as featureless boxes in Figure 15, preferably bear images so that they can be readUy identified and dist-nguished from one another.
  • the simple multimedia apphcation 900 of Figure 9 is in the process of being created.
  • Multimedia apphcation 900 is created in an "apphcation area" 1512.
  • apphcation area 1512 is a smaU area on the computer screen. However, an apphcation area can also fiU up the entire screen.
  • Apphcation 900 includes the four visible controls which are shown in Figure 10.
  • Tool bar 1520 contains icons representing control elements which a user may select for inclusion in a multimedia apphcation
  • tool bar 1522 contains icons representing a number of media elements
  • tool bar 1524 contains icons representing a number of process elements for selection by an author.
  • the tool bars are movable and sizable.
  • the tool bars may optionaUy be turned off so that the entire apphcation area is visible. Users may also create new tool bars which contain combinations of elements of their choice.
  • the user interface also preferably provides a flow diagram window 1528 which Ulustrates the interrelationship of the various elements of the multimedia apphcation 900 under construction.
  • Authoring software 1400 maintains a data structure such as table 1402 which lists ah of the links between elements in the start process chain 970 for the apphcation.
  • Table 1402 may be the same as, or a superset of, the table 1140 which is used by run-time interpreter 1130.
  • authoring software 1400 When a user begins authoring or editing elements in a process chain in an apphcation then the author can cause authoring software 1400 to send a message to database management system 124 requesting that the records relating to elements in that process chain be locked. This may be done, for example, by writing a value to a field in the first process element in a process chain to signify that the process chain is "locked".
  • authoring software 1400 retrieves a new process chain from database 126 it checks this field. If the process chain is "locked” then the authoring software wiU aUow the author to display, but not modify, the process chain. This prevents other users from changing that process chain whUe it is being edited simultaneously by another user. Locking functions may also be provided by database management system 124 in ways which are known in the art.
  • authoring software 1400 updates table 1402 and, when new elements are added or existing elements are changed, writes the new elements and/ or changes to database 126.
  • the user can cause authoring software 1400 to refresh table 1402 by rereading from database 126 the process chain which is being edited. In this manner changes which have been made by other authors (if the author has decided not to lock the process chain) can be displayed in flow diagram 1528.
  • Tool bars 1520 and 1522 show either a complete set, or a subset, of prototype control elements, prototype process elements and media elements which are stored in authoring software 1400 and/ or in database 126. Users may query database 126 to select elements which may not be displayed on the tool bars. For example, a user could query database 126 to find aU hot-spot type control elements for a particular project. The set of control elements 702 of type "HotSpof'in database 126 which match the query are returned in a new tool bar. These control elements may be moved into other tool bars by dragging and dropping them onto the other tool bars.
  • Tool bar 1524 is a window into a media element management program.
  • the media element management program may be a separate program from authoring software 1400.
  • Authoring software 1400 runs the media element management program when an author requests media elements.
  • the author can use media elements located by the media element management program by dragging an icon representing a desired media element from the media element management program window 1524 and dropping the icon on the control where the author wishes to use the media element.
  • Media element management program and authoring software 1400 can communicate by exchanging messages between themselves in ways which are weU known in the art.
  • authoring software 1400 and the media element management program are ruTLning under the Microsoft WINDOWSTM operating system then they may communicate by using Windows messaging functions to exchange messages containing the descriptors 685 for media elements selected by a user for incorporation in an apphcation.
  • prototypes for program elements 700 are stored on workstation 272, where they can be accessed locaUy by authoring software 1400.
  • Each prototype has a preset type and a selected set of default properties which may be set to selected values.
  • authoring software 1400 contains a set of allowed values for each property of each type of program element.
  • Authoring software 1400 prevents an author from setting any property of an element to an illegal value. Tables II, III and IV contain an exemplary list of the types of prototype elements that are avaUable for authoring purposes.
  • Audio Player plays audio chps
  • MPEG Player plays MPG, AVI, Video data
  • Check Box A box that can be toggled between checked and unchecked states
  • Combo Box a box into which a user can enter text or select predefined text from a puU-down list
  • Form a form for placing other controls on Frame a frame for placing around other controls
  • Group Button a group of buttons only one of which can be depressed at once
  • a hot spot Spot may be superimposed over an image control or a portion thereof - rectangular
  • Option Button a group of check boxes, only one of which can be checked at once
  • Text Box a box for accepting and/ or displaying text
  • Activate occurs when a form is activated
  • Chck occurs when a mouse button is chcked with the cursor on the control
  • DoubleChck occurs when mouse button is double chcked with cursor on control
  • ExternChck occurs when a user chcks the mouse with the cursor outside of the control
  • GotFocus occurs when a control becomes the current control
  • KeyPress occurs when a keyboard key is pressed and relesed
  • Load occurs when a control is loaded
  • MouseDown occurs when user pushes a mouse button down
  • MouseEnter occurs when the mouse cursor enters the active area for the control
  • MouseLeave occurs when the mouse cursor leaves the active area for the control
  • MouseUp occurs when a user releases a mouse button
  • OpenMedia occurs when a control opens a new media element
  • Paint occurs when an image control redraws its display
  • QueryUnload occurs before a form unloads (this element can be used to run a process chain to check to ensure that any information from a form that needs to be saved has been saved before the form is closed)
  • Resize occurs when a control is resized
  • ScroU occurs when the position of a scroU bar is changed
  • ScroUStart occurs when a user starts to move a scroU bar
  • ScroUEnd occurs when a user stops moving a scroU bar
  • Select occurs when text is selected in a text control
  • Timer occurs at a set time relative to a trigger event
  • AddProperty adds a property to a control element
  • CreateControl creates one or more new controls based upon one or more control elements
  • CreateTable creates a table.
  • DoOnAUControl performs an action in respect of aU controls (e.g. moves s or scales aU controls by a certain amount)
  • Method performs an action against a control i.e. zooms an image in a specified way, moves the control etc.
  • SetProperty sets a property of a control
  • StartSub a start element for a subroutine
  • TypeDef define a user defined type
  • TypeUnDef deletes a user defined type
  • apphcation area 1512 is blank and flow diagram window 1528 shows only a single process element which represents the start process element 901 of the new apphcation.
  • An author can add controls to the apphcation to permit end users to interact with the completed apphcation.
  • the user interface preferably aUows a user to create and place a new control by dragging a desired new control from tool bar 1520 onto apphcation area 1512.
  • TypicaUy a user would begin by creating a new control element by chcking on a desired control type from toolbar 1520 and then using the mouse to drag out an area in appUcation area 1512 where a control will be placed.
  • the authoring system displays a window which shows the properties for the control, which the author can set. InitiaUy the properties window shows default properties which are possessed by a prototype element for a control of the type which is being created.
  • an author could begin to author apphcation 900 of Figure 9 by opening a new apphcation and creating a form control in apphcation area 1512 .
  • the author does this by cUcking on a form control icon in toolbar 1520 and then dragging out a rectangular area inside apphcation area 1512.
  • the position and size properties of the control are set according to the area defined by the user in apphcation area 1512.
  • the author can set other properties of the form control 1000 by opening a properties window which lists aU properties available for form control 1000.
  • authoring software 1400 automaticaUy opens a properties window whenever a new control is created.
  • authoring software 1400 automaticaUy When the author has completed setting the properties for the control then authoring software 1400 automaticaUy generates "Create Control" process element 922 and form control element 924 and sends these elements to database management system 124 for storage in database 126.
  • database management system 124 creates and returns to authoring software 1400 a unique descriptor 685 which unambiguously identifies the record in database 126 which constitutes that element.
  • Authoring software 1400 uses this information to update table 1402.
  • authoring software 1400 sets the CHILD_ID pointer field 712 of the process element which chains to the new element to the descriptor 685 for the new element. This is done by sending a message to database management system 124.
  • Figures 17A and 17B illustrate, by way of example, the sequence of steps which take place when an author begins to create the apphcation 900 of Figure 9 by creating a form control and an image control. These steps produce elements 920 through 932 of Figure 9.
  • An author may load an apphcation which has been previously created for editing. For example, an author may start and begin to author an apphcation 900 as described above. After completing the steps Ulustrated in Figures 17A and 17B the auhor could save the apphcation. At a later date the author could select the partiaUy completed apphcation 900 and proceed to add the remaining elements of apphcation 900.
  • the author first causes authoring software 1400 to query database management system 124 for a hst of the apphcations which are stored in database 126.
  • the query returns the start process element for each apphcation in database 126 which matches the query.
  • Authoring software 1400 displays the results of the query in a window.
  • the author can limit the number of apphcations which are located by the query by restricting the query to apphcations for a particular project, or apphcations which are associated with a selected key word.
  • authoring software 1400 When the author locates the apphcation which is to be edited the author selects that apphcation by, for example, chcking on the name of the apphcation in the query results window.
  • authoring software 1400 first retrieves aU of the elements in the starting process chain for the apphcation and recreates table 1402.
  • Authoring software 1400 does this by reading the pointer field of start process element 901 (which contains the descriptor 685 for the next element in the apphcation) and requesting the next element in the apphcation using descriptor 685, as described above, and so on.
  • the author can continue to author the apphcation or edit properties of the elements which are aheady present in the apphcation. For example, the author can proceed to add hot spot control 1006, an audio control 1008, and button control 1004 to apphcation 900.
  • Figure 18 shows a typical properties window for a button control element. By changing properties for a control element the user can change the shape, size, text and other characteristics of the associated control as it will appear to a person rurning the ultimate multimedia apphcation.
  • a user chcks on the control for which he or she wishes to specify an action element.
  • the user could chck on control button 1004 of Figure 15.
  • Flow diagram 1528 would then show the action elements associated by default with button 1004.
  • the user could then select, for example, the CLICK action element for button 1004.
  • a CLICK action is typicaUy associated with aU button control elements.
  • this invention permits aU apphcation elements to be recorded in a shared database 126 as an author creates a multimedia apphcation such as apphcation 900.
  • An author who is authoring a multimedia apphcation may, at any time, play back the apphcation, to the extent that it has been completed, by selecting the playback button from navigation tool bar 1526.
  • authoring software 1400 incorporates run-time interpreter software 1130 so that an author can play back an apphcation and see exactly what a user would see when the apphcation is played back by run-time interpreter 1130.
  • Apphcation elements which have been combined may be reused many times either within the same multimedia apphcation or by other multimedia apphcations.
  • a user wishes to reuse components of a multimedia apphcation the user only needs to change the properties of the program elements and/ or add or remove program elements to change the functionality of the apphcation.
  • a user can also change the media content of an apphcation, without changing the overaU way that the apphcation functions, by simply pointing those elements of the apphcation that point to media elements to alternative media elements.
  • database 126 may contain several different apphcations.
  • Figure 19 is a schematic iUustration showing the organization of database 126.
  • a completed apphcation 1910 (Apphcation A) aheady authored by a user is recorded in database 126.
  • Apphcation 1910 comprises a coUection of mterlinked program elements 700 and media elements 680.
  • database 126 also contains an element hbrary 1930.
  • Library 1930 contains commonly avaUable predefined sets of apphcation elements (App Sets) and subroutines which can be used by authors in the authoring of new apphcations.
  • Elements from hbrary 1930 can be retrieved by the author of new apphcation 1920 by querying database management system 124 for the descriptors 685 of elements in hbrary 1930 which meet specified criteria.
  • An author may also manuaUy insert the descriptor 685 for an apphcation element to be linked to another apphcation element into the property window for the other apphcation element.
  • New apphcation 1920 may be made up of newly created apphcation elements, apphcation elements from old apphcation 1910, elements from hbrary 1930, or a combination of aU of these.
  • New apphcation 1920 may also contain references to external media 1940 such as fUes stored within a fUe management system and/ or data in an external database 1950. Unnecessary references to external media should generaUy be avoided.
  • Database management system 124 ensures the concurrency and integrity of elements stored in database 126. The system cannot prevent the problems that can occur if external elements are deleted or renamed and the records of those external elements in database 126 are not updated.
  • Project management system 130 tracks the status of aU of the projects which are taking place on system 120. At any given time several apphcations, at various stages of completion, may be on system 120. These apphcations between them may comprise hundreds or thousands of media elements as weU as hundreds or thousands of program elements. Project management system 130 tracks the amount of work done on each project, including statistics such as the number of images digitized, the amount of time spent by authors in authoring the apphcation, etc.
  • the progress of scheduled tasks such as the routine digitization of physical media is automaticaUy tracked by project management system 130.
  • Unscheduled tasks such as cropping or editing images are preferably also monitored by project management system 130. This may be done, for example, by providing acces to the software tools and/ or computer hardware needed to complete each unscheduled task through the project management system.
  • the start task function records information about who is performing the task, what time the task is being started, and other project management information and then makes the resources necessary to perform the task avaUable to the person perf orrning the task.
  • the start task function may also check to ensure that the necessary resources are avaUable for use.
  • project management system 130 runs an end task function.
  • the end task function stores a record in database 126 which contains information about whether or not the task was completed, how much time the task took, what person did the task, on what workstation was the task done, what devices were used etc.
  • the end task function may also store accounting information in database 126.
  • Tasks may have associated costs which vary from task to task.
  • Project management system 130 can calculate the cost of completing a task from the time taken, the type of task, and/ or other project management information. Project management system 130 can therefore keep an up-to-date running taUy of the costs incurred in respect of any project as the project is progressing. This can be impossible with other systems in which there is no good way to keep track of aU of the work that is being done on a project.
  • an apphcation After an apphcation has been authored then it is immediately avaUable to aU users on computer network 270 who have the necessary permissions to access the apphcation.
  • the apphcation may be played back by such users either by running the authoring system, which is described above, or by using a stand alone program which simply runs the apphcation and does not provide any authoring tools.
  • multimedia apphcations authored on system 120 wUl be exported for use on other computer systems or networks.
  • TypicaUy for example, a multimedia apphcation wiU be written to a CD-ROM.
  • Output system 132 retrieves aU of the elements necessary to run the apphcation from database 126, optimizes the arrangement of those elements for inclusion on a particular output media, and writes the completed apphcation to the desired output media. Optimization includes excluding elements which were needed solely for authoring and managing the production of the work being output, compressing and/ or paUetizing media elements for efficient retrieval, and removing redundant logic and/ or consolidating repeated logic within an apphcation. Output system 132 may also write ancUlary fUes which can be used to instaU the apphcation on a different computer system and a program for running the instaUed apphcation.
  • the invention is used to prepare a service manual for a piece of equipment, such as a piece of electronic equipment.
  • the apphcation includes an image control which displays an image showing components of the part being serviced. For example, as shown in Figure
  • An apphcation may produce a screen display 2000 which includes an image control 2010 showing a picture of a chcuit board 2014.
  • Circuit board2014 has a number of components 2018 mounted on it.
  • a hot spot control 2020 is associated with each component 2018. By chcking on hot spots 2020, a user can view inforrnation about corresponding components 2018.
  • Each component 2018 has a name, such as a part number or other reference number.
  • the hot spot control 2020 associated with each component 2018 includes index fields which include the name of the component so that a program author can easUy associate the hot spot controls 2020 with the components 2018 to which they relate.
  • Screen display 2000 includes a nother image control 2030 which displays a schematic diagram 2032 for the chcuit of chcuit board 2014.
  • the components represented on schematic diagram 2032 also have hot spots 2036.
  • Hot spots 2036 also include index fields which include the names of the components to which they relate.
  • An author can readUy author an apphcation which, when a user chcks on one of hot spots 2020 on chcuit board 2014, highhghts the hot spot 2036 on schematic diagram 2032 which corresponds to the same component 2018, or vice-versa. This may be done by hnking the "Chck" action of hot spot controls 2036, 2020 with a process which runs a method which retrieves the component name from the hot spot control and then locates aU other hot spot controls which are indexed as belonging to the same component by scanning table 1140. The method then highhghts aU hot spot controls 2036, 2020 which are associated with that component 2018 (by setting the appropriate properties in table 1140) and makes aU other hot spot controls non-highlighted.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un système de médiatisation et de publication pour des opérations multimédias, ayant un système intégré de gestion de projets, permettant de gérer et de surveiller l'entrée du support, la gestion de la base de données, leur médiatisation, ainsi que les opérations des sous-systèmes de sortie. Le sous-système de gestion de la base de données enregistre tous les éléments d'une application dans une base de données, ce qui élimine le besoin de suivre les noms des trajets, pour les éléments des supports. Les éléments de supports et les éléments de programmes pour les opérations multimédias sont enregistrés dans la même base de données et ils peuvent être mis à disposition de tous les utilisateurs sur un réseau d'ordinateurs. Plusieurs auteurs peuvent médiatiser un seul travail.
PCT/CA1997/000039 1996-01-18 1997-01-20 Systeme de mediatisation et de publication pour des applications ordinateur multimedias interactives WO1997026608A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU12984/97A AU1298497A (en) 1996-01-18 1997-01-20 Authoring and publishing system for interactive multimedia computer applications

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US1021496P 1996-01-18 1996-01-18
US60/010,214 1996-01-18
US59708796A 1996-02-05 1996-02-05
US08/597,087 1996-02-05

Publications (1)

Publication Number Publication Date
WO1997026608A1 true WO1997026608A1 (fr) 1997-07-24

Family

ID=26680919

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA1997/000039 WO1997026608A1 (fr) 1996-01-18 1997-01-20 Systeme de mediatisation et de publication pour des applications ordinateur multimedias interactives

Country Status (2)

Country Link
AU (1) AU1298497A (fr)
WO (1) WO1997026608A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2335125A (en) * 1998-03-04 1999-09-08 Ibm System and method for creating or editing mutimedia presentation
WO2001013287A1 (fr) 1999-06-11 2001-02-22 Cci Europe A/S Systeme informatique de gestion de contenus destine a la gestion d'objets a contenu d'edition
WO2001059607A2 (fr) * 2000-02-08 2001-08-16 Audioshock Inc. Procede, appareil et systeme d'integration de fichier de divertissement et d'informations correspondantes
WO2002003697A1 (fr) * 2000-07-06 2002-01-10 Centre National D'etudes Spatiales (C.N.E.S.) Dispositif serveur de telerealisation multi-realisateur en libre-service et de telediffusion et reseau de television
EP1742477A1 (fr) * 2002-04-30 2007-01-10 Zootech Limited Creation d'un contenu audiovisuel
US8543977B2 (en) 1999-06-03 2013-09-24 Het Han Fund Pte., Llc General purpose interpreter and database for accessing enterprise servers over an internet protocol network

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0578391A2 (fr) * 1992-07-01 1994-01-12 Wnm Ventures Inc. Constructeur en temps réel de fichiers de données
WO1994008310A1 (fr) * 1992-10-01 1994-04-14 Quark, Inc. Gestion et coordinateion d'un systeme de publication
WO1994028480A1 (fr) * 1993-05-24 1994-12-08 Media Station, Inc. Procede et systeme de developpement multimedia interactif
EP0669587A2 (fr) * 1994-02-24 1995-08-30 AT&T Corp. Système de réseau pour la visualisation des présentations multimédia

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0578391A2 (fr) * 1992-07-01 1994-01-12 Wnm Ventures Inc. Constructeur en temps réel de fichiers de données
WO1994008310A1 (fr) * 1992-10-01 1994-04-14 Quark, Inc. Gestion et coordinateion d'un systeme de publication
WO1994028480A1 (fr) * 1993-05-24 1994-12-08 Media Station, Inc. Procede et systeme de developpement multimedia interactif
EP0669587A2 (fr) * 1994-02-24 1995-08-30 AT&T Corp. Système de réseau pour la visualisation des présentations multimédia

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2335125A (en) * 1998-03-04 1999-09-08 Ibm System and method for creating or editing mutimedia presentation
US8543977B2 (en) 1999-06-03 2013-09-24 Het Han Fund Pte., Llc General purpose interpreter and database for accessing enterprise servers over an internet protocol network
WO2001013287A1 (fr) 1999-06-11 2001-02-22 Cci Europe A/S Systeme informatique de gestion de contenus destine a la gestion d'objets a contenu d'edition
WO2001059607A2 (fr) * 2000-02-08 2001-08-16 Audioshock Inc. Procede, appareil et systeme d'integration de fichier de divertissement et d'informations correspondantes
WO2001059607A3 (fr) * 2000-02-08 2004-01-15 Audioshock Inc Procede, appareil et systeme d'integration de fichier de divertissement et d'informations correspondantes
WO2002003697A1 (fr) * 2000-07-06 2002-01-10 Centre National D'etudes Spatiales (C.N.E.S.) Dispositif serveur de telerealisation multi-realisateur en libre-service et de telediffusion et reseau de television
US7454774B2 (en) 2000-07-06 2008-11-18 Centre National D'etudes Spatiales (C.N.E.S.) Self-service multiple-director remote production and programme broadcasting server device and television network
EP1742477A1 (fr) * 2002-04-30 2007-01-10 Zootech Limited Creation d'un contenu audiovisuel

Also Published As

Publication number Publication date
AU1298497A (en) 1997-08-11

Similar Documents

Publication Publication Date Title
US5748956A (en) Method and system for managing multimedia assets for proper deployment on interactive networks
US7054878B2 (en) Context-based display technique with hierarchical display format
US7194692B2 (en) Method and apparatus for binding user interface objects to application objects
US9348946B2 (en) Interface and method for exploring a collection of data
US6851089B1 (en) Software application and associated methods for generating a software layer for structuring semistructured information
EP0937288B1 (fr) Procede et appareil pour organiser et traiter des informations a l'aide d'un ordinateur numerique
US6240412B1 (en) Integration of link generation, cross-author user navigation, and reuse identification in authoring process
EP0793824B1 (fr) Interface graphique definissable par l'utilisateur servant a acceder a des informations dans un systeme de fichiers electronique
DE69400204T2 (de) Ladesystem
US5442746A (en) Procedural user interface
US20050060277A1 (en) Computer systems and methods for platform independent presentation design
US20050188304A1 (en) Method and system for browsing a low-resolution image
US20010032084A1 (en) Multimedia information structuring and application generating method and apparatus
US20060195426A1 (en) Information management apparatus, information management method, and information management program
US20030212984A1 (en) Sequence analysis method and apparatus
US9110942B2 (en) Programmer interface for manufacturing execution system
US20040139100A1 (en) Context-based display technique
WO1997026608A1 (fr) Systeme de mediatisation et de publication pour des applications ordinateur multimedias interactives
Hoppe Integrated management of technical documentation: the system SPRITE
Pollitt et al. WebKB-GE—A visual editor for canonical conceptual graphs
Newmarch Using tcl to Replay Xt Applications
KR100278667B1 (ko) 데이타 베이스 파일의 생성 및 등록방법과 그 참조방법
Müller et al. Soundium’s design tree: Supporting multimedia composition and performance
CN1384417A (zh) 自动分类的输入/输出系统和方法
Kashyap Logging and TraceBack of a graphics package

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK TJ TM TR TT UA UG US UZ VN AM AZ BY KG KZ MD RU TJ TM

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): KE LS MW SD SZ UG AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

NENP Non-entry into the national phase

Ref country code: JP

Ref document number: 97525556

Format of ref document f/p: F

122 Ep: pct application non-entry in european phase