JPH09134272A - Object-oriented method and apparatus for processing of multimedia data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To control the multimedia devices in a human-based and object- oriented form by leading the multimedia data to a 2nd multimedia device related to a 2nd multimedia object from a 1st multimedia device related to a 1st multimedia object. SOLUTION: The multimedia devices are controlled through the display of icons 228 to 238 of a display 226 which are generated by a graphic user interface. That is, a compact disk player, a microphone, a speaker, a mixer, a splitter and a reverberation filter open the objects in response to the icons 228, 230, 232, 234, 236 and 238 respectively and select two objects. These selected object are led with control to the speaker serving as a 2nd multimedia device from the compact disk player serving as a 1st multimedia device. Thus the multimedia devices can be controlled in a human-based and object-oriented form.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention
Computer systems, especially object-oriented graphics for controlling and connecting multimedia
Regarding the user interface.

[0002]

2. Description of the Related Art Recently, there is a tendency to create and use computer programs in an object-oriented environment. In such an environment, objects make use of the concept of representing items in a general way. Each object contains data and operations called methods that operate on the data. Such data and operations are treated as a unit. One feature of object-oriented programming is that the data contained in an object can only be accessed by the containing method. That is, the data can only be modified by consistent methods predefined by the program interacting with the object. This feature is called encapsulation.

Items that can be represented by multiple objects include telephones and speakers, and common real-world office objects such as computer peripherals, computer programs, and documents. Encapsulating the text of a word processing document is an example of such an object (hereinafter referred to as an application object). The object may include methods that perform its function and automatically call the word processing program and, after selecting the object, cause the word processing program to load it.

Object-oriented graphic user
In the interface, each application object typically contains methods for displaying the object on a visible display device such as a monitor. Generally, such display is usually in the form of an icon which is a bitmap of a predetermined size, and the appearance of the icon is defined by the bitmap.

Graphical user interfaces typically use a pointing device such as a mouse to select object icons. The mouse can point to the icon by the user generating and controlling a pointer graphic on the display. When the pointing device is moved, the pointer graphic is also moved, and the user can move the pointer graphic on the icon on the display.
Once the pointer graphic is over the desired icon, the icon can be "selected" by manipulating a button or other selection device.

Alternatively, the icon can be moved by using the pointing device and the select button. In this kind of operation called “drag and drop”, first select the icon, as described above,
The pointing device is then moved while actuating the select button. Such a combination of operations causes the icon to follow the pointer graphic on the display device. When you release the select button, the icon will appear in its new position.
In this way, one icon can be dragged and dropped on top of another to cause two underlying objects to interact.

Object-oriented graphic user
The interface allows the pointer and selection device to be used to call methods of the application object corresponding to the displayed icon. For example, if you select a particular icon, the interface will
Typically, it sends a message to the underlying object indicating that it has selected the icon. In response to this message, one of the corresponding plurality of object methods may be invoked. Similarly, double-click on the icon (place the pointer graphic on the icon and activate the select button twice within a predetermined time interval) to generate another message and use this message You can also call the other of the object methods.

Still other techniques can be used to call object methods. The selected icon area can be called, for example, a "hot spot". The hot spots of the two icons may be connected to each other by a line segment on the display. The connection then causes the multiple messages to be forwarded to the underlying objects and subsequently causes these objects to execute the selected method.

Multimedia has been a rapidly emerging field in personal computing with the trend toward object-oriented environments. For example, multimedia devices include compact disc players, laser disc players, VCRs, audio devices and digital audio tape players.
In multimedia systems, these devices
Typically, it is directly or indirectly connected to a computer system that includes a display device with audio speakers driven by a specialized sound board. The computer digitizes and stores both visual and audio information and controls the operation of various devices to form an integrated system for storing and simultaneously reproducing various information segments. Thus, it is possible to display digital video on the display screen while playing the synchronous digital audio segment from the speaker.

Conventionally, each multimedia device is
It was working as a component of the alone. First, the user controls each device from the physical control panel mounted on the multimedia device. Later, multimedia devices became more integrated into computer systems and it became possible to control the devices by using commands entered into the computer. With the advent of the graphic user interface, computers can be used to control underlying devices such as control panels.
It is now possible to generate graphics on the screen.

However, multimedia devices still operate in a discrete format. For example, when a user added a reverberation effect to an acoustic segment, it was necessary to first record the acoustic segment using a recorder, then digitize the acoustic segment and store it in an acoustic file. Therefore, I was able to search the audio file and edit it to add a reverberation effect. Finally, the edited file could be played back by the playback device.

[0012]

Therefore, the method of controlling a multimedia system was limited to emulating device operation due to the stand-alone prior art form which was essentially non-integrated. Still, there is a need to fully realize the power of this multimedia system.

It is an object of the present invention to allow users to control multimedia devices in a more human-centered, object-oriented format.

It is an object of the present invention to provide a computer system in which multimedia devices can be dynamically connected and controlled.

Another object of the present invention is to allow a user to control an independent multimedia device in a user-friendly integrated environment.

It is a further object of the present invention to provide an intuitive connection mechanism for connecting multimedia devices and to process multimedia files in real time.

[0017]

According to the present invention, a computer system is responsive to a user operating a plurality of icons representing a plurality of multimedia devices on a graphical user interface. Can connect and control multiple multimedia devices as desired by simply manipulating multiple icons representing multiple multimedia devices displayed on the graphical user interface.

The present invention is applicable to both video and audio as well as other types of data such as tactile sensor data, and represents a first multimedia device to which a user should simply be connected to a mouse pointing device. Selecting an icon and dragging and linking the mouse cursor to the icon pointing to the second multimedia device facilitates connecting multiple multimedia devices through the computer system. A connecting line is drawn on the display as the mouse cursor is moved until the user clicks on the icon representing the second multimedia device he wishes to connect to the first multimedia device. For example, a user can click on the compact disc player icon to link the compact disc player icon to the speaker icon on the graphic user interface. The computer then makes the necessary internal hardware connections to make the connection between the compact disc player and the speakers as shown on the graphic user interface.

Once connected, the user can control the plurality of multimedia devices via a graphical user interface displayed on the computer display. Each independent device may be controlled by selecting the icon associated with it to open the setting mechanism. The configuration mechanism may be implemented by a dialog box, allowing the user to directly control the multimedia device via the graphic user interface. For example, a user may select the compact disc player and speaker, or any other multimedia device in the computer system, by simply selecting the device's icon and operating multiple dialog box controls. Controllable, each dialog box setting a device-controllable parameter (eg, volume, balance, bass, ...) Displayed on the graphic user interface. It is a thing.

Further, the present invention allows a user to control the presentation of multimedia information from a computer system in near real time. For example, if the computer system has a graphic equalizer that routes the audio signal before outputting it to a set of speakers, the user may adjust the frequency response of the equalizer through the graphic user interface to send the audio. It can be pre-treated before. This pre-processing may be performed by simply selecting the icon associated with the equalizer and opening the associated object, for which the user controls the equalizer to pre-process the sound being played by the speaker. A dialog box that can be included may be included.

In addition to this, the user may further edit (ie, pre-process) the signal using filters, mixers or other known devices. As used herein, the term "pre-processing" includes signal processing steps that modify the characteristics of the received data signal. For example, pre-processing may include mixing the received data signal with other signals, or filtering or delaying the received signal before providing the processed signal to the user.

The user disconnects the device from the computer system by connecting and controlling various multimedia devices, and by simply disconnecting the displayed icon that is connected to the computer system via the graphical user interface. You can also The computer system will then perform all the hardware disconnection needed to actually perform the disconnect.

The present invention provides a multimedia user interface operable by a user through a graphic user interface.
It is intended to quickly and easily interconnect multiple multimedia devices in a system to control multiple multimedia devices via a single graphic user interface.

The foregoing and further advantages of the present invention will be better understood with reference to the following description in connection with the accompanying drawings.

[0025]

DETAILED DESCRIPTION OF THE INVENTION FIG. 1 shows the IBM PS / 2 ™.
1 illustrates a system architecture for a typical computer system such as a personal computer (PC). The example computer system of FIG. 1 is for illustration only. Although the following description may refer to terms commonly used in describing a particular computer system such as an IBM PS / 2PC, the description and concepts include systems having an architecture that is not identical to FIG. Apply equally to other systems.

The illustrated computer 100 includes a central processing unit (CPU) 105, which includes a conventional microprocessor, system random access memory (RAM) 110 for temporary storage of information, and permanent storage of information. A read-only memory (ROM) 115 for use with the device may be included. A memory controller 120 is provided to control the RAM 110, a bus controller 125 is provided to control the bus 130, and an interrupt controller 135 is used to accept and process various interrupt signals.

Mass storage may also be provided by diskette 142, CD-ROM disk 147 or hard disk 152. The diskette 142 is adapted to be inserted into the diskette drive 141, and the diskette drive 141 is further connected to the bus 130 by the controller 140. Similarly, the CD-ROM disc 147 is designed so that it can be inserted into the CD-ROM drive 146.
The drive 146 is also connected to the bus 130 by the controller 145.
It is connected to the. Finally, the hard disk 152 is a part of the fixed disk drive 151, and the fixed disk drive 151 is connected to the bus 130 by the controller 150.

Input / output to / from the computer 100 is performed by a large number of devices. For example, the key board and mouse controller 155 may be connected to the bus 13 to control the key board input device 156 and the mouse input device 157.
Connected to 0. A DMA controller 160 is provided to perform direct memory access to RAM 110. A visual display is generated by video controller 165, which controls video output display 170. The video output display 170 produces, under the control of the computer 100, pixels of a two-dimensional array that may be independently controllable to form an image. Other input / output devices such as audio, video or special purpose cards may be connected to the computer system via expansion slots 190 and are shown here as multimedia devices 80-83.

Generally, computer 100 is an O available from International Business Machines Corporation (IBM) of Boca Raton, Florida.
It is controlled and coordinated by operating system software such as S / 2 ™. The operating system is responsible for processing execution and scheduling.
It provides resource management for the entire computer system including tasks such as memory management file system services, networking and scheduling services and input / output services, and user interface provisioning. Users such as editors and spreadsheets
Applications depend directly or indirectly on these and other features of the operating system.

In addition to this, modern computers store and organize various forms of data, including text data digital audio data, digital video data and digital music data (eg MIDI). And is increasingly using the multimedia technology it presents. For example, a computer using multimedia technology plays video and audio data and synchronizes the video output display 17 with the audio output from the multimedia devices 80-83.
A movie clip video can be generated on the 0.

A user is an object oriented graphic user generated by the operating system, controlled by an application program, and displayed on the video output display 170 to control the multimedia devices 80-83 in accordance with the present invention. The computer 100 is controlled via the interface.

Object Oriented Programming (OO
P) Technology involves the definition, creation, use and deletion of "objects", as will be appreciated by those skilled in the art. These objects are software entities that contain data elements and routines or methods that operate on the data elements. The data and associated functionality may be treated by the software as an entity and created, used and deleted as if it were an item. Together, data and functionality enable virtual modeling of real-world entities by virtue of the characteristics that an object can represent by a data element and the behavior that can be represented by its data manipulation functions.

A plurality of objects, although not the objects themselves, are defined by creating a "plurality of classes" that act as templates to instruct the program compiler how to construct the actual object. For example, a class may specify the number and type of data variables and the steps associated with the functions that manipulate the data. An object is actually an object-oriented program at runtime due to a special function called a constructor that constructs that object using the corresponding class definition and additional information such as arguments provided during object creation. Created by. Memory space is allocated for object data and methods at creation. Similarly, multiple objects are deleted at run time by a special feature called a destructor. Deletion of the object frees the allocated memory. Multiple objects may be used with data and by calling these functions.

The main advantages of object-oriented programming techniques come from three basic principles: encapsulation, polymorphism and inheritance. In particular, multiple objects may be designed to hide or encapsulate all or some of the internal data structures and functionality.
In particular, during program design, the program developer can define objects in which all or part of the data variables and all or part of the associated functionality are private or only the object itself uses them. . Other data or functionality can be declared available by public or other programs. Access to private variables by other programs can be controlled by defining public functions for the object that access the object's private data. Public functions form a controlled, consistent interface between private data and the "outer" world. Attempting to write program code that directly accesses private variables causes the compiler to error stop the compilation process during program compilation, causing the compiler to generate an error that prevents program execution.

Polymorphism is a concept that allows multiple objects and multiple functions that have the same overall shape but operate on different data to perform different functions in order to produce consistent results. Is. For example, the add function may be defined as variable A + variable B = (A + B), and the same format can be used whether A + B is a number, a letter, or dollars and cents. However, the actual program code that performs the addition is
Depending on the type of variable consisting of A and B, it may differ significantly. Polymorphism allows writing three independent function definitions, one for each type of variable (number, letter and dollar). After multiple functions have been defined, the program can later call the add function in its general format (A + B), and during compilation, the C ++ compiler actually looks up the variable type to get one of the three functions. Determine which one you are using. The compilation then replaces the correct function code Polymorphism causes similar functions that produce similar results to be "grouped" by the program source code to produce a more logical and well-defined program flow.

A third basic principle underlying object-oriented programs is inheritance, which allows existing programs to be easily reused and prevents scratches in the software. The inheritance policy allows for declaring multiple classes (and objects subsequently generated from these classes) that are considered relevant to the software developer. In particular, multiple classes may be represented as subclasses of other base classes. Subclasses are "inherited"
Has access to all the public functionality of the base class, just as if they appeared in a subclass. Alternatively, some or all of the inherited features may be ignored, or some or all of the inherited features may be modified by simply defining a new feature with the same format (ignore or change). Does not change the function in the base class, but simply changes the use of the function in subclasses). Creating new subclasses with some functionality of other classes (by selective modification) allows software developers to easily adapt existing code to meet their particular needs.

Although other object-oriented programming improves other program concepts, program development can be quite time consuming and laborious, especially when modifying existing software programs. Thus, the prior art processing method creates a set of predefined objects and a set of pre-defined routines that the program developer is instructed to perform the tasks normally found in a particular environment. Was to provide an interconnect class that did. Such pre-defined classes and libraries are typically
Called the "application framework," it essentially provides a prebuilt structure for working applications.

For example, an application framework for a user interface would provide a set of predefined graphic user interface objects that create windows, scroll bars, menus, etc. User·
Provides support for interface objects and "default" behavior. Since the application framework is based on object-oriented technology, it is possible to use a pre-defined class as a base class and the built-in default behavior can be inherited by developer-defined subclasses, transformed or ignored. Enable developers to extend the framework and create customized solutions in specific areas of expertise. This object-oriented solution offers significant advantages over traditional programming because the programmer does not change the original program, but rather extends the functionality of the original program. In addition to this, the framework provides architectural guidance and models while at the same time freeing developers to supply special operations specific to the problem, allowing them to work blindly according to layers of code. Disappears.

There are many types of application frameworks available depending on the level of system involved and the type of problem to be solved. Framework types range from high-level application frameworks that support the development of user interfaces to low-level frameworks that provide basic system software services such as communication, printing, file systems, etc. Up to. A commercial example of an application framework is Apple Computer, Inc.
Computer, Inc.) MacApp, Symantec (Symant
Bedrock and Borland International Inc. by ec Corpotation).

According to the invention, a plurality of multimedia
A multimedia application framework containing objects is provided. A particular multimedia object is associated with each type of multimedia device. Each multimedia object contains an icon that is displayed when the object is created. For each multimedia icon, an internal method for controlling the associated multimedia device, setting parameters for operating the multimedia device together, guiding input / output of the multimedia device, and controlling operation of the multimedia device. Is included. Regarding the relationship of the devices related to these,
This will be described in detail below.

FIG. 2 shows an object-oriented graphic
27 illustrates an exemplary display 226 generated by a user interface. In addition to the normal window area 210 associated with an application program, representations of various selectable objects are represented by icons 228 ...
It is displayed in the format of 238. Each icon 228-2
38 is associated with one underlying object, which in turn controls the multimedia device. Such devices include, for example, compact disc players, microphones, speakers, mixers, splitters, and reverberation filters, which have icons 228, 230, 232, respectively.
234, 236 and 238 correspond. In addition, other multimedia devices may also have associated objects with different icons (not shown) for digital audio tape players / recorders. Each icon 22
8 to 238 have connection points or "hot spots" that can be used to connect icons, as described in detail below, which in turn connect these icons between the underlying multimedia devices. I do. Some icons have only one hotspot, which represents simply receiving or generating data, while other icons have two hotspots, both of which receive input. It also indicates that an output is generated. For example, the reverberation icon 238 has an input hot spot icon 237 and an output hot spot icon 238.

Further, by the usual method, the graphic
The user interface also includes pull down menu icons 242, 243 and communication control icons 244-247 for various multimedia devices represented by a plurality of icons. The icons for communication control include a stop icon 244, a pause icon 245, a playback icon 246, and a rewind icon 247 that the user can select via the mouse 114 or the keyboard 116. ing. Additionally, additional icons may be used to represent multimedia files or recording devices for recorded data in files such as icon 240 for output files.

The underlying multimedia device is controlled by manipulating the displayed icons to control the associated object. To manipulate an object, select the relevant object, open the object, and 2
It can take the form of connecting two objects and supplying the output of one object to the input of another. Additionally, the multimedia device may be controlled by common control icons 244-247. These latter icons operate in conjunction with the selected object and control the underlying device in the manner described below.

FIG. 3 is a flow chart of a routine for opening and selecting an object displayed on the graphic user interface. The routine shown in FIG. 3 and here below the routine is preferably
It is part of each object. The routine shown in FIG.
According to a typical object oriented program, it would be contained in a parent or base class containing routines common to all multimedia objects.

This routine starts at step 300. At step 300, the user determines whether an object has been selected. As mentioned above, an object can be selected in various ways. One typical method is to place a pointer graphic on the icon associated with the object. The latter two operations cause the graphical user interface to forward the message to the object associated with that icon. This object may change the appearance of the icon to visually indicate the selection in response to the message.

When no object is selected,
The routine is completed by step 320. Alternatively, if the object is selected by the determination at step 302, the routine proceeds to step 304 to determine if the object has been opened. Opening an object is as easy as, for example, placing the pointer graphic on the icon to select the object and double-clicking the select button on the mouse. Also, other operations can be used to open related objects. For example, an object may be selected and then opened using the Open Object menu command 243.

When a command is issued to open the object, as determined in step 304, a subroutine is called to display the dialog box associated with the object opened by the user. Such a dialog box has a push button, as described in detail below.
Multiple graphic controls that may be operated by the user, such as buttons, dials, sliders, potentiometers, etc., may be included. These graphic controls invoke methods on the object to adjust the parameters of the underlying multimedia device, respectively.

FIG. 4 is a more detailed flow chart of a subroutine 306 for manipulating parameters of a multimedia device associated with a user generated object.
Generally, an icon 2 for each multimedia object
28-238 is specific to the underlying multimedia object type and is its own dialog box (or multiple dialog boxes) that is displayed on the graphical user interface when the user opens the object. Have. The dialog box routine begins at step 400 and proceeds to step 40.
Proceeding to step 2, step 402 causes the dialog box routine on the opened object to be called to display the dialog box screen for. Generally, a dialog box routine creates a dialog box object, which then causes the display of the dialog box screen.

In step 404, the user sets the parameters of the multimedia device associated with the opened object in response to the user's operation of one of the control icons in the dialog box. Operation and control of such icons is well known to those skilled in the art. Subsequently, each control icon includes a method of setting the parameters of the associated object. Thus, the manipulating control causes a message to be sent to the control object, which then uses one of the included methods to actually set the device parameters.

After manipulating the control, step 406 determines if the dialog box should be closed. Generally, the dialog box is closed in response to a command generated by the user. Such a command may take the form of operating, for example, an "OK" push button, an "Cancel" push button or a dialog box window box selection. When you close the dialog box, you erase the dialog box object and all control objects.

If no command has been received and the dialog box is closed, from step 406 the routine loops and re-executes steps 404 and 406 until the user closes the dialog box.

FIGS. 5 and 6 show exemplary dialog boxes 570, 572 that may be used to set the parameters of the amplifier / speaker device and the compact disc player device, respectively. Dialog box 5 when the speaker object (represented by the speaker icon 232 in FIG. 2) is opened.
When 70 is displayed and the compact disc player (represented by the compact disc icon 228 in FIG. 2) is opened, a dialog box 572 is displayed.

The control object in the speaker dialog box 570 is the volume 74, the bus 7
Graphic for 8, Treble 76 and Balance 80
Represented by a potentiometer icon. The user is typically able to manipulate these controls to control (ie, process) the sound from the associated amplifier / speaker by selecting the control and dragging the control to the desired position. . As mentioned above, manipulation of a control icon, such as the volume icon 74, sends a message to the associated control object. In response, the control object causes the speaker amplifier to supply the necessary commands using one of the methods of controlling the multimedia device (FIG. 1) to adjust the speaker volume.

Similarly, FIG. 6 shows a dialog box 57.
2 and this dialog box may be used to control the exemplary compact disc player represented by the compact disc icon 228 (FIG. 2). The control icon associated with dialog box 572 is a plurality of push buttons 591-59.
8 By placing the pointer graphic 592 over the selected button, when one of the push buttons 591-598 is selected, the associated control object causes one track on the compact disc to be selected for playback. Call the method.

Returning to FIG. 3, at decision block 304, the user makes a determination that the object is not open, and at decision block 308 the object.
The determination is made as to whether or not a hot spot on the icon has been selected, and this hot spot indicates whether or not the user has selected an object for connecting to another object.

When a hot spot is selected, the routine proceeds to step 310, which determines whether the select button remains depressed. If the button remains depressed, proceed with the drag operation. In the latter case,
As shown in step 312, a method that draws a line from the selected hot spot in the underlying object to the position of the pointer graphic is called. This line can also be drawn by creating a separate connection object and calling the method for drawing the line on the connection object. The line connecting the selected hotspot to the pointer graphic will continue to be drawn as long as the select button remains depressed.

Upon determining in step 310 whether the select button has been released, decision block 314 makes a further determination whether the pointer graphic is located on a hotspot on another target icon. To do. if,
If yes, the routine proceeds to step 318, where a line is drawn from the hotspot in the source icon to the hotspot in the target icon, indicating that the two underlying devices are connected together. In addition to this, a method is called in the target object that returns the address of the target object data input. This address is then provided to the source object, which directs the output of the underlying multimedia device to the input of the target device.

Alternatively, at decision block 314, if the pointer graphic is not located on the hotspot on the target icon when the select button is released, then at step 316 the source icon and pointer graphic are selected. The lines connecting the selected hot spots are erased and completed in step 320.

To illustrate how various multimedia devices can be quickly and easily connected using an object-oriented graphic user interface, FIG. 7 shows an object-oriented graphic. A compact disc player icon 62 showing a user interface 600 and operably connected to a speaker icon 632.
8 is displayed. To make this connection, the user simply positions the pointer graphic on the hotspot 629 of the compact disc player 628 and depresses the select button to select the hotspot. Then, while holding down the select button, the user moves the pointing tool and drags the pointer graphic to the hot spot 631 of speaker icon 632.

Releasing the select button draws line 635 to connect the two hot spots. The connect routine shown in FIG. 3 then causes the output of the compact disc player multimedia device to actually connect to the input of the speaker amplifier in the underlying object to allow the compact disc player to play through the speaker. Invoke the method required to do this. Instead, a compact disc player icon 62
8 output may be directed to a sound file by connecting the output hot spot 629 of the icon 628 of the compact disc player to the input hot spot 641 of the sound file. This latter connection opens a new sound file and causes the recording device to begin receiving input from the compact disc player device.

When the connection between the compact disc player and the speaker is established, the control icons 644-6
The player can be controlled by 47. For example,
By placing the pointer graphic on the icon of the compact disc player and pressing the select button,
A compact disc player can be selected. The play icon 646 can then be selected, causing the method in the underlying control object to execute and start the compact disc player device. Stop icon 6 with similar method
The player can be stopped by selecting 44. Furthermore, the player can be paused by selecting the pause icon 645. Rewind icon 6
The player playing by selecting 47 can be returned to the beginning of the selection.

The present invention is not limited to operably connecting only two multimedia devices, but any number of compatible multimedia devices may be connected to create a complete multimedia system. it can. For example, FIG. 8 shows a plurality of icons on a graphical user interface 710 operably connected to form a "karaoke" system. Object-oriented graphic user interface 710
Shows icons 728-740 of various karaoke systems operably connected. Icons 728 and 730, which represent a compact disc player and a microphone device, respectively, are connected to a mixer icon 734, which is a compact icon.
The background music output from the disc player device is mixed or combined with the user / singer's audio output output from the microphone device. The mixed signal is then reverberation filter and recorder / output file 740 represented by icon 738.
The singer can make the recording a souvenir. These may be added by connecting other devices, such as equalizer 748, in series with the remaining components.

A component may be detached by pressing the pointer graphic across the lines, pressing the select button, and simply selecting the connecting line. Once selected, the line may be deleted by activating a further control such as the "Delete" key. Actuation of the "Delete" key then calls one of the object methods that breaks the connection between the multimedia devices in the underlying multimedia object.

When the user wants to increase the volume output from the karaoke system speaker, the user simply opens the speaker object represented by icon 732, and also the speaker dialog box 5
72 (FIG. 6) will appear on the graphic user interface. The user then increases the volume or adjusts the audio as desired. Similarly, the user simply opens the compact disc player object represented by icon 728 and plays the compact disc player dialog box 570 to play another song for the next karaoke singer.
(FIG. 6) appears through the graphical user interface to allow the user to select the next track on the compact disc to select. Once the song is queued and the singer is ready to sing, the user can begin singing the song by selecting play icon 746. In addition, other control icons 744, 745
And 747 may be selected to stop, pause and rewind the player.

As used herein, the term "preprocessing" refers to any modification of the characteristics of one or more incoming signals, such as mixing two or more signals, adjusting the audio or video characteristics of the signals. Technology, or, in general,
Any other technique of modifying the characteristics of audio, video or other signals in analog or digital form is included.

The present invention can connect the various devices in the system and perform the necessary synchronization between the devices. In addition to this, when a system such as a karaoke system is defined, the present invention allows the user to define and save the entire system including connection and settings of the variable multimedia device as an object and display the icon display of the object. Intended to be able to complete. In this way, connecting and configuring the device to form a karaoke machine may again be performed by simply selecting the user-defined karaoke icon 748.

The invention is illustrated in connection with the preferred embodiment,
Although explained, those skilled in the art can make various other changes, deletions and additions to the formation of the present invention and its details without departing from the spirit and scope of the present invention. Should be understood. For example, the present invention is not limited to use with mice. A user may operate a plurality of icons displayed on the graphical user interface using a trackball, light pen, touch screen or any similar device.

Additionally, although the exemplary embodiment of the multimedia computer system was a personal computer, it should be clear that the invention is not so limited. Computer systems other than personal computers may use the methods and apparatus of the present invention to operably connect and control multiple multimedia devices via an object-oriented graphical user interface.

In another embodiment, the invention is a computer / computer.
It may be implemented as a computer program that uses the system. Such an implementation may be a computer-readable medium such as a diskette 142, CD-ROM.
147 or fixed disk 152 (FIG. 1) fixed on a tangible medium, by optical or analog communication lines, or by intangible use including, without limitation, microwave, infrared or other transmission techniques. The wireless technology employed provides a series of computer readable instructions that can be transmitted to a computer system via a modem or other interface device. The series of computer readable instructions includes all or part of the functionality previously described herein in the context of the present invention. Those skilled in the art should understand that such computer-readable instructions may be written in many programming languages used in many computer architectures or operating systems. Further, such instructions may be stored using current or future memory technologies, including without limitation semiconductor, magnetic, optical or other memory devices, or optical, infrared, microwave, or other. It may be transmitted using current or future communication technology, including without limitation transmission technology. Such computer program products may be distributed as removable media accompanied by printed documents or electronic documents preloaded by the computer system into, for example, a system ROM or fixed disk, or via a network, such as It is contemplated that it may be distributed by a server or electronic bulletin board over the Internet or World Wide Web.

As a summary, the following items are disclosed regarding the configuration of the present invention.

(1) Object oriented method for controlling a plurality of multimedia devices and processing multimedia data in a computer system having a display device and a graphic user interface for generating an icon display on the display device (A) creating a multimedia object associated with each of the plurality of multimedia devices, the plurality of object methods controlling the multimedia device with which the multimedia object is associated; And a step of controlling the graphic user interface to generate an iconic representation of a multimedia object on the display, and (b) the graphic user interface. Controlling the device to visually connect a first icon representing the first multimedia object to a second icon representing the second multimedia object; and (c) the first and second The at least one of a plurality of object methods in the multimedia object to bring the multimedia data from the first multimedia device associated with the first multimedia object to the second multimedia device. .Second associated with objects
Object-oriented method including the step of directing to a multimedia device. (2) at least one of the plurality of multimedia devices has a device parameter for controlling processing of the multimedia device by the at least one multimedia device, and (d) setting the device parameter Creating a control object having one control object method and a second control object method displaying a control icon; and (e) invoking the control object to set the device parameter. The object-oriented method according to (1) above. (3) Step (d) includes (d1) the multimedia associated with the at least one multimedia device.
Opening an object; (d2) creating a dialog box object on the graphic user interface in response to opening the multimedia object in step (d1); (d3) Creating the control object, invoking the first control object method to display the control icon, and (d4) invoking the second control object in response to a user operation of the control icon. The object-oriented method according to (2) above, including the step of setting a parameter. (4) Step (b) includes: (b1) invoking one of the plurality of object methods in the first and second multimedia objects to control the graphic user interface, 1
The object-oriented method according to (1) above, which includes a step of drawing a line connecting the icon of (1) to the second icon. (5) Step (b) further comprises (b2) creating a connection object having an object method for controlling the graphic user interface to connect the first icon to the second icon. The object-oriented method according to (1) above, including the steps of: (b3) calling a connection object method. (6) A computer program product for use in a computer system having a display device for displaying a graphic user interface, the computer program product including computer readable program code means for controlling a plurality of multimedia devices. Feasible media and program code means for creating an object associated with each of the plurality of multimedia devices, the object comprising a plurality of object methods for controlling the associated multimedia device, and the graphic user. A program code means for controlling an interface to generate an iconic representation of the object on the display device; and a first user interface for controlling the graphic user interface. Program code means for visibly connecting a first icon representing a media object to a second icon representing a second multimedia object; and a plurality of said plurality of said first and second multimedia objects. Invoking at least one of the object methods to transfer multimedia data from a first multimedia device associated with the first multimedia object to a second multimedia device associated with the second multimedia object. A computer program product including a program code means for a media device. (7) At least one of the plurality of multimedia devices has a device parameter for controlling processing of the multimedia data by the at least one multimedia device, and the program for creating the object, The code means further comprises a program for creating a control object having a first control object method for setting the device parameter and a second control object method for displaying a control icon.
The computer program product according to (6) above, including code means and program code means for calling the control object frame to set the device parameters. (8) The program code means to be created further comprises:
Program code means for opening a multimedia object associated with the at least one and creating a dialog box object on the graphic user interface in response to opening the multimedia object. Program code means, program code means for creating a control object and invoking the first control object method for display on a control icon, and second program code means in response to a user operation of the control icon.
The computer program product according to (7) above, which comprises a program code means for calling the control object method of (1) to set the device parameter. (9) The program code means for controlling the graphic user interface calls one of a plurality of object methods in the first and second multimedia objects to call the graphic user interface. To control the first
The computer program product according to (6) above, including a program code means for drawing a line connecting the icon of (1) to the second icon. (10) A computer system comprising a display device, a graphic user interface for generating an icon display on the display device, and an instruction device under the control of an application program executed on the computer system, the computer system comprising: A device for controlling a multimedia device to process multimedia data, the device being controlled by the application program to create a multimedia object associated with each of the plurality of multimedia devices. An object on the display device, the object having at least one object method for controlling the associated multimedia device, and a method for controlling the graphic user interface. And said means for generating the icon display of the defect, the in cooperation with the pointing device graphical
Connection means for controlling a user interface and visually connecting the first and second icons, the first icon representing the first multimedia object and the first icon representing the second multimedia object. In response to connecting to two icons, invoking at least one object for each of said first and second multimedia objects,
Means for directing data from a first multimedia device associated with the first multimedia object to a second multimedia device associated with the second multimedia object. (11) At least one of the first and second multimedia devices has device parameters for controlling processing of the multimedia data by the at least one multimedia device, and further controlled by the instructing device. Means for opening one of the plurality of multimedia objects, the multimedia
Means for creating a control object having a first control object method for setting the device parameter and a second control object method for displaying a control icon in response to opening one of the objects; The device according to (10) above, further comprising means for calling the control object method that is controlled by the indicating device and sets the device parameter. (12) The means for creating is controlled by the pointing device to open a multimedia object associated with the at least one multimedia device; and the graphic user in response to the multimedia object. Means for creating a dialog box object on the interface, and means for creating the control object in response to creating the dialog box object and invoking a first control object method for displaying a control icon And a unit controlled by the pointing device to call a second control object method and set the device parameter in response to a user operation of control.

[Brief description of the drawings]

FIG. 1 is a block diagram of a functional embodiment of a data processing system embodying the present invention.

FIG. 2 shows an example of a screen display generated by an object-oriented graphic user interface, displaying various selectable icons, some of which represent multimedia devices.

FIG. 3 is a flowchart illustrating a routine for opening and selecting an icon displayed on an object-oriented graphic user interface.

FIG. 4 is a flowchart illustrating a subroutine for handling a multimedia device associated with a selected icon.

FIG. 5 is a diagram of a dialog box display on a graphical user interface for a speaker.

FIG. 6 is a diagram of a dialog box display on a graphic user interface for a compact disc player.

FIG. 7 is a diagram showing two icons connected and displayed on an object-oriented graphic user interface.

FIG. 8 Graphic to form a karaoke machine
It is a figure which shows the some icon connected and displayed on the user interface.

[Explanation of symbols]

 80-83 multimedia device 105 computer 110 system RAM 115 ROM 165 video controller

[Procedure amendment]

[Submission date] August 27, 1996

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction item name] 0053

[Correction method] Change

[Correction contents]

The control object in the speaker dialog box 570 is the volume 74, the bus 7
Graphic for 8, Treble 76 and Balance 80
Represented by a potentiometer icon. The user is typically able to manipulate these controls to control (ie, process) the sound from the associated amplifier / speaker by selecting the control and dragging the control to the desired position. . As mentioned above, manipulation of a control icon, such as the volume icon 74, sends a message to the associated control object. In response, the control object causes the speaker amplifier to supply the necessary commands using one of the methods of controlling the multimedia device (FIG. 1) to adjust the speaker volume. ────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] September 11, 1996

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] Brief description of drawings

[Correction method] Change

[Correction contents]

[Brief description of the drawings]

FIG. 1 is a block diagram of a functional embodiment of a data processing system embodying the present invention.

FIG. 2 shows an example of a screen display generated by an object-oriented graphic user interface, displaying various selectable icons, some of which represent multimedia devices.

FIG. 3 is a flowchart illustrating a routine for opening and selecting an icon displayed on an object-oriented graphic user interface.

FIG. 4 is a flowchart illustrating a subroutine for handling a multimedia device associated with a selected icon.

FIG. 5 is a diagram of a dialog box display on a graphical user interface for a speaker.

FIG. 6 is a diagram of a dialog box display on a graphic user interface for a compact disc player.

FIG. 7 is a diagram showing two icons connected and displayed on an object-oriented graphic user interface.

FIG. 8 Graphic to form a karaoke machine
It is a figure which shows the some icon connected and displayed on the user interface.

[Description of Reference Signs] 80-83 multimedia device 105 computer 110 system RAM 115 ROM 165 video controller

Claims (12)

[Claims] 1. A display device and a graphic user for generating an icon display on the display device.
An object-oriented method for controlling a plurality of multimedia devices to process multimedia data in a computer system having an interface, comprising: (a) providing a multimedia object associated with each of the plurality of multimedia devices. A step of creating a plurality of object methods for controlling a multimedia device with which the multimedia object is associated, and controlling the graphic user interface to display an icon of the multimedia object on the display. And (b) a first controlling the graphical user interface to represent a first multimedia object.
Visually connecting a second icon to a second icon representing a second multimedia object, and (c) at least one of a plurality of object methods in the first and second multimedia objects. And directing the multimedia data from a first multimedia device associated with the first multimedia object to a second multimedia device associated with the second multimedia object. Object-oriented methods including. 2. At least one of the plurality of multimedia devices has a device parameter for controlling processing of the multimedia device by the at least one multimedia device, and (d) further comprising: Creating a control object having a first control object method to set and a second control object method to display a control icon, and (e) the control object method to set the device parameter. The object-oriented method of claim 1, including the step of calling. 3. Step (d) includes: (d1) opening the multimedia object associated with the at least one multimedia device; and (d).
2) creating a dialog box object on the graphic user interface in response to opening the multimedia object in step (d1), and (d3) creating the control object, Calling the first control object method to display the control icon; and (d4) calling the second control object and setting device parameters in response to a user operation of the control icon. The object-oriented method of claim 2, including: 4. Step (b) comprises: (b1) invoking one of the plurality of object methods in the first and second multimedia objects to control the graphical user interface, The object oriented method of claim 1, including the step of drawing a line connecting the first icon to the second icon. 5. Step (b) further comprises (b2) controlling the graphic user interface to perform the first step.
2. The object oriented method according to claim 1, further comprising the steps of: creating a connection object having an object method for drawing a line connecting the icon of FIG. 2 to the second icon; and (b3) invoking the connection object method. . 6. A computer program product for use in a computer system having a display device for displaying a graphic user interface, the computer program product including computer readable program code means for controlling a plurality of multimedia devices. A computer usable medium, and program code means for creating an object associated with each of the plurality of multimedia devices, the object comprising a plurality of object methods for controlling the associated multimedia device, and the graphic. Controlling the user interface to generate an iconic representation of the object on the display device, and controlling the graphic user interface The second to the first icon representing one of multimedia object representing the second of multimedia objects
Program code means visibly connected to the icon and at least one of the plurality of object methods in the first and second multimedia objects is invoked to transfer multimedia data to the first multimedia object. Computer program product leading from a first multimedia device associated with a multimedia object to a second multimedia device associated with the second multimedia object. 7. At least one of the plurality of multimedia devices has device parameters that control the processing of the multimedia data by the at least one multimedia device, and creates the object. Program code means further comprises: program code means for creating a control object having a first control object method for setting the device parameter and a second control object method for displaying a control icon; Computer program product according to claim 6, comprising program code means for calling an object method to set the device parameter. 8. The program code means for creating,
Further, program code means for opening a multimedia object associated with the at least one multimedia device, and a dialog on the graphic user interface in response to opening the multimedia object. Program code means for creating a box object; program code means for creating the control object and calling the first control object method to display it on a control icon; and for user operation of the control icon. Computer program product according to claim 7, responsive to program code means for invoking the second control object method to set the device parameters. 9. The program code means for controlling the graphic user interface calls one of a plurality of object methods in the first and second multimedia objects to call the graphic user. 7. A computer program product as claimed in claim 6 including line drawing program code means for controlling an interface and connecting the first icon to the second icon. 10. A computer system having a display device, a graphic user interface for generating an icon display on the display device, and a pointing device under the control of an application program executing on the computer system. A device for controlling a plurality of multimedia devices to process multimedia data, the device being controlled by the application program to create a multimedia object in association with each of the plurality of multimedia devices. Wherein the object has at least one object method for controlling the associated multimedia device and a method for controlling the graphic user interface on the display device. Means for generating an icon display of the object on the display, and the graphic user in cooperation with the pointing device.
Connection means for controlling the interface and visibly connecting the first and second icons, and the first icon representing the first multimedia object and the second icon representing the second multimedia object. Responsive to connecting an icon, invoking at least one object method for each of the first and second multimedia objects to associate multimedia data with the first multimedia object. Means for directing from a first multimedia device to a second multimedia device associated with said second multimedia object. 11. At least one of the first and second multimedia devices has device parameters for controlling processing of the multimedia data by the at least one multimedia device, and further comprising: Means for opening one of the plurality of multimedia objects controlled by a device, and first setting the device parameter in response to opening one of the multimedia objects Means for creating a control object having a second control object method for displaying a control object method and a control icon, and a means for invoking the control object method controlled by the indicating device to set the device parameter. 11. The device of claim 10 including. 12. The means for creating comprises: means for opening a multimedia object controlled by the pointing device and associated with the at least one multimedia device; and responsive to the multimedia object, Dialog on Graphic User Interface
Means for creating a box object, means for creating a control object and invoking a first control object method for displaying a control icon in response to the creation of the dialog box object; In response to an operation, controlled by the pointing device to call a second control object method,
11. A means for setting the device parameters.
An apparatus according to claim 1.